Fix quadratic rendering time for many inline links - #1621
Conversation
Scanning from index 0 after every inline-pattern match rescanned the unprocessed text repeatedly, making conversion quadratic in the number of inline elements. Return the index just past the inserted placeholder so the next scan starts at the unprocessed tail. Fixes Python-Markdown#1619.
There was a problem hiding this comment.
This changes the API for all inline patterns and processors. @facelessuser I think you have a better handle on this part of the code. Any input on that change here?
There was a problem hiding this comment.
Thanks. I kept the search-index behavior change and documented it under Unreleased > Changed in b745bba, treating it as an inline-processor API/behavior change rather than a Fixed-only entry.
There was a problem hiding this comment.
I do have some mild concerns about this change. We are assuming that because the API changed here without breaking things, others using this API won't be broken.
When we changed the API here in the first time, we did so in a way that was non-breaking, one you could opt into. This was done by providing a different class that could be checked.
It is possible that we may need to employ some method ot indicate this is a different version of the new way. I really need to do some testing to understand the implications of the change. I will have to do some testing with this over in Pymdown Extensions, where we have a number of plugins using the new style, so I can get a better idea of what the impact here is.
There was a problem hiding this comment.
It seems that all the tests over on Pymdown Extensions passed with these changes, so that is a good sign.
|
Thanks for the detailed review. Commit b745bba addresses the points raised: it removes the wall-clock regression test, replaces it with a deterministic search-offset assertion, and moves #1619 under Unreleased > Changed to document the inline-processor behavior/API impact. The full unittest suite passes with 1,089 tests and 13 skipped; changed-file flake8 and git diff --check pass. |
|
Thank you for addressing my concerns. I am going to let @facelssuser review this as he has a better grasp of the affected code. |
|
Thanks, @waylan. I appreciate the follow-up; I’ll wait for @facelssuser’s review of the affected code. |
|
I will hopefully get to this sometime this week. I've become somewhat exhausted with the onslaught of everyone pointing AI at every project on the internet. I'm already dealing with similar issues on numerous projects I support. |
|
Understood. I will leave the draft unchanged while you assess it; no further follow-up is needed from me right now. |
Description
Rendering a paragraph containing many inline links took quadratic time:
each match replaced the link with a placeholder and then rescanned the
whole remaining text from index 0, so N links cost O(N²) pattern work.
'[link](x)' * 8192took ~7s before this change and ~0.3s after.__applyPatternnow returns the index just past the inserted placeholderso the next scan starts at the unprocessed tail.
Fixes #1619.
AI Assistance Disclosure
Claude (via Command Code) assisted with implementation and testing; the change was fully reviewed and verified by a human.
Checklist