prevent unexpected post-processing & simplify internal feed.

[kivikakk]

Some minor fixes and cleanups, and the main course: prevent unexpected post-processing when elements are escaped.

This brought to my attention by - \[x] hello with tasklist (only) enabled raising an assertion: it's still trying to parse it as a tasklist, and the tasklist code asserts the text element's sourcepos start matches the containing paragraph's. In this case, it doesn't: the escape isn't included, so it throws for a mismatch.

The solution here seems to be to let the Escaped node actually always enter the document tree, preventing this from getting any post-processing treatment at all. We remove them after Text node unification and post-processing (unless the user wants them left in).

[github-actions]

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
./bench.sh ./comrak-834aa08	109.2 ± 1.3	107.2	112.7	1.84 ± 0.05
./bench.sh ./comrak-main	107.4 ± 1.0	105.8	110.8	1.81 ± 0.04
./bench.sh ./pulldown-cmark	59.3 ± 1.3	58.0	66.2	1.00
./bench.sh ./cmark-gfm	94.1 ± 6.3	83.5	106.2	1.59 ± 0.11
./bench.sh ./markdown-it	262.7 ± 4.3	257.1	280.8	4.43 ± 0.12

Run on Sun Nov 2 05:45:55 UTC 2025

[kivikakk]

Cursed. The following Markdown:

[email protected]_

formats to CommonMark as follows:

a.b-c\[email protected]\_

The status quo is that this is post-processed the same as the original input; one big text node with the escapes parsed out/removed:

• document
  • paragraph
    • text [email protected]_

With autolinks turned on, these aren't autolinked; the trailing underscore disqualifies it (I guess).

With this PR, the roundtripped CommonMark parses as:

• document
  • paragraph
    • text a.b-c
    • escaped
      • text _
    • text [email protected]
    • escaped
      • text _

This does get parsed as an autolink, and so we fail a roundtrip test. Grr.

I'm kind of inclined to think this parse is actually correct (although different to the status quo), and it's the formatter that's changing the meaning here and should be corrected. But that's a Hard problem, as it's difficult to fully generally determine when a character needs to be escaped to be reinterpreted the same way, seeing as preceding and following characters will influence its interpretation.

[kivikakk]

This change is necessary for escaped_char_spans to not bifurcate the interpretation of post-processed extensions in Comrak's eyes anyways: those Escaped elements being inserted only when the option is turned on visibly influence the interpretation. So we have to do it unilaterally, and remove the elements from the AST if not wanted, if we don't want it to actually change the meaning of the document when turned on.

The alternative is to track such characters without modifying the tree (e.g. vec of character indices), but that becomes hard to maintain. Hrmmmm.

[kivikakk]

This does get parsed as an autolink, and so we fail a roundtrip test. Grr.

I've decided this parse is correct: without the backslash, the _ is part of the surrounding text and the autolink rules say the underscore disqualifies it:

., -, and _ can occur on both sides of the @, but only . may occur at the end of the email address, in which case it will not be considered part of the address:

With the backslash, (a) it's not a _ at the end, it's a backslash, and (b) even if we say it's not a backslash, it's not a _, it's an escaped _. The purpose of escaping is to prevent the usual interpretation.

[kivikakk]

It occurs to me that the answer here is to not keep the autolinks extension turned on while doing roundtrip tests: any GFM extension autolinks recognised in the original text will be written out as core spec links (of some variety). Doing further extension autolink parsing on the resulting roundtripped text may well not be stable, because we don't roundtrip GFM extension autolinks to begin with!