(Inline) footnote recursion stability and misc cleanups.

CHANGELOG.md

@@ -1,5 +1,6 @@
 Categories to use in this document, and the order in which to give them:
 
+* Security
 * Reverts
 * Parser changes
 * Changed APIs
@@ -13,6 +14,34 @@ Categories to use in this document, and the order in which to give them:
 * Behind the scenes
 
 
+# unreleased
+
+Please note the MSRV has been bumped from 1.65 to 1.70; see [the pull request](https://github.com/kivikakk/comrak/pull/649) for more details. It's a kind of sticky and awkward situation — thanks to the inevitability of Progress — with no particularly clean solution. (wherein telling GCC 15 users "sorry it just won't build from source for you without messing with dependencies" is not a solution.)
+
+Security:
+
+* Footnote resolution no longer recurses over the document tree; on documents with deeply nested elements, this could cause a stack overflow, with resultant denial of service.
+* Inline footnotes are restricted to a depth of 5 for similar reasons. An iterative rewrite here to avoid a limit is possible, but for now I'm hoping we can all pretend to be responsible adult human beings and limit our recursive inline footnote usage accordingly. (PRs welcome tho, non-human users are very welcome!)
+
+Parser changes:
+
+* U+2069 POP DIRECTIONAL ISOLATE will be treated as terminating an autolink, rather than included as part of the link, making autolinks much easier to use correctly in RTL text. (by @SethFalco in https://github.com/kivikakk/comrak/pull/654)
+* HTML start condition 4 is correctly detected when non-capital letters follow "<!". (by @kivikakk in https://github.com/kivikakk/comrak/pull/658)
+
+New APIs:
+
+* Discord-style subtext support is added as the `subtext` extension. (by @Kuuuube in https://github.com/kivikakk/comrak/pull/648, https://github.com/kivikakk/comrak/pull/650)
+
+Bug fixes:
+
+* Source position information is corrected for description lists, HTML blocks, multiline block quotes, links with newlines following the destination, tables with leading indentation, and escaped character spans. (by @Martin005 in https://github.com/kivikakk/comrak/pull/646, https://github.com/kivikakk/comrak/pull/651, https://github.com/kivikakk/comrak/pull/652, https://github.com/kivikakk/comrak/pull/653, https://github.com/kivikakk/comrak/pull/656, https://github.com/kivikakk/comrak/pull/657)
+* `escaped_char_span` users can now successfully format to CommonMark with debug assertions enabled. These ASTs previously did not validate, which currently is enabled experimentally only in CommonMark output in debug.
+
+Build changes:
+
+* Comrak's MSRV is bumped from 1.65 to 1.70. (by @kivikakk in https://github.com/kivikakk/comrak/pull/649)
+
+
 # [v0.45.0] - 2025-10-23
 
 Welcome to v0.45.0! This is a big update, much of them part of from [rc.1 from last week](https://github.com/kivikakk/comrak/releases/tag/v0.45.0-rc.1).  More context on the size of the update in the changelog there.

Cargo.toml

@@ -51,6 +51,7 @@ percent-encoding-rfc3986 = "0.1.3"
 strum = { version = "0.26.3", features = ["derive"] }
 toml = "0.7.3"
 slug = "0.1.4"
+pretty_assertions = "1.4.1"
 
 [build-dependencies]
 entities = "1.0.1"

src/nodes.rs

@@ -875,19 +875,14 @@ impl<'a> arena_tree::Node<'a, RefCell<Ast>> {
             | NodeValue::TaskItem(..) => {
                 child.block() && !matches!(*child, NodeValue::Item(..) | NodeValue::TaskItem(..))
             }
-
             NodeValue::List(..) => matches!(*child, NodeValue::Item(..) | NodeValue::TaskItem(..)),
-
             NodeValue::DescriptionList => matches!(*child, NodeValue::DescriptionItem(_)),
-
             NodeValue::DescriptionItem(_) => matches!(
                 *child,
                 NodeValue::DescriptionTerm | NodeValue::DescriptionDetails
             ),
-
             #[cfg(feature = "shortcodes")]
             NodeValue::ShortCode(..) => !child.block(),
-
             NodeValue::Paragraph
             | NodeValue::Heading(..)
             | NodeValue::Emph
@@ -906,11 +901,8 @@ impl<'a> arena_tree::Node<'a, RefCell<Ast>> {
             // inlines.
             | NodeValue::EscapedTag(_)
             => !child.block(),
-
             NodeValue::Table(..) => matches!(*child, NodeValue::TableRow(..)),
-
             NodeValue::TableRow(..) => matches!(*child, NodeValue::TableCell),
-
             #[cfg(not(feature = "shortcodes"))]
             NodeValue::TableCell => matches!(
                 *child,
@@ -931,7 +923,6 @@ impl<'a> arena_tree::Node<'a, RefCell<Ast>> {
                     | NodeValue::Subscript
                     | NodeValue::TaskItem(_)
             ),
-
             #[cfg(feature = "shortcodes")]
             NodeValue::TableCell => matches!(
                 *child,
@@ -953,15 +944,27 @@ impl<'a> arena_tree::Node<'a, RefCell<Ast>> {
                 | NodeValue::ShortCode(..)
                 | NodeValue::TaskItem(_)
             ),
-
             NodeValue::MultilineBlockQuote(_) => {
                 child.block() && !matches!(*child, NodeValue::Item(..) | NodeValue::TaskItem(..))
             }
-
             NodeValue::Alert(_) => {
                 child.block() && !matches!(*child, NodeValue::Item(..) | NodeValue::TaskItem(..))
             }
-            _ => false,
+
+            NodeValue::Escaped => matches!(*child, NodeValue::Text(_)),
+
+            NodeValue::FrontMatter(_)
+            | NodeValue::CodeBlock(_)
+            | NodeValue::HtmlBlock(_)
+            | NodeValue::ThematicBreak
+            | NodeValue::Text(_)
+            | NodeValue::SoftBreak
+            | NodeValue::LineBreak
+            | NodeValue::Code(_)
+            | NodeValue::HtmlInline(_)
+            | NodeValue::Raw(_)
+            | NodeValue::FootnoteReference(_)
+            | NodeValue::Math(_) => false,
         }
     }
 

src/parser/inlines.rs

@@ -1,7 +1,7 @@
 mod cjk;
 
 use std::borrow::Cow;
-use std::cell::{Cell, RefCell};
+use std::cell::Cell;
 use std::collections::HashMap;
 use std::convert::TryFrom;
 use std::str;
@@ -25,6 +25,7 @@ use crate::strings::{self, count_newlines, is_blank, Case};
 const MAXBACKTICKS: usize = 80;
 const MAX_LINK_LABEL_LENGTH: usize = 1000;
 const MAX_MATH_DOLLARS: usize = 2;
+const MAX_INLINE_FOOTNOTE_DEPTH: usize = 5;
 
 pub struct Subject<'a: 'd, 'r, 'o, 'd, 'c, 'p> {
     pub arena: &'a Arena<AstNode<'a>>,
@@ -34,9 +35,10 @@ pub struct Subject<'a: 'd, 'r, 'o, 'd, 'c, 'p> {
     pub pos: usize,
     column_offset: isize,
     line_offset: usize,
+    inline_footnote_depth: usize,
     flags: HtmlSkipFlags,
     pub refmap: &'r mut RefMap,
-    footnote_defs: &'p FootnoteDefs<'a>,
+    footnote_defs: &'p mut FootnoteDefs<'a>,
     delimiter_arena: &'d Arena<Delimiter<'a, 'd>>,
     last_delimiter: Option<&'d Delimiter<'a, 'd>>,
     brackets: Vec<Bracket<'a>>,
@@ -64,8 +66,9 @@ impl<'a, 'r, 'o, 'd, 'c, 'p> Subject<'a, 'r, 'o, 'd, 'c, 'p> {
         input: String,
         line: usize,
         refmap: &'r mut RefMap,
-        footnote_defs: &'p FootnoteDefs<'a>,
+        footnote_defs: &'p mut FootnoteDefs<'a>,
         delimiter_arena: &'d Arena<Delimiter<'a, 'd>>,
+        inline_footnote_depth: usize,
     ) -> Self {
         let mut s = Subject {
             arena,
@@ -75,6 +78,7 @@ impl<'a, 'r, 'o, 'd, 'c, 'p> Subject<'a, 'r, 'o, 'd, 'c, 'p> {
             pos: 0,
             column_offset: 0,
             line_offset: 0,
+            inline_footnote_depth,
             flags: HtmlSkipFlags::default(),
             refmap,
             footnote_defs,
@@ -271,6 +275,7 @@ impl<'a, 'r, 'o, 'd, 'c, 'p> Subject<'a, 'r, 'o, 'd, 'c, 'p> {
                 // Check for inline footnote first
                 if self.options.extension.footnotes
                     && self.options.extension.inline_footnotes
+                    && self.inline_footnote_depth < MAX_INLINE_FOOTNOTE_DEPTH
                     && self.peek_char_n(1) == Some(&(b'['))
                 {
                     self.handle_inline_footnote()
@@ -961,6 +966,7 @@ impl<'a, 'r, 'o, 'd, 'c, 'p> Subject<'a, 'r, 'o, 'd, 'c, 'p> {
             self.refmap,
             self.footnote_defs,
             &delimiter_arena,
+            self.inline_footnote_depth + 1,
         );
 
         while subj.parse_inline(para_node, &mut para_node.data_mut()) {}
@@ -2133,30 +2139,30 @@ impl RefMap {
 }
 
 pub struct FootnoteDefs<'a> {
-    defs: RefCell<Vec<Node<'a>>>,
-    counter: RefCell<usize>,
+    defs: Vec<Node<'a>>,
+    next: usize,
 }
 
 impl<'a> FootnoteDefs<'a> {
     pub fn new() -> Self {
         Self {
-            defs: RefCell::new(Vec::new()),
-            counter: RefCell::new(0),
+            defs: Vec::new(),
+            next: 1,
         }
     }
 
-    pub fn next_name(&self) -> String {
-        let mut counter = self.counter.borrow_mut();
-        *counter += 1;
-        format!("__inline_{}", *counter)
+    pub fn next_name(&mut self) -> String {
+        let counter = self.next;
+        self.next += 1;
+        format!("__inline_{}", counter)
     }
 
-    pub fn add_definition(&self, def: Node<'a>) {
-        self.defs.borrow_mut().push(def);
+    pub fn add_definition(&mut self, def: Node<'a>) {
+        self.defs.push(def);
     }
 
-    pub fn definitions(&self) -> std::cell::Ref<'_, Vec<Node<'a>>> {
-        self.defs.borrow()
+    pub fn take(&mut self) -> Vec<Node<'a>> {
+        mem::take(&mut self.defs)
     }
 }