Repository navigation
Conversation
…next scanned scalar The scanner resolved a plain scalar under whatever tag the parser had most recently seen (ScanOptions.tag). The tag was passed to more scans than the one producing the tagged node's content, so it leaked into unrelated plain scalars: after a tagged block-mapping value, into the next entry's key (`a: !!str b` followed by `0x10: x` gave the key "0x10" instead of 16, `~: x` gave "~" instead of null); after a tag on the line before a block mapping, into that mapping's first key. parse_block_indented worked around one instance of this by rewinding and re-scanning an abandoned sibling. The scanner now records both the core-schema value and the text of a plain scalar, and parse_node's Scalar arm picks one once it knows which node the collected properties belong to (a tag on an earlier line than an implicit key is the block mapping's, mirroring the anchor rule). This removes ScanOptions.tag, the rewind, and ScalarStyle::Plain's multiline flag.
|
Warning Review limit reached
Next review available in: 19 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (2)
Comment |
|
Reproduced on bun 1.4.0 with Fix: #38147 (this PR). Tests: test/js/bun/yaml/yaml.test.ts, six new tests fail before and pass after; yaml-test-suite and the block scalar matrix stay green. |
|
Updated 4:45 AM PT - Aug 13th, 2026
✅ @robobun, your commit 76781377c01710e761e2144e00f78f934446cfe6 passed in 🧪 To try this PR locally: bunx bun-pr 38147That installs a local version of the PR into your bun-38147 --bun |
There was a problem hiding this comment.
I re-reviewed after f00e679 and 7678137 and didn't find any bugs — the flow-sequence-pair nit from my earlier pass is now pinned by a test and explained in the description, and the comment-cop feedback is addressed. Because this restructures the scanner/parser boundary (TokenScalar now carries text+resolved, ScanOptions.tag and the parse_block_indented rewind are gone), a human look is still worthwhile.
What was reviewed:
NodeTag::keepsis the same table that was inlined inScalarResolverCtx::resolve/try_resolve_number; the new resolution-table test pins it.YamlString::to_expris a straight move of the oldNodeScalar::Stringarm, arena-copy ofListbytes preserved.- All
TokenScalar/ScalarInitconstruction sites (plain, block, both quoted paths) updated; block/quoted setresolved: Nonesoto_exprfalls through to text as before. ScalarStyle::Plain { multiline }had no reader left after the rewind deletion.
Extended reasoning...
Overview
The PR moves YAML tag application from scan-time to parse-time in src/parsers/yaml.rs. TokenScalar is restructured from { data: NodeScalar<Enc>, style } to { text: YamlString<Enc>, resolved: Option<NodeScalar>, style }; NodeScalar drops its String variant and its Enc parameter; ScanOptions.tag, ScalarResolverCtx.{tag,multiline}, ScalarStyle::Plain { multiline }, and the rewind-and-rescan block in parse_block_indented are deleted. parse_node's Scalar arm now applies the tag via TokenScalar::to_expr → NodeTag::keeps, with a line-based split for the [200] implicit-key case mirroring take_implicit_key_anchors. Roughly 200 lines of Rust net, plus ~100 lines of new tests covering the variant matrix (tag kind × scalar style × key position × dedent) and a table test pinning the unchanged resolution behavior.
Security risks
None. This is pure parser logic on already-tokenized input; no new untrusted-length arithmetic, no new allocation sizing, no FFI boundary changes. The one unsafe block (from_raw_parts in YamlString::to_expr) is moved verbatim from the old NodeScalar::to_expr String arm with the same SAFETY invariant (Enc::KIND == Utf16 ⇒ Enc::Unit == u16).
Level of scrutiny
Moderate-to-high. Bun.YAML.parse feeds the bundler's config loading and the pnpm-lock migration path, so a silent misparse would propagate. The change is architecturally the right fix per REVIEW.md ("fix at the layer that owns the invariant" — only the parser knows which node a tag belongs to), and the author reports yaml-test-suite.test.ts, yaml-block-scalar-matrix.test.ts, resolve/yaml, bundler yaml, and pnpm-lock migration tests all pass. But the data-type restructure touches every scalar-token construction and consumption site, which is more than a localized bugfix.
Other factors
My earlier nit (the key_tag line-split reaching the flow-sequence-pair path and changing [!!str\n0x10: a] on invalid input) was answered with a coherent rationale — tags now follow the same rule anchors already followed there, and the pair mapping is what begins at the property so it is the natural target — and pinned by the new "property on the line before a flow-sequence pair's key" test. The comment-cop bot's six long-comment flags were resolved by 7678137 (comments now one line each). All inline threads are resolved. Given the scope of the restructure I'm deferring rather than approving so a human can confirm the yaml-test-suite CI result and sanity-check the TokenScalar shape change.
|
Note for whoever lands second: #44630 adds a guard to the radix branch of |
|
Closing as stale: this PR conflicts with main, and its last commit is from 2026-08-13. The branch stays, so the work is not lost. To pick it up again, rebase the branch on main and reopen this PR. |
Problem
Bun.YAML.parse("a: !!str b\n0x10: x")gives{ a: "b", "0x10": "x" }; the key should be16, as it is without the tag.Bun.YAML.parse("a: !!str b\n~: x")gives{ a: "b", "~": "x" }; the key should benull.Bun.YAML.parse("x: !!map\n 0x10: a")gives{ x: { "0x10": "a" } }; the key should be16.!!str,!,!local,!<verbatim>, and!!int/!!float/!!bool/!!null, which block keys of any other kind), after a tagged value of any scalar style (plain, quoted, block), and wherever the next key lives (same mapping,?key,:value, compact mapping in a sequence item, or an outer mapping reached by dedenting). js-yaml and theyamlpackage resolve the key independently in all of these.ScanOptions.tagat scan time (src/parsers/yaml.rs,ScalarResolverCtx::resolve/try_resolve_number). The parser passed the tag to more scans than the one that produces the tagged node's content:parse_node'sScalararm passed it to the scan that steps past the tagged scalar (whose result is the next key), and theTagarm /parse_block_indentedpass it to the scan after a tag even when that token turns out to be a block mapping's first key rather than the tagged node.parse_block_indentedalready carried a rewind-and-rescan workaround for a third instance (a: !!str\n0x10: x).Fix
TokenScalar { text, resolved, style }).parse_node'sScalararm is the only consumer of scalar tokens; it applies the node's tag there viaTokenScalar::to_expr, using the same keep-or-string table the scanner used (NodeTag::keeps). For an implicit key it only applies a tag that sits on the key's line; a tag on an earlier line belongs to the block mapping (YAML [200]), which is the ruletake_implicit_key_anchorsalready applies to anchors.keepsis the table that was inresolve/try_resolve_number, and the new table test pins it.ScanOptions.tag, the rewind inparse_block_indented, andScalarStyle::Plain { multiline }(only the rewind read it).Tokengrows from 56 to 72 bytes; the parser holds one token at a time, so this is a per-token copy, not a buffer.[!!str\n0x10: a], now gives[{ 16: "a" }]instead of[{ "0x10": "a" }], because the tag goes where an anchor in that position already went ([&a\n0x10: 1, *a]aliases the pair mapping, before and after). js-yaml and theyamlpackage reject that input; the valid[!!str 0x10: a]is unchanged. Pinned by the "flow-sequence pair" test.cargo clippy -p bun_parsersandcargo fmtare clean.Background
0x10is 16,~is null,trueis a boolean, anything else is a string). This applies to keys and values alike.!!str,!!int,!foo, ... annotate the single node they precede. In this parser a tag of a different kind does not validate the text, it only switches the core-schema resolution off (!!str 0x10is the string "0x10",!!int foois the string "foo"); that behavior is unchanged here.!!map\n0x10: aor&m\n0x10: a, the property on the earlier line annotates the mapping, while in!!str 0x10: ait annotates the key. The parser cannot tell which until it has scanned the key and seen the:, which is why the decision has to live in the parser rather than the scanner.scan()produces one token at a time intoself.token;parse_nodecollects a node's properties (anchor, tag), dispatches on the content token, and for a scalar scans once more to see whether a:follows (implicit key) before returning.parse_block_indentedis the shared property loop for the token after:,?, or-; when the token after the properties belongs to the parent, the properties annotate an empty node.