Repository navigation
Conversation
|
Updated 8:46 AM PT - Oct 26th, 2026
✅ @robobun, your commit 1820aed1e8b6e5e37b201b09bb51d8fa6a15c62f passed in 🧪 To try this PR locally: bunx bun-pr 45172That installs a local version of the PR into your bun-45172 --bun |
|
Status Reproduced on 1.4.3-canary (367d939) and on main (d1cb7db). Reproduced again on main 449877f, the base of this branch since the rebase of 2026-10-26. On a debug build of that main the 3 new tests fail: The fix is this PR, #45172. |
|
No actionable comments were generated in the recent review.
|
| Layer / File(s) | Summary |
|---|---|
Decode strings and verify compiler behavior src/react_compiler/optimization/constant_propagation.rs, test/bundler/transpiler/react-compiler.test.ts |
push_units converts 8-bit string data from UTF-8 to UTF-16; the UTF-16-backed path is unchanged. CLI and API tests cover concatenation, templates, assignments, comparisons, branch evaluation, and invalid bytes. A lint test checks diagnostics for constant conditions formed from non-ASCII and ASCII strings. |
Suggested reviewers: jarred-sumner
Priority:
Merge Risk: 
Minimal · up to 1820a
-
No actionable merge-blocking issue is established; the change is mergeable after normal checks.
Pre-merge checks: 
4
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.
Comment @coderabbitai help to get the list of available commands.
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked whether sizing the output buffer to bytes.len() u16 slots can trip the panic in convert_utf8_to_utf16_in_buffer (src/bun_core/string/immutable.rs:2501): it cannot, since both the simdutf path and the WTF-8 fallback consume at least one input byte per emitted code unit (4 bytes for a surrogate pair), so the output never exceeds the input byte count. The Latin-1 .env to U+FFFD change and the unfolded template-literal join were also examined and are consistent with the PR's stated scope.
Extended reasoning...
The change replaces byte-widening in push_units (src/react_compiler/optimization/constant_propagation.rs) with a WTF-8 to UTF-16 transcode into a pre-resized Vec slot, plus itBundled and bun lint tests. It touches no security-sensitive surface; the only hazard examined was the panicking converter, whose buffer bound holds here. Two confirmed sibling encoding-blind folds (== against numbers, .length on invalid UTF-8) remain as inline findings, so a human should weigh whether they belong in this PR.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🟣
src/react_compiler/optimization/constant_propagation.rs— Users who compare a folded non-ASCII string to a number with == get a wrong constant and a pruned branch, the same encoding-blind string folding this PR fixes one arm of. At constant_propagation.rs:1077 every UTF-16 string yields NaN via as_bytes().map(js_to_number).unwrap_or(NaN), so (" " + ONE) == 1 with --define ONE="1" folds to false while JavaScript gives true. Fix: run ToNumber on the UTF-16 code units (trim JS whitespace, parse) or decline the fold when the string is UTF-16, so the arm never asserts a wrong equality for a non-ASCII operand.Why this was flagged
Trigger: a constant string with any non-ASCII content (a lexer literal, or the result of a join that push_units now produces correctly) compared with == or != to a number constant, reached through js_abstract_equal from the Equal/NotEqual folding in evaluate_binary_op. constant_propagation.rs:1077 maps a UTF-16 string straight to NaN and :1079-1083 then folds the comparison to false, which the compiler uses to prune the branch; JS ToNumber of " 1" is 1, so the user's taken branch is deleted. The base behaves the same, but REVIEW.md requires fixing the whole class in the same PR: this arm is in the same function family, has the same root cause (treating UTF-16 storage as non-numeric and byte-level checks without encoding branching), and the PR author lists it only as 'Not fixed here'. Remedy: decode UTF-16 to a numeric parse (or return None for UTF-16 strings) in the string/number arm.
Verification: Pre-existing; acknowledged in the PR description's "Not fixed here" section. At src/react_compiler/optimization/constant_propagation.rs:1077
s.as_bytes().map(js_to_number).unwrap_or(f64::NAN)yields NaN for any UTF-16 string, so==to a number constant folds tofalse. Lines 1056-1096 are untouched by this diff; on the base commit the same inputs fail by the same route. -
🟣
src/react_compiler/optimization/constant_propagation.rs— A component reading .length of an inlined env or define value containing bytes that are not valid UTF-8 gets a folded length that disagrees with the string the compiler itself folds for the same value. At constant_propagation.rs:523 len_utf16 (js_string.rs:85) uses simdutf's utf16-from-utf8 length for 8-bit non-ASCII data, which counts stray continuation bytes as 0 units, while push_units at :817 turns each such byte into one U+FFFD unit. Fix: compute .length by the same WTF-8 decoding used for the join (count units produced by convert_utf8_to_utf16_in_buffer or an equivalent fallback-aware counter) so length and content folds agree.Why this was flagged
Trigger: --env=inline with a value such as three 0x80 bytes (test row Utf16ThenThreeBadBytes at react-compiler.test.ts:3633 shows the join folds to 4e2d fffd fffd fffd), and a component that reads V.length or branches on it. js_string.rs:81-85 returns element_length_utf8_into_utf16, which with USE_SIMDUTF (unicode.rs:1123-1124) is simdutf::length::utf16::from::utf8, giving 0 for continuation-only input, so V.length folds to 0 while ("x" + V).length folds to 4 via the UTF-16 join. The PR adds the U+FFFD convention to the UTF-16 join without making the sibling length fold use the same decoder, so the two constants the compiler emits for one value contradict each other. Pre-existing on base for the 8-bit path, but this PR extends the U+FFFD rule to a second site and REVIEW.md says never compare byte counts to code-unit counts across a conversion. Remedy: derive the length from the same WTF-8 decode that produces the units.
Verification: Pre-existing: base already folds
V.lengththe same wrong way by the same route. Trigger:--env=inlinewhose value contains stray continuation bytes such as\x80\x80\x80, and a component under--react-compilerthat readsV.length. constant_propagation.rs:521-523 folds.lengthwiths.len_utf16(), which counts continuation bytes as 0 units. So for\x80\x80\x80,V.lengthfolds to 0, while the realV.lengthis 3.
|
Both findings are in code that this PR does not change, and both fold the same way on main. I checked each on release builds of main and of this branch.
Confirmed, and it is wider than reported. For an inlined env value:
The length fold, the join and the printer each decode such bytes in their own way. The numbers are the same on main. A decode where the value is made settles it for every reader, and #42019 does that for define and env values. I added this to "Not fixed here". |
|
#46915 has the same |
… it to a UTF-16 string `push_units` widened each byte of an 8-bit string to one UTF-16 code unit. An 8-bit string holds UTF-8, so a character that is not ASCII became one code unit per byte: `"中" + K` with the define `K="é"` folded to "中é". `bun lint` gives the compiler every string literal as UTF-8 bytes, so `"é" + "é" + "é" === "ééé"` folded to false there. The 8-bit arm now decodes the bytes into the tail of the output buffer, with the transcoder that `JsString::from_wtf8_bytes` uses for a join of two 8-bit strings. Half of a surrogate pair stays as it is, and a byte that is not UTF-8 is U+FFFD in both joins.
7e13ddc to
1820aed
Compare
There was a problem hiding this comment.
LGTM — a small, self-contained fix with a thorough test matrix.
What was reviewed:
push_unitsinsrc/react_compiler/optimization/constant_propagation.rs: checked thatbytes.len()u16 slots always suffice for the panickingconvert_utf8_to_utf16_in_buffer— the WTF-8 fallback only emits two units after a 4-byte decode whose continuation bytes all validated, and every other path emits at most one unit per byte, so the buffer cannot be too small;resize/truncateuses the returned slice length, leaving no stale zeros.- Sibling sites: no other
b as u16widening of 8-bitEStringdata remains insrc/react_compiler; the new arm matches the shapeJsString::from_wtf8_bytesalready uses, so both join paths now transcode identically. - Tests:
backend,define,dotenv,env, and Buffer-valuedfilesare all supported byexpectBundled; the lint test drains pipes viaPromise.all, asserts stdout before the exit code, and cleans up withusing.
Extended reasoning...
The diff replaces a byte-to-code-unit cast in the React Compiler constant-propagation string-join helper with the in-tree UTF-8 to UTF-16 transcoder (8 lines of Rust) and adds a parameterized bundler test plus a bun lint test covering accented, astral, split-surrogate, env-sourced invalid-byte, module.filename, .length, ===, and branch-pruning shapes. It touches no security-sensitive surface (no auth, crypto, permissions, or path handling) and no CODEOWNERS-covered path. The panicking transcoder variant is provably safe because output units can never exceed input bytes, the change mirrors the existing from_wtf8_bytes pattern, and a grep found no remaining sibling sites of the same bug class in the module. The bug hunt ran to dry_streak with no findings, and the new commit since the earlier bot comment is the rebased version of the same change.
…sign target, more tests - `fold_constant_index` takes its target by reference, so the longest `a["b"]["b"]...` chain is main's again. - `order_utf16` is the code of main again. `checked_order_utf16` compares two ASCII texts as bytes and decodes other texts from the start. - `.length` of a string is not folded where it is assigned to, updated or deleted. - `push_units` of the React Compiler is the code of main again: #45172 owns that join. - The checked iterator is private to the string module, and the note on `init_re_encode_utf8` stays. - Tests: the bounds of a folded index, the order and key folds under `bun run`, long texts in the seeded differential, and the folds with no minify flag beside the other define folds. - bench/snippets/transpiler-define-folds.mjs times the folds by kind of text and length.
Problem
bun build --react-compiler --define 'K="é"'compiles"中" + Kto"中é"and folds=== "中é"on it tofalse.bun lintfolds"é" + "é" + "é" === "ééé"tofalseand misses an unconditionalsetState.push_units(src/react_compiler/optimization/constant_propagation.rs:812) widens each byte of an 8-bit string to a code unit. 8-bit strings hold UTF-8.Fix
push_unitsdecodes the 8-bit operand into the output buffer (strings::convert_utf8_to_utf16_in_buffer). It adds no allocation.JsString::from_wtf8_bytesuses the same transcoder for two 8-bit strings. Half of a surrogate pair stays as it is.test/bundler/transpiler/react-compiler.test.ts(CLI,Bun.build,bun lint) fail on main. Both fixture suites pass.todotest row (CI never runs it). Not taken: the lint test intest/cli/lint(that file timed out on debug builds).Background
EString(the AST string) is 8-bit or UTF-16. The lexer makes non-ASCII literals UTF-16.definevalues, inlined env values andbun lintliterals stay 8-bit UTF-8.a + bof constant strings. With a UTF-16 operand,push_unitscopies both into UTF-16.bun lintskips them).Downsides
"中" + "x"costs 171 more instructions per fold (100 to 271). 64 ASCII bytes cost 128 fewer.\xE9) folds to U+FFFD, not U+00E9.push_unitsgrows 121 bytes..textis unchanged.Notes
How it was found, and who it reaches
thisfor template tags #32297 names it: "It has no PR yet"). There is no user report and no issue.--react-compileris experimental and off by default.push_unitsis wrong in 1.4.0 to 1.4.4.bun lintis not in a release yet.definevalue, an inlinedprocess.envvalue (--env=inline),module.filenamein a file with a non-ASCII name, or any literal underbun lint(src/lint/react_compiler/convert.rs:914).K + K + Kis wrong too:K + Kis UTF-16.+=, and read in a nested function. Also.lengthof the join (3, not 2),===against the right literal, theifpruned on it, and a pair of surrogate halves split across the two strings.Not fixed here
==fold of a string and a number (constant_propagation.rs:1071). It takes every UTF-16 string for NaN, andjs_to_numberuses Rust'strimandparse. With--define 'ONE="1"',("\u00a0" + ONE) == 1folds tofalse(JavaScript:true)."infinity" == 1 / 0folds totrue(JavaScript:false). Two fixes are possible, and each changes folds that are right today. Reading a UTF-16 string throughjs_to_numbergives it the wrong folds ofjs_to_number. Not folding a UTF-16 string against a number drops the fold of"é" != 1in a build, and underbun lintthe report of asetStatecall belowif ("é" + "é" != 1)..lengthof an 8-bit string that is not valid UTF-8 (diagnostics/js_string.rs:85). For an inlined env value of the bytes80 80 80,V.lengthfolds to 0, a folded join has 3 units for it, and the printed string has length 3 at run time. ForF0the three give 2, 1 and 1. ForE4 B8they give 1, 2 and 1. The length fold, the join and the printer each decode such bytes in their own way, on main too. Store non-ASCII define, env, macro and inlined path strings as UTF-16 so they fold like literals #42019 re-encodes define and env values where they are made.:304,:343), the template substitution (:593) and the fbt lookup. The value stays right and the fold is missed. Underbun lint, thesetStateunder`x${"é" + "é"}` === "xéé"is not reported.--minify-syntaxand no React compiler,K[0]is"\x00"and(K + "b").lengthis 3. See Store non-ASCII define, env, macro and inlined path strings as UTF-16 so they fold like literals #42019, js_parser: do not fold .length of a folded string with a non-ascii segment #38989 and js_parser: fold +, - and ~ of a constant-folded string; stop folding non-ascii strings to NaN #39072."pre" + `fix/${props.x}`printspreXwith--react-compiler --minify-syntax. See react_compiler: keep every folded segment of template heads, tails and computed keys #38993.JsStringhashes its storage and compares its content. Nothing calls the hash.Measurements (release builds of main d1cb7db without and with this diff, linux x64. Not repeated after the rebase onto main 449877f: the source hunk and
convert_utf8_to_utf16_in_bufferdid not change)Allocations per folded join with a UTF-16 operand: 1 on main and 1 here (the
Vec::with_capacityof the arm), 0 regrows. A join of two 8-bit strings does not enter the arm. Counted with scratch counters in a debug build, 10,000 folds per class.Instructions per call of
push_units, callees included.valgrindandperfare not available on the machine (no package source, no PMU). The count is a single-step trace in gdb, 40 calls per class, min equal to max:é, 2 bytesA fold calls
push_unitsonce per operand. Per fold: two 8-bit strings +0 (no call), UTF-16 and 1 ASCII byte +171 (100 to 271), UTF-16 and 64 ASCII bytes -128 (415 to 287), two UTF-16 strings +8 (132 to 140), UTF-16 andé+222 (105 to 327, wrong result on main).Size:
push_unitsis 144 bytes on main and 265 here (llvm-nm)..textis 79,690,917 bytes and the stripped binary 105,223,912 bytes in both (llvm-size). Alignment padding takes the 121 bytes.evaluate_binary_opis inlined intoconstant_propagation_impl, 11,611 bytes in both.Outputs that were right stay the same:
"中" + "x","x" + "中","中" + ""and a 64-byte ASCII operand are new test rows and pass on main. 0 of 14 snapshots ofreact-compiler.test.tschange. 0 of 1,807 fixture outputs differ, but no fixture callspush_units(0 calls, counted at a gdb breakpoint).A byte that is not UTF-8: with
\xE9,"中" + process.env.Vfolds to4e2d e9on main and4e2d fffdhere."x" + process.env.Vfolds to78 fffdon both. With\x80\x80\x80the two joins give4e2d fffd fffd fffdand78 fffd fffd fffdhere. Both inputs are test rows.Self-review, not taken
todotest row for the folds under "Not fixed here". CI does not run atodobody, and the list above is the record.bun linttest totest/cli/lint/react-compiler.test.ts. Three ESLint-format tests of that file have the default 5 s timeout, and CI sets a longer one. On a debug build of the first base they timed out when the whole file ran. Each passed alone. On a debug build of this branch on main 449877f they took 1.7 s, 2.8 s and 3.6 s in one run. So a debug run of that file is not a reliable proof for the new test. The test can move if a maintainer prefers that file.Tests (debug build of the branch after the rebase onto main 449877f)
bun bd test test/bundler/transpiler/react-compiler.test.ts: 124 pass, 1 skip, 1 todo. The 3 new tests fail on a debug build of main 449877f.react-compiler-fixtures.test.ts: 3,293 pass, 320 skip.react-compiler-fixtures-run.test.ts: 849 pass.test/cli/lint/react-compiler.test.ts: 26 pass, 1 skip.react-compiler.test.ts, the 3 new tests failed on a debug and on a release build of main, andtest/cli/lint/react-compiler.test.tshad 27 pass on the release build with this diff.[human-review] gate passed · iteration 0 · 2 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 1 passed · 0 rejected · iteration 0
evidence per changed file