Repository navigation
Conversation
|
Preview build of 5ed1127: |
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline findings, I also checked the count-then-fill rewrite of encode and escape: the per-code-unit output lengths (3/6/9/12 for UTF-8 octet counts, 3/6 for %XY/%uWXYZ) match what the fill pass writes, and the unchecked *++cursor after a lead surrogate in the fill pass is safe because the count pass already threw on a trailing or unpaired lead. The restructured unescape loop advances k by the same 3 and 6 as the old 8-bit and 16-bit branches, so the parse is unchanged.
Extended reasoning...
The change rewrites the six URI globals in Source/JavaScriptCore/runtime/JSGlobalObjectFunctions.cpp to return the argument JSString when nothing changes and to size the encode/escape result in one pass, plus a new stress test, a microbenchmark, an updated out-of-memory test and a BUN-TEST-DIFFERENCES.md row; it touches no auth, injection or data-exposure surface. Two confirmed inline findings (substring-rope retention from returning the argument, and the unescape/decode 16-bit overflow asymmetry) already signal a human look, so this note only records what else was examined and ruled out.
| static JSValue unchanged(JSGlobalObject* globalObject, JSString* string, std::span<const CharacterType> characters) | ||
| { | ||
| if constexpr (sizeof(CharacterType) == 1) | ||
| return string; |
There was a problem hiding this comment.
🟡 (optional) Callers that store a URI function's result computed from a slice of a large string now keep that whole large string alive, which the base does not. unchanged() at JSGlobalObjectFunctions.cpp:83 returns the argument JSString itself, and for a substring rope view() leaves it unresolved, so the result still points at the base cell. Fix: when string->isSubstring(), return a fresh flat string built from characters (one copy, as the old builder made) and keep the identity return for non-substring strings only; the 16-bit return at line 86 needs the same. The author's note compares this to slice, but slice results are usually transient while decoded values are what gets stored.
Why this was flagged
Input: any string of 3 or more code units produced by String.prototype.split or slice, e.g. a form-body or query-string parser doing decodeURIComponent(part) for each part of body.split('&'). withStringArgument calls string->view() at JSGlobalObjectFunctions.cpp:69; for a substring rope JSRopeString::view at JSString.h:1281-1284 returns the base's view without resolving. When the part has no '%' (decode, line 237-239) or every unit is unescaped (encode line 121-122, escape line 729-730), unchanged() at line 83 (8-bit) or line 86 (16-bit non-Latin1) returns that same substring rope cell. A caller that keeps the decoded value now retains the entire source string for the lifetime of that value. On the base branch every result came from builder.toString() (old lines 138, 237, 668) or StringImpl creation, an independent flat string with no reference to the source, so the source was collectable once the parts were dropped. No safeguard exists: nothing in unchanged() checks isSubstring() or the base length.
Verification: unchanged() at JSGlobalObjectFunctions.cpp:83 does return string; for 8-bit input (and for 16-bit non-Latin1 input), so the returned value is the substring rope whose fiber1 is the base JSString (substringBase(), JSString.h:768). On the base commit, every path went through a StringBuilder and jsString(vm, builder.toString()), producing a fresh StringImpl independent of the argument's storage.
| int length = static_cast<int>(characters.size()); | ||
|
|
||
| StringBuilder builder(OverflowPolicy::RecordOverflow); | ||
| builder.reserveCapacity(length); |
There was a problem hiding this comment.
🟣 pre-existing, not blocking: Users calling unescape on a large 16-bit string get RangeError when it contains one '%', while the same string without '%' now succeeds. JSGlobalObjectFunctions.cpp:791 still does reserveCapacity(length) as 8-bit, and the first non-Latin1 unit appended at :813 upconverts to a doubled 16-bit buffer, which overflows for inputs near 2^30. The PR and A10 in JSTests/BUN-TEST-DIFFERENCES.md present the %-free case as fixed, leaving the sibling path with the old cap. Fix: for a 16-bit argument reserve the capacity in 16-bit (or skip reserveCapacity) so the builder does not double on upconvert, and cover the '%'-containing case in the updated test.
A small fix can ride a push you are already making; otherwise a short reply is enough.
Why this was flagged
The input is a 16-bit string of about 2^30 code units that contains at least one '%', passed to unescape, reaching the builder path at JSGlobalObjectFunctions.cpp:790-798. reserveCapacity(length) at :791 allocates an 8-bit buffer of length; the first append of a unit >= 0x100 at :813 calls extendBufferForAppendingWithUpconvert, which allocates a char16_t buffer of min(2*length, maxCapacity), so a 2^30 input asks for a ~4 GiB buffer and records overflow, giving RangeError: Out of memory at :818. The base branch behaved the same for this input, but it also threw for the %-free input; the change removes the throw only for the %-free case (:779-780), so after merging the outcome depends on whether the string happens to contain a '%'. The new test at JSTests/stress/out-of-memory-in-globalFuncUnescape.js:6-8 asserts the %-free case returns the argument and does not exercise the sibling. Remedy: avoid the 8-bit reserve for 16-bit input so the upconvert does not double, and add the '%' case to the test.
Verification: Pre-existing; acknowledged in diff. A 16-bit argument of about 2^30 code units that contains a '%' is passed to unescape. Source/JavaScriptCore/runtime/JSGlobalObjectFunctions.cpp:791 allocates an 8-bit buffer; the first unit >= 0x100 at :813 upconverts to a doubled char16_t buffer, and :817-818 throws the out-of-memory RangeError. The base branch fails this input by the same route; merging makes nothing worse.
94dbd7b to
52f0bfd
Compare
52f0bfd to
187ae6b
Compare
…nd build the result in one string otherwise encodeURI, encodeURIComponent, escape, decodeURI, decodeURIComponent and unescape built every result in a StringBuilder, one append per code unit, and made a new StringImpl and JSString for it, also when the result had the argument's code units. encodeURIComponent of a short plain string cost about twice what V8 charges. Each function now scans first. When no code unit changes, it returns the argument JSString (a 16-bit argument whose code units fit in 8 bits is narrowed, as the builder did). encode and escape then count the output length in one pass, with the URIError checks of the spec in the same places, and fill one StringImpl of that size. decode and unescape keep their parse. It starts at the first '%', with the code units before it appended in one piece. JSTests/stress/out-of-memory-in-globalFuncUnescape.js: unescape of a '%'-free string is that string, so the overflow case moves to encodeURIComponent. JSTests/stress/uri-globals-return-the-argument.js pins the results for 8-bit and 16-bit strings against V8's. JSTests/microbenchmarks/uri-globals-short-strings.js times the six functions on short strings.
187ae6b to
5ed1127
Compare
…eturn their argument when nothing changes Adds test/js/bun/jsc/uri-globals.test.ts and bench/snippets/encode-uri.mjs.
Related to oven-sh/bun#45700.
Problem
encodeURIComponent("photos")costs 40 ns in bun against 17 ns in node. A 26 char value with 7 escapes costs 93 ns against 50.escapeandunescapeof a plain string cost 4 to 5x node.runtime/JSGlobalObjectFunctions.cpp. All six URI functions go throughtoStringView, append to aStringBuilderone code unit at a time, and make a newStringImplandJSStringfrom it. They do that also when the result has the argument's code units, which is the common case for a path piece or a query value.Fix
withStringArgument, which hands the callback theJSString. A scan runs first. When no code unit changes (encode: every unit is in the unescaped set, decode: no%), the function returns the argument.unchangednarrows a 16-bit argument whose code units fit in 8 bits, as the builder did, so a caller still reads an 8-bit result.encodeandescapecount the output length in one pass, with the URIError checks of the spec at the same steps, and fill oneStringImplof that size. A length overStringImpl::MaxLengthis the out-of-memoryRangeErroras before.decodeandunescapekeep their parse. It starts at the first%. The code units before it are appended in one piece (8-bit input only: a bulk append of 16-bit units would make the builder 16-bit).JSTests/stress/uri-globals-return-the-argument.js(new, 8-bit and 16-bit strings against V8's results, and the 8-bit representation of the result), the five existing URI stress tests, the updatedout-of-memory-in-globalFuncUnescape.js, and the 208 test262 files underbuilt-ins/{encodeURI,encodeURIComponent,decodeURI,decodeURIComponent}andannexB/built-ins/{escape,unescape}(208/208 before and after, with a one-off runner).Background
toStringView(runtime/ParseInt.h) converts the argument and passes only aStringViewto its callback. The callback cannot return the argument.parseIntandparseFloatkeep using it.JSStringthat is a rope resolves in place inview(). The returned cell is the same cell. A substring rope is returned unresolved, whichString.prototype.slicealso does.JSTests/stress/out-of-memory-in-globalFuncUnescape.jsexpectedRangeErrorfromunescapeof 2^30\u0100with no%. That came from the builder's doubling on upconvert. The input now returns itself, and the file keeps an overflow case throughencodeURIComponent(six code units per input unit).JSTests/BUN-TEST-DIFFERENCES.mdrow A10 records it.decodeandunescapetoo: their count pass is the whole%XXparse, so the dense case would run it twice. Considered a bun-side override of the three encode globals in bun's global object table: a second encoder in the process, andnode:vmcontexts keep the slow one.Downsides
StringImpland oneJSStringfor the narrowed copy, as before. AdecodeURIComponentof 60 KB with an escape every 25 units is 6% slower in the bun build (88 to 94 us, node 57 us), and 5% faster in thejscshell (80 to 75 us). The per unit parse is unchanged, so this is code layout.encodeandescaperead an input with escapes twice. On 60 KB with 7 escapes per 26 units that is still 200 to 135 us. Text grows by 8.9 KB in bun's release binary (size, 103,175,513 to 103,184,409).Measurements
Release builds of bun, linked against this commit (B) and against
87ee10da(A), node 26.3.0, linux x64 in a shared container, so the absolute numbers are high./tmp/one2.mjs: 200k calls warm-up, median of 11 runs of 200k calls (2k for 60 KB), three interleaved processes per runtime. ns per call.JSString cells per call of
encodeURIComponent("photos"), 100k calls kept alive,bun:jscheapStats().objectTypeCounts.stringafter a full GC: A 1.00, B 0.00.jscshell,JSTests/microbenchmarks:encode-uri-component-query-string.js110 to 126 ms to 68 to 71 ms,encode-uri-component-unicode.js95 to 100 ms to 50 to 54 ms,uri-globals-short-strings.js(new) 127 to 130 ms to 66 to 76 ms.decodeURIComponent("a".repeat(2**29 + 1) + "%E2%82%AC"):RangeError: Out of memoryon A, the decoded string on B (the prefix buffer is exact, not a doubled capacity).2**30fails on both: the builder's upconvert cap, #631.Instruction and allocation counts are not measured: the container has no
perf,valgrindorbloaty. The cell count stands in for the allocation count.