Skip to content

Fix ~300x slowdown of /\p{RGI_Emoji}/v vs Node (Yarr first-character prefilter) - #37294

Closed
robobun wants to merge 1 commit into
mainfrom
farm/4d0150ed/rgi-emoji-prefilter
Closed

robobun wants to merge 1 commit into
mainfrom
farm/4d0150ed/rgi-emoji-prefilter

Conversation

@robobun

@robobun robobun commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Fixes #37290

Repro

const rgi = /^\p{RGI_Emoji}$/v
const chars = [..."─".repeat(156)]
// each .test() on a non-emoji char took ~23µs; ~300x slower than Node
for (const c of chars) rgi.test(c)

string-width@8 runs this regex per grapheme cluster, so any Ink/TUI line containing box-drawing characters hit it constantly; the issue measured 84% of process CPU in this one regex.

Cause

JSC Yarr expands a v-mode property of strings (\p{RGI_Emoji} has ~3700 strings) into one flat alternative per string, and a failing match attempt walks every alternative. The existing DFG first-character bitmap filter is Latin-1 only, so non-Latin-1 input (U+2500 box drawing) bypassed it.

Fix

oven-sh/WebKit#401 (included here by bumping WEBKIT_VERSION to 887d61174f34): when expanding a class set with strings, Yarr now wraps the string alternatives in a lookahead on the set of possible first characters:

(?: s1 | ... | sN | [singles] )  ->  (?: (?=[firstChars]) (?: s1 | ... | sN ) | [singles] )

A non-matching input fails after one character-class check (binary search) in both the Yarr interpreter and JIT. Skipped for backward matching, sets containing the empty string, and sets with fewer than 16 strings (where the anchored string-list fast path applies instead).

Numbers

/^\p{RGI_Emoji}$/v.test() over 156 distinct non-emoji chars, release, linux x64:

miss (U+2500) vs \p{Extended_Pictographic} control
before 3.68 ms/iter ~1170x
after 0.008 ms/iter ~1.8x
node 26 0.016 ms/iter ~5.5x

Verification

test/js/bun/jsc/webkit-upgrade-887d6117.test.ts pins the perf ratio (fails at ~1016x on the previous WebKit, ~2x after) plus RGI/\q{} matching semantics: longest-alternative capture order, modifier/flag/keycap/ZWJ/tag sequences, set subtraction, ignore-case, empty-string \q sets, and lookbehinds. JSC regexp/yarr stress tests pass on a debug+ASAN build of the patched WebKit.

Note: bun CI needs the oven-sh/WebKit autobuild artifacts for 887d61174f34; if CI ran before that build published, the failure is just the missing prebuilt tarball.

…ns in Yarr

/^\p{RGI_Emoji}$/v.test() on a non-emoji character was ~300x slower
than Node because Yarr expands the property of strings into ~3700 flat
alternatives and a failing match walked all of them. string-width@8
runs that regex per grapheme cluster, which made Ink TUIs drawing
box-drawing characters spend ~84% of process CPU in this one regex.

The WebKit side (oven-sh/WebKit#401) guards the expanded string
alternatives with a lookahead on the set of possible first characters,
so a non-matching input fails after one character-class check. Misses
go from 3.68ms to 0.008ms per 156 calls in release, about 2x faster
than Node 26 on the same benchmark.

Fixes #37290
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 7 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8176ae21-24f2-4664-825c-c9a26a00f757

📥 Commits

Reviewing files that changed from the base of the PR and between cbc209f and 4ee42e2.

📒 Files selected for processing (2)
  • scripts/build/deps/webkit.ts
  • test/js/bun/jsc/webkit-upgrade-887d6117.test.ts

Comment @coderabbitai help to get the list of available commands.

Comment on lines +83 to +86
// An empty string in the set matches without consuming input; the
// prefilter must not apply (nothing to filter on).
expect(/^[\q{ab|}]$/v.test("")).toBe(true);
expect(/^[\q{ab|}]$/v.test("ab")).toBe(true);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The comment says the prefilter "must not apply" here because of the empty string, but /^[\q{ab|}]$/v has only 2 alternatives — below the 16-string threshold — so the prefilter is already skipped for that reason and this fixture can't detect a regression in the empty-string guard. To actually isolate that skip condition, use a ≥16-string set that includes an empty alternative, e.g. new RegExp(^[\q{${Array.from({length:20},(_,i)=>"s"+i).join("|")}|}]$, "v").test("") (or just reword the comment to say it's testing empty-string \q{} semantics, not the prefilter skip).

Extended reasoning...

What the bug is

The comment at lines 83–84 frames this fixture as coverage for the empty-string prefilter skip: "An empty string in the set matches without consuming input; the prefilter must not apply (nothing to filter on)." Per the PR description, the prefilter is skipped for three independent reasons: backward matching, sets containing the empty string, and sets with fewer than 16 strings. The fixture /^[\q{ab|}]$/v has only 2 string alternatives ("ab" and ""), so it already falls under the <16 threshold. The empty-string guard is not what keeps the prefilter off here.

Why this matters

REVIEW.md's test-quality section says: "trace the fixture through every earlier guard/threshold using real constants from source" and "confirm deleting each load-bearing clause of your fix breaks at least one test." If a future WebKit change removed or broke the contains-empty-string guard — e.g. wrapped a set containing "" in (?=[firstChars]), which would incorrectly reject an empty match — this test would still pass, because a 2-element set stays on the below-threshold anchored-string-list fast path and never reaches the prefilter code at all. The empty-string skip clause therefore has no isolating coverage in this file.

The author clearly understood the threshold matters — the two preceding fixtures in the same test deliberately use 40 and 20 strings to get above 16. The empty-string case is the odd one out.

Step-by-step proof

  1. The PR description states the prefilter is "Skipped for backward matching, sets containing the empty string, and sets with fewer than 16 strings (where the anchored string-list fast path applies instead)."
  2. /^[\q{ab|}]$/v → the class set contains exactly 2 string alternatives: "ab" and "".
  3. 2 < 16, so the below-threshold skip applies independently of whether "" is present.
  4. Thought experiment: delete the "set contains the empty string" guard from Yarr. For this fixture, the <16 guard still fires → prefilter still skipped → .test("") still returns true → assertion still passes.
  5. Therefore this fixture cannot detect a regression in the empty-string guard; it only tests that empty-string \q{} semantics work on the below-threshold expansion path.

Fix

Either reword the comment to say it's exercising empty-string \q{} matching semantics (not the prefilter skip), or — better, since the PR description's Verification section claims this covers "empty-string \q sets" as a skip condition — use a set that clears the threshold:

// An empty string in the set matches without consuming input; even above
// the 16-string threshold the prefilter must not apply (nothing to filter on).
const withEmpty = new RegExp(`^[\\q{${Array.from({ length: 20 }, (_, i) => `s${i}`).join("|")}|}]$`, "v");
expect(withEmpty.test("")).toBe(true);
expect(withEmpty.test("s7")).toBe(true);

Now if the empty-string guard regressed and Yarr wrapped this 21-alternative set in (?=[s])(?: s0 | ... | s19 | ), the empty input would fail the lookahead and .test("") would return false, catching the regression.

Severity

This is a coverage gap plus an inaccurate comment on one of three skip conditions — the assertions themselves are semantically correct, the primary perf regression and RGI_Emoji semantics are well-covered elsewhere in the file, and nothing breaks at runtime if merged as-is. Nit.

// input (release; the interpreter-only gap is ~130x). With the prefilter
// both are a single character-class check, ratio ~2x. The threshold sits
// well above fixed-build noise and well below every unfixed measurement.
const chars = [..."\u2500".repeat(156)];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The comment says "156 distinct single-character non-emoji strings" (and the PR description says "156 distinct non-emoji chars"), but [..."─".repeat(156)] produces 156 identical copies of "─" (U+2500), not 156 distinct characters. Either drop the word "distinct" from the comment, or use Array.from({length: 156}, (_, i) => String.fromCodePoint(0x2500 + i)) to actually match the described box-drawing-TUI-line scenario.

Extended reasoning...

What the code does vs. what the comment claims

Line 33 constructs the test input as:

const chars = [..."─".repeat(156)];

"─".repeat(156) produces a 156-character string where every character is U+2500 BOX DRAWINGS LIGHT HORIZONTAL (─). U+2500 is a single BMP code unit, so spreading the string yields an array of 156 identical one-character strings: ["─", "─", ..., "─"].

The comment immediately above (lines 26–27) reads:

156 distinct single-character non-emoji strings (a TUI line of box drawing, the string-width worst case from the issue).

And the PR description's Numbers section says:

/^\p{RGI_Emoji}$/v.test() over 156 distinct non-emoji chars, release, linux x64:

Both claim the 156 inputs are distinct. They are not.

Step-by-step proof

  1. "─" is the single code point U+2500, encoded as one UTF-16 code unit.
  2. .repeat(156) → a string of length 156, every code unit U+2500.
  3. [...str] iterates the string by code point → 156 elements, each the string "─".
  4. new Set(chars).size would be 1, not 156.

Why it doesn't affect test validity

The perf assertion still discriminates fixed vs. unfixed builds. Yarr does not memoize .test() results per input value — the prefilter cost is paid on every call regardless of whether the input string was seen before. So 156 identical misses cost the same as 156 distinct misses, and the measured ratio (rgi / control) is unchanged. This is purely a comment-accuracy issue, not a behavioral bug.

Why it's still worth fixing

Per REVIEW.md, comments are reviewed as code ("Only comment what the code cannot say"), and an inaccurate comment misleads future readers about what invariant the test relies on. Someone reading "156 distinct" might infer that input distinctness matters to the measurement (e.g., to defeat some per-input cache), and preserve that non-existent property when refactoring — or conversely, wonder why the code doesn't match and assume one side is a bug.

How to fix

Two options, either is fine:

Option A — fix the comment (simplest, since distinctness doesn't matter):

// 156 single-character non-emoji strings (a TUI line of box drawing, ...

Option B — fix the code to match the comment (more faithful to "a TUI line of box drawing", which in practice mixes several box-drawing glyphs):

const chars = Array.from({ length: 156 }, (_, i) => String.fromCodePoint(0x2500 + i));

Either way, the PR description's "156 distinct non-emoji chars" should be updated to match.

@robobun

robobun commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regex: /^\p{RGI_Emoji}$/v is ~300x slower than Node (breaks string-width, cripples Ink TUIs)

1 participant