Review follow-ups for rust-lang/regex#1350 (literal-prefix capture fast path)

[Dandandan]

This branch is rust-lang#1350 (6 commits) with 3 follow-up commits on top, for cherry-picking into the upstream PR. Only the last 3 commits are new.

Follow-up commits

1. automata: validate only Unicode-class spans in literal prefix capture (correctness)
   The fast path validated the whole haystack as UTF-8 whenever the capture class or the .* class was a Unicode class. When UTF-8 mode is off, the literal prefix and any (?-u:…) span may legitimately contain invalid UTF-8, so valid matches were dropped:

   • ^((?-u:[^/])+)/.*$ on b"\xA9/" returned no match; PikeVM matches 0..2
   • ^([^\n]+)\n(?-u:.)*$ on b"x\na\xA9bb" returned no match
   • ^(?-u:\xFF)([^/]+)/.*$ on b"\xFFab/x" returned no match

   This only happens through regex_automata::meta with syntax.utf8(false) and utf8_empty(true). The regex crate's defaults never reach it. The fix validates exactly the capture span and the tail span. A failure now skips to the next prefix instead of returning None. Because validation is now exact, Unicode classes no longer need gating on utf8_empty, so regex::bytes::Regex also gets the fast path (Q28 bytes replace: 1411 → 298 ns).

2. automata: use a single memchr to find the capture terminator (simplification)
   [^X]+ may contain \n, so the memchr2(X, '\n') loop always ends at the first X. A plain memchr gives the same result. Performance is unchanged.

3. regex: avoid Captures allocation in single-match replace (perf)
   For replace (limit == 1): a $N replacement with N < 4 uses stack slots via meta::Regex::search_slots. Other templates call captures() directly instead of captures_iter, which allocates a Captures and then clones it per match.

Benchmarks

The harness is a standalone binary, not committed. It uses 200k synthetic Referer-style URLs (60% https, 50% www., 20–160 bytes, some non-ASCII), pinned to one core, and reports the best of 3×7 runs.

workload	master	rust-lang#1350	rust-lang#1350 + fixes	rust-lang#1350 vs master	fixes vs rust-lang#1350
Q28 replace(s, "$1")	1371 ns	164 ns	129 ns	8.3x	1.28x
Q28 replace(s, "x${1}")	1375 ns	259 ns	202 ns	5.3x	1.28x
Q28 captures	1227 ns	106 ns	99 ns	11.6x	1.07x
Q28 is_match	191 ns	63 ns	61 ns	3.0x	–
Q28 replace, 2 KB URLs	24.0 µs	369 ns	324 ns	65x	1.14x
Q28 replace, non-matching	162 ns	109 ns	78 ns	1.5x	1.39x
Q28 bytes::Regex replace	1390 ns	1411 ns	298 ns	1.0x	4.74x
Q28 bytes::Regex (?-u) replace	1364 ns	247 ns	232 ns	5.5x	–
(\w+)@ first-match replace "$1" (not Q28 shape)	93 ns	90 ns	61 ns	1.0x	1.49x
replace_all "$1", 40k matches	5.57 ms	4.83 ms	4.83 ms	1.15x	–

Ablations on rust-lang#1350 itself:

• Dropping Input::new_utf8, so the fast path always validates: +30 ns on short URLs, 2x slower on 2 KB URLs. The new public API is worth something.
• Dropping single_capture_ref: Q28 replace goes 164 → 256 ns.

Testing

• Added regression tests for the UTF-8 span bug and for the stack-slot replace path (tests/misc.rs, tests/replace.rs).
• Ran a differential fuzzer (random recognized-shape patterns × random byte haystacks × 3 syntax configs × utf8_empty on/off) comparing meta::Regex capture spans against PikeVM. Before fix: 4957 mismatches in 2.8M checks. After: 0 in about 17M checks across 4 seeds.
• cargo test passes for regex and for regex-automata --all-features. --no-default-features builds pass.

Other review notes on rust-lang#1350 (no code change here)

• Public API surface: Input::new_utf8 (regex-automata) and Replacer::single_capture_ref (regex) are both new public items. Upstream may prefer #[doc(hidden)] or a narrower design.
• The MAX_PREFIX_VARIANTS comment says 32 Box<[u8]> fit "on one cache line". They take 512 bytes.
• Alternations of single bytes get lowered to classes ((?:H|h)ttps? becomes [Hh]ttps?), so they are not recognized. Expanding small byte classes in prefix_variants would widen coverage.
• The full Core is still built, but it is only used for which_overlapping_matches and the cache. Build time and memory are unchanged, which is fine.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JKuwE1LGom89mpJoo8YDtj

---

Generated by Claude Code