Rollup merge of rust-lang#155279 - Sandijigs:test/lexer-unicode-pattern-white-space, r=jdonszelmann

Test/lexer unicode pattern white space

This PR adds a test for the Rust lexer to verify it correctly accepts vertical tab (`\x0B`) as valid whitespace between tokens. Vertical tab is part of Unicode Pattern_White_Space, which the Rust language specification uses to define whitespace.

Related: Outreachy tracking [Pattern_White_Space](https://www.unicode.org/reports/tr31/#R3a)

Author: JonathanBrouwer

tests/ui/lexer/unicode-pattern-white-space.rs

@@ -0,0 +1,12 @@
+//@ run-pass
+// Test that the Rust lexer accepts vertical tab (\x0B) as valid whitespace
+// between tokens. Vertical tab is part of Unicode Pattern_White_Space, which
+// the Rust language specification uses to define whitespace tokens.
+// See: https://unicode.org/reports/tr31/#Pattern_White_Space
+//
+// The space between "let" and "_" below is a vertical tab character (\x0B),
+// not a regular space.
+
+fn main() {
+    let_ = 1;
+}