Report text_direction_codepoint_in_literal when parsing #141004

matthewjasper · 2025-05-14T13:37:36Z

The lint is now reported in code that gets removed/modified/duplicated by macro expansion, and spans are more accurate so we don't get ICEs from trying to split a span in the middle of a character.

This removes support for lint level attributes for text_direction_codepoint_in_literal except at the crate level, I don't think that there's an easy way around this when the lint can be reported on code that's removed by cfg or that is only in the input of a macro.

Fixes #140281

rustbot · 2025-05-14T13:37:41Z

r? @davidtwco

rustbot has assigned @davidtwco.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-05-14T16:48:38Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

mu001999 · 2025-05-16T04:47:24Z

compiler/rustc_lint_defs/src/builtin.rs

@@ -3793,7 +3793,7 @@ declare_lint! {
    /// ```rust,compile_fail
    /// #![deny(text_direction_codepoint_in_comment)]
    /// fn main() {
-    ///     println!("{:?}"); // '‮');
+    #[doc = "    println!(\"{:?}\"); // '\u{202E}');"]


why should we use #[doc = "..."] as the example?

The example isn't changed here. The attribute is used so that the RLO can be escaped. This is to avoid having to set #![allow(text_direction_codepoint_in_literal)] on the whole crate because the allow attribute no longer works at the item level.

mu001999 · 2025-05-16T14:05:49Z

I think we can find the text in the source of the given span, and then use new span to calculate and collect spans for forbidden chars.

davidtwco · 2025-05-27T09:58:46Z

I think we can find the text in the source of the given span, and then use new span to calculate and collect spans for forbidden chars.

I'm not sure I understand what you're suggesting, could you rephrase it?

compiler/rustc_parse/src/lexer/mod.rs

compiler/rustc_lint/src/hidden_unicode_codepoints.rs

- The lint is now reported in code that gets removed/modified/duplicated by macro expansion. - Spans are more accurate - Fixes rust-lang#140281

rustbot · 2025-05-27T16:17:35Z

This PR changes a file inside tests/crashes. If a crash was fixed, please move into the corresponding ui subdir and add 'Fixes #' to the PR description to autoclose the issue upon merge.

rustbot · 2025-05-27T16:17:36Z

⚠️ Warning ⚠️

There are issue links (such as #123) in the commit messages of the following commits.
Please remove them as they will spam the issue with references to the commit.
- 65bdb31

davidtwco · 2025-05-29T10:31:35Z

@bors r+

bors · 2025-05-29T10:31:37Z

📌 Commit f652067 has been approved by davidtwco

It is now in the queue for this repository.

Rollup of 8 pull requests Successful merges: - #133823 (Use `cfg_attr_trace` in AST with a placeholder attribute for accurate suggestion) - #141004 (Report text_direction_codepoint_in_literal when parsing) - #141407 (Refactor the two-phase check for impls and impl items) - #141430 (remove `visit_clobber` and move `DummyAstNode` to `rustc_expand`) - #141507 (atomic_load intrinsic: use const generic parameter for ordering) - #141538 (implement `va_arg` for x86_64 systemv) - #141669 (float: Replace some approximate assertions with exact) - #141747 (rustdoc: display doc(cfg(false)) properly) r? `@ghost` `@rustbot` modify labels: rollup

Rollup merge of #141004 - matthewjasper:unicode-before-expansion, r=davidtwco Report text_direction_codepoint_in_literal when parsing The lint is now reported in code that gets removed/modified/duplicated by macro expansion, and spans are more accurate so we don't get ICEs from trying to split a span in the middle of a character. This removes support for lint level attributes for `text_direction_codepoint_in_literal` except at the crate level, I don't think that there's an easy way around this when the lint can be reported on code that's removed by `cfg` or that is only in the input of a macro. Fixes #140281

Kobzol · 2025-05-30T15:19:57Z

@rust-timer build d7f039e

(Trying perf. due to #141753)

rust-timer · 2025-05-30T18:38:34Z

Finished benchmarking commit (d7f039e): comparison URL.

Overall result: ❌ regressions - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.3%	[0.1%, 0.6%]	15
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.2%	[-0.2%, -0.2%]	1
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary 0.9%, secondary 0.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.9%	[2.9%, 2.9%]	1
Regressions ❌ (secondary)	1.2%	[0.5%, 3.4%]	5
Improvements ✅ (primary)	-1.1%	[-1.1%, -1.1%]	1
Improvements ✅ (secondary)	-0.6%	[-0.7%, -0.4%]	7
All ❌✅ (primary)	0.9%	[-1.1%, 2.9%]	2

Cycles

Results (secondary 0.2%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.5%	[0.5%, 5.8%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-1.2%	[-3.2%, -0.5%]	7
All ❌✅ (primary)	-	-	0

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: missing data
Artifact size: 370.26 MiB -> 370.29 MiB (0.01%)

Rollup of 8 pull requests Successful merges: - rust-lang/rust#133823 (Use `cfg_attr_trace` in AST with a placeholder attribute for accurate suggestion) - rust-lang/rust#141004 (Report text_direction_codepoint_in_literal when parsing) - rust-lang/rust#141407 (Refactor the two-phase check for impls and impl items) - rust-lang/rust#141430 (remove `visit_clobber` and move `DummyAstNode` to `rustc_expand`) - rust-lang/rust#141507 (atomic_load intrinsic: use const generic parameter for ordering) - rust-lang/rust#141538 (implement `va_arg` for x86_64 systemv) - rust-lang/rust#141669 (float: Replace some approximate assertions with exact) - rust-lang/rust#141747 (rustdoc: display doc(cfg(false)) properly) r? `@ghost` `@rustbot` modify labels: rollup

rustbot assigned davidtwco May 14, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 14, 2025

This comment has been minimized.

Sign in to view

matthewjasper force-pushed the unicode-before-expansion branch from f5d463a to 92445ef Compare May 14, 2025 13:59

This comment has been minimized.

Sign in to view

matthewjasper force-pushed the unicode-before-expansion branch from 2f169d6 to c5f6a06 Compare May 15, 2025 12:41

This comment has been minimized.

Sign in to view

matthewjasper force-pushed the unicode-before-expansion branch from c5f6a06 to 8e2a331 Compare May 15, 2025 15:23

mu001999 reviewed May 16, 2025

View reviewed changes

davidtwco requested changes May 27, 2025

View reviewed changes

compiler/rustc_parse/src/lexer/mod.rs Show resolved Hide resolved

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 27, 2025

mu001999 reviewed May 27, 2025

View reviewed changes

compiler/rustc_lint/src/hidden_unicode_codepoints.rs Show resolved Hide resolved

matthewjasper added 3 commits May 27, 2025 15:57

Report text_direction_codepoint_in_literal when parsing

65bdb31

- The lint is now reported in code that gets removed/modified/duplicated by macro expansion. - Spans are more accurate - Fixes rust-lang#140281

Avoid including text direction codepoints in lint messages

e5bfd02

Warn on non-crate level text direction lints

f652067

matthewjasper force-pushed the unicode-before-expansion branch from 8e2a331 to f652067 Compare May 27, 2025 16:17

davidtwco approved these changes May 29, 2025

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 29, 2025

matthiaskrgr mentioned this pull request May 30, 2025

Rollup of 8 pull requests #141753

Merged

bors merged commit 5fc3f26 into rust-lang:master May 30, 2025
7 checks passed

rustbot added this to the 1.89.0 milestone May 30, 2025

This comment has been minimized.

Sign in to view

rustbot added the perf-regression Performance regression. label May 30, 2025

sminez mentioned this pull request Jun 1, 2025

Searching for multi-byte utf-8 characters in the minibuffer results in a panic sminez/ad#128

Merged

Report text_direction_codepoint_in_literal when parsing #141004

Report text_direction_codepoint_in_literal when parsing #141004

Uh oh!

Conversation

matthewjasper commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented May 14, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

rustbot commented May 14, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

mu001999 May 16, 2025

Choose a reason for hiding this comment

Uh oh!

matthewjasper May 16, 2025

Choose a reason for hiding this comment

Uh oh!

mu001999 commented May 16, 2025

Uh oh!

davidtwco commented May 27, 2025

Uh oh!

Uh oh!

Uh oh!

rustbot commented May 27, 2025

Uh oh!

rustbot commented May 27, 2025

Uh oh!

davidtwco commented May 29, 2025

Uh oh!

bors commented May 29, 2025

Uh oh!

Uh oh!

Kobzol commented May 30, 2025

Uh oh!

This comment has been minimized.

rust-timer commented May 30, 2025

Overall result: ❌ regressions - please read the text below

Instruction count

Max RSS (memory usage)

Cycles

Binary size

Uh oh!

Uh oh!

matthewjasper commented May 14, 2025 •

edited

Loading