Add an LLM policy for rust-lang/rust

[jyn514]

FCP link

this comment

Summary

This document establishes a policy for how LLMs can be used when contributing to rust-lang/rust. Subtrees, submodules, and dependencies from crates.io are not in scope. Other repositories in the rust-lang organization are not in scope.

This policy is intended to live in Forge as a living document, not as a dead RFC. It will be linked from CONTRIBUTING.md in rust-lang/rust as well as from the rustc- and std-dev-guides.

Moderation guidelines

This PR is preceded by an enormous amount of discussion on Zulip. Almost every conceivable angle has been discussed to death; there have been upwards of 3000 messages, not even counting discussion on GitHub. We initially doubted whether we could reach consensus at all.

Therefore, we ask to bound the scope of this PR specifically to the policy itself. In particular, we mark several topics as out of scope below. We still consider these topics to be important, we simply do not believe this is the right place to discuss them.

No comment on this PR may mention the following topics:

• Long-term social or economic impact of LLMs
• The environmental impact of LLMs
• Anything to do with the copyright status of LLM output
• Moral judgements about people who use LLMs

We have asked the moderation team to help us enforce these rules.

Feedback guidelines

We are aware that parts of this policy will make some people very unhappy. As you are reading, we ask you to consider the following.

• Can you think of a concrete improvement to the policy that addresses your concern? Consider:
  • Whether your change will make the policy harder to moderate
  • Whether your change will make it harder to come to a consensus
• Does your concern need to be addressed before merging or can it be addressed in a follow-up?
  • Keep in mind the cost of not creating a policy.

If your concern is for yourself or for your team

• What are the specific parts of your workflow that will be disrupted?
  • In particular we are only interested in workflows involving rust-lang/rust. Other repositories are not affected by this policy and are therefore not in scope.
• Can you live with the disruption? Is it worth blocking the policy over?

---

Previous versions of this document were discussed on Zulip, and we have made edits in responses to suggestions there.

Motivation

• Many people find LLM-generated code and writing deeply unpleasant to read or review.
• Many people find LLMs to be a significant aid to learning and discovery.
• rust-lang/rust is currently dealing with a deluge of low-effort "slop" PRs primarily authored by LLMs.
  • Having a policy makes these easier to moderate, without having to take every single instance on a case-by-case basis.

This policy is not intended as a debate over whether LLMs are a good or bad idea, nor over the long-term impact of LLMs. It is only intended to set out the future policy of rust-lang/rust itself.

Drawbacks

• This bans some valid usages of LLMs. We intentionally err on the side of banning too much rather than too little in order to make the policy easy to understand and moderate.
• This intentionally does not address the moral, social, and environmental impacts of LLMs. These topics have been extensively discussed on Zulip without reaching consensus, but this policy is relevant regardless of the outcome of these discussions.
• This intentionally does not attempt to set a project-wide policy. We have attempted to come to a consensus for upwards of a month without significant progress. We are cutting our losses so we can have something rather than adhoc moderation decisions.
• This intentionally does not apply to subtrees of rust-lang/rust. We don't have the same moderation issues there, so we don't have time pressure to set a policy in the same way.

Rationale and alternatives

• We could create a project-wide policy, rather than scoping it to rust-lang/rust. This has the advantage that everyone knows what the policy is everywhere, and that it's easy to make things part of the mono-repo at a later date. It has the disadvantage that we think it is nigh-impossible to get everyone to agree. There are also reasons for teams to have different policies; for example, the standard for correctness is much higher within the compiler than within Clippy.
• We could have different standards for people in the Rust project than for new contributors. That would make moderation much easier, and allow us to experiment with additional LLM use. However, it reinforces existing power structures, creates more of a gap between authors and reviewers, and feels "unfriendly" to new contributors.
• We could have a more lenient policy that allows "responsible and appropriate" use of LLMs. This raises the question of what "responsible and appropriate" means. The usual suggestion is "self-review, and judging the change by the same standard as any other change"; but this neglects the reputational and social harm of work that "feels" LLM generated. It also makes our moderation policy much harder to understand, and increases the likelihood of re-litigating each moderation decision.
• We could have a more strict policy that removes the threshold of originality condition. This has the advantage that our policy becomes easier to moderate and understand. It has the disadvantage that it becomes easy for people to intend to follow the policy, but be put in a position where their only choices are to either discard the PR altogether, rewrite it from scratch, or tell "white lies" about whether an LLM was involved.
• We could have a more strict policy that bans LLMs altogether. It seems unlikely we will be able to agree on this, and we believe attempting it will cause many people to leave the project.

Prior art

This prior art section is taken almost entirely from Jane Lusby's summary of her research, although we have taken the liberty of moving the Rust project's prior art to the top. We thank her for her help.

Rust

• Moderation team's spam policy
• Compiler team's "burdensome PRs" policy

Other organizations

These are organized along a spectrum of AI friendliness, where top is least friendly, and bottom is most friendly.

• full ban
  • postmarketOS - also explicitly bans encouraging others to use AI for solving problems related to postmarketOS - multi point ethics based rational with citations included
  • zig
    • philosophical, cites Profession (novella)
    • rooted in concerns around the construction and origins of original thought
  • servo
    • more pragmatic, directly lists concerns around ai, fairly concise
  • qemu
    • pragmatic, focuses on copyright and licensing concerns
    • explicitly allows AI for exploring api, debugging, and other non generative assistance, other policies do not explicitly ban this or mention it in any way
• allowed with supervision, human is ultimately responsible
  • scipy
    • strict attribution policy including name of model
  • llvm
  • blender
  • linux kernel
    • quite concise but otherwise seems the same as many in this category
  • mesa
    • framed as a contribution policy not an AI policy, AI is listed as a tool that can be used but emphasizes same requirements that author must understand the code they contribute, seems to leave room for partial understanding from new contributors.

      Understand the code you write at least well enough to be able to explain why your changes are beneficial to the project.

  • forgejo
    • bans AI for review, does not explicitly require contributors to understand code generated by ai. One could interpret the "accountability for contribution lies with contributor even if AI is used" line as implying this requirement, though their version seems poorly worded imo.
  • firefox
  • ghostty
    • pro-AI but views "bad users" as the source of issues with it and the only reason for what ghostty considers a "strict AI policy"
  • fedora
    • clearly inspired and is cited by many of the above, but is definitely framed more pro-ai than the derived policies tend to be
• curl
  • does not explicitly require humans understand contributions, otherwise policy is similar to above policies
• linux foundation
  • encourages usage, focuses on legal liability, mentions that tooling exists to help automate managing legal liability, does not mention specific tools
• In progress
  • NixOS
    • How do we deal with AI-generated issues? NixOS/nixpkgs#410741

Unresolved questions

See the "Moderation guidelines" and "Drawbacks" section for a list of topics that are out of scope.

Rendered

[rustbot]

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Fallback group: @Mark-Simulacrum, internal-sites
• @Mark-Simulacrum, internal-sites expanded to Mark-Simulacrum, Urgau, ehuss, jieyouxu
• Random selection from Mark-Simulacrum, Urgau, ehuss, jieyouxu

[jyn514]

@rustbot label T-libs T-compiler T-rustdoc T-bootstrap

[jieyouxu]

I really like this version, and thanks a ton for working on it. Specifically:

• It doesn't try to dump entire walls of text, which is unfortunately a good way to be sure nobody reads it. Instead, it gives you concrete examples, and a guiding rule-of-thumb for uncovered scenarios, and acknowledges upfront that it surely cannot be exhaustive.
• I also like where it points out the nuance and recognizes the uncertainties.
• I like that it covers both "producers" and "consumers" (with nuance that reviewers can also technically use LLMs in ways that are frustrating to the PR authors!)

I left a few suggestions / nits, but even without them this is still a very good start IMO.

(Will not leave an explicit approval until we establish wider consensus, which likely will take the form of 4-team joint FCP.)

View changes since this review

[ChayimFriedman2]

The links to Zulip are project-private, FWIW.

[jyn514]

The links to Zulip are project-private, FWIW.

I'm aware. This PR is targeted towards Rust project members moreso than the broad community.

[davidtwco]

I'm happy with this as an initial policy for the rust-lang/rust repository.

View changes since this review

[Manishearth]

@rust-rfcbot concern clearly-marked-ai-generated-reports

More details in #1040 (comment) . I think explicitly allowing clearly marked, vetted, AI-generated reports be attached to PRs/issues is something we will have to do in order to not impede security work.

[Manishearth]

@rust-rfcbot resolve clearly-marked-ai-generated-reports

This is addressed by:

- ℹ️ This does not apply if the LLM content is clearly quoted and marked, you can post that.
    However, the content of the comment must stand on its own even without the LLM content; it's not a substitute for your own words.

AI-generated reports are supplemental material, so that's fine. We still expect issues/PRs to be readable without needing to go through the report.

---

Regarding the reverted change about T-security-response; overall security-response doesn't write much code. There is a small chance that we will end up in a situation where Mythos/etc file piles and piles of low grade issues and the libs team needs to rely on LLMs to fix them faster, and if that's the case then that exemption can be explicitly requested (it would be a nontrivial conversation; LLM use to fix security issues is a complex topic).

security-response's needs are sufficiently handled by being able to quote reports in issues/PRs.

[Kobzol]

Leaving aside the details of the policy (I haven't read the latest draft, actually, but I felt like the previous version was close-ish), my personal opinion is that the policy on rust-lang/rust is central enough that this is effecively "committing the project" and should be a leadership council decision.

This is e.g. the same logic that led me to withdraw my proposed project goal for lang team involvement in Symposium (as @Kobzol and others argued, that was effectively committing the project to an AI policy, and it seemed like that was a reputational decision that ought to have input from all teams).

I don't think that this is the same as Symposium. That was accepting an AI-related initiative as a Project Goal, which is something that IMO clearly reflects on the Project as a whole just by its name, and exposes us to it ("our Goal is to change Rust to be better for AI", or something along those lines). Eventually, you pushed Symposium to be in RIL, which certainly also counts as a non-trivial endorsement, but this decision has received no official support from any Rust teams at all,
so you probably didn't think that the fear of committing us is a big concern at that time.

I agree that having a policy for rust-lang/rust is something that will be very publicly visible, but there are large differences between the tone of this policy and Symposium. Symposium is one-sided; it strongly presents the case that AI in programming is good, and that Rust should change to accommodate that (or it did that at the time, it might have changed in the meantime). If this policy was something like Zig's "no LLMs ever", I would agree that it can be seen as setting a tone for the whole Project. But this policy has been very carefully designed to allow LLM usage in various complicated ways, and to avoid making comments on the larger scope discussion topics about AI in general. It does not make either side of the camp really happy, and I don't think that anyone could read it in good faith to be either "Rust is pro-AI" OR "Rust is anti-AI".

---

I have to say that I also share jyn's dissatisfaction about the way you presented the procedural concern. Jack has presented a specific concern about things in the policy that he finds are too limiting. That is a good approach; we can do targeted text changes to resolve that specific concern, and move on. But I don't understand why you file a procedural concern at this point. This policy was specifically designed to not be Project-wide, to target the repository where AI currently hurts us the most, and to be FCPed by the teams that actually do most of their work in this repository. Going to LC subverts the original motivation of the policy, which was explicitly created to NOT be Project-wide, and to not have to go through the LC, in order to have any chance of landing in a reasonable timeframe (where reasonable would IMO be something like a few months ago, but things got complicated, so at the very least we should try to move forward with the best policy that we have at the moment).

Why haven't you filed this concern a month ago? If we knew that such a concern would exist, we might as well just not try at all, because this is exactly the same filibuster that already stopped several previous policies in their tracks. The pattern repeats all over again:

1. Propose a specific policy carve-out for rust-lang/rust, to avoid limiting teams in other repositories, and make it easier to land, and to solve a specific problem that hurts r-l/r's maintainers.
2. Someone proposes for the policy to become a Project-wide thing.
3. The policy gets blocked, because landing a "Project-wide" thing is a hundred times more difficult than landing a repository-specific policy, which is already super difficult when it's about AI.

I specifically asked you about this literally a month ago, just to avoid this situation (a policy being taken over to be converted to Project-wide and thus blocked) from happening yet again.

[jackh726]

Can talk more about this elsewhere but this doesn't resolve the concern for me. Thoughts:

• I would try to be a bit stronger with "solicited" - using the word "discouraged from using an LLM unless they first talk with a reviewer" kind of weakens general point. In my mind, someone coming and making a PR that has LLM-generated code in it, without having first talked to a reviewer, is grounds for an instant-reject. Along these lines, I want to be clear that it's not just any reviewer that can say "yes you can make a PR with LLM-generated code" but that they have to do the review.

• It is not okay to have to ping an entire team to merge any LLM-generated code. This falls under "well-reviewed" and "high-quality" to me: if the reviewer is not confident in the changes, they shouldn't be merging it. We trust our reviewers, and this is not trust.

• I don't completely agree with the examples of "changes that occasionally affect soundness", and I also am not sure I like the word "discourage". Ultimately, I think the two prior lines cover enough. If anything, you could say something like "Changes that can be tricky if done improperly should undergo additional scrutiny by both the reviewer and author"

View changes since the review

[Darksonn]

If this policy was something like Zig's "no LLMs ever", I would agree that it can be seen as setting a tone for the whole Project. But this policy has been very carefully designed to allow LLM usage in various complicated ways, and to avoid making comments on the larger scope discussion topics about AI in general. It does not make either side of the camp really happy, and I don't think that anyone could read it in good faith to be either "Rust is pro-AI" OR "Rust is anti-AI".

The policy bans the number one most important use-case of LLMs in the programming space (writing code), so of course it can be seen as an anti-AI policy. Allowing some other use-cases of LLMs doesn't change that. Reading it that way requires no bad faith.

[Kobzol]

I personally consider writing code to be one of the less interesting use-cases of LLMs in the programming space, so it is definitely not the most important use-case to me; it depends. The policy also does not actually completely ban writing code with LLMs either, in its current form.

In the end, potentially any action done by any Project member can be seen as speaking for the Project. If I (or anyone else) tweet that I vibe-code 95% of my software, people may interpret it as Rust going all-in on AI. If I share that I don't like LLMs and would never use them, some people may interpret that as Rust being anti-AI. All the comments in this public GitHub PR thread can be interpreted positively or negatively by either side.

My point was that this policy is very (almost painfully) nuanced, and it clearly discusses the motivation for why it is being created (which is not simply "LLMs BAD"), and also explicitly says that there likely never will be consensus on this topic across the whole Project. Together with an accompanying blog post, I think that is more than enough to explain that this is a policy to stop certain kinds of contributor behavior in rust-lang/rust, not an be-all end-all policy about a general stance of the Rust Project on AI. I talked about this to put it into contrast with Symposium, which Niko used as a precedent, but I see those two cases very differently, as I tried to explain.

[nikomatsakis]

So for starters, let me acknowledge the frustration I caused in my comment. I get it. I can't blame y'all for being frustrated. I've given your comments a lot of thought. Here are my conclusions.

First, the TL;DR: I will back off from my procedural concern. I agree it's within the rights of the teams that review code on rust-lang/rust to set the policy, what I wrote earlier was too strong in this respect. I disagree with that it's the best way to set rust-lang/rust policy, but I don't think it's improper.

Feel free to stop reading now if you don't want to understand my thinking. Otherwise, here's a lot of words. =)

It seems to be a common assumption that I was trying to block or stall this RFC by saying that I felt it ought to be decided by the Leadership Council. This is not, in fact, the case. It's not obvious to me that a 40+-person list of checkboxes will be resolved more quickly than the LC. My concerns were not about the content. I hadn't read the latest changes at the time--I've since skimmed them--but I had seen that Jack was engaging and I know he and I see eye to eye.

What I was concerned about is the fact that we are, de facto, setting what will be seen as the AI policy for the project without going through the existing project-wide decision making body. I recall somebody saying, in the whole Symposium goal conversation, that if we accepted the Symposium goal they were going to have to explain things to their friends. Well, I feel the same way: when we set this policy, it is going to be seen as the Rust project setting a policy. People are going to ask me about it. Quite possibly, reporters are going to call the Foundation and ask questions, etc. That's not a bad thing: it seems like this policy might indeed be one that I would be happy to defend! However, it seems to me that it'd be better all around for the LC to be involved in making that kind of policy decision.

@jyn514, I don't know if you meant to imply that I was "hiding the actual concerns underlying [my] objections" or simply to note that this was a concern before. I don't feel I was doing that. It is true though that at some point in the discussion that led to this RFC, I simply got overwhelmed and decided to check out. I also don't have a checkbox here, so yeah, it's true that I don't feel "represented" by this decision in particular. I imagine many others feel similar.

If I have any takeaway from all this, it's that I think that we as a project need to think hard about how we want to make hard decisions. I'm impressed @jyn514 that you've had the staying power to drive this RFC forward, kudos. I don't think it should've been necessary. I also don't really feel we've taken the temperature of the project in any meaningful sense. I feel partly responsible: I had plans to do a survey, I even met with some survey experts to discuss how to setup the survey in a way that would avoid leading questions and so forth, but I didn't complete the work. Ah well.

Ultimately, I don't feel this is a good way to make decisions. I continue to think that the correct way forward would've been for the LC to set out a group of people to gather feedback, deliberate, gather data, and come to some conclusions that the rest of us can accept, endorsed by the LC in the end. I think that said group might have reached an interim agreement quite rapidly to ban "slop" contributions or what have you to bring down the temperature, and I would've been happy with that. That said, it feels like that ship has sailed, that's certainly the vibe I'm getting here. So be it. But once the dust has settles, let's talk about how we can approach these sorts of conversations in the future.

(On a more practical level, multi-team FCPs are awfully unwieldy. If the precedent is that changing this policy requires getting 40 checkboxes, I think it's going to be very hard to make any changes. It just doesn't seem like the right way to make this sort of decision.)

(Regarding the comparison to Symposium, I understand @Kobzol that it doesn't seem comparable to you. To me, it does. The way I see it, in a narrow sense, the lang team had the right to decide as a team to liase with the Symposium org. But it's also true that this would've been seen as a move by the project and hence, given the strong feelings folks had, it would've been "impolite" for us to do so as a team without having had broader LC endorsement. And this is why I decided to withdraw the goal. In the same way, I think it's valid to move this RFC forward, I just don't know that it's wise.)

[traviscross]

Suggested change

	- Asking an LLM to summarize comments on an issue, PR, or RFC.
	- Asking an LLM to summarize comments on an issue or PR.

This is a policy for r-l/r. Speaking about RFCs may confuse the reader as to the scope of this policy.

View changes since the review

[traviscross]

Suggested change

	- Using an LLM in the creation of experimental code changes that are not meant to be reviewed and will never be merged but must live as draft PRs on `rust-lang/rust` for tooling reasons, such as to run crater or perf.

I don't believe the policy means to prohibit this, and I believe it'd be better to say this explicitly.

View changes since the review

[jyn514]

Nice catch, added.

[clarfonthey]

I mean, considering how easily we could generate code to run via crater or perf, how expensive this is, and how sensitive it can be to minute changes, I would rather not allow any code to be run in those circumstances that wouldn't be valid to merge normally.

And to be clear, I mean code here. Most of the time, the issue is just that the code needs to be made more generic or documented, not that the code will be substantially different from the end result. If you need to rewrite the entire thing after a perf run, I would say it wasn't a useful perf run, and I only think that should happen due to misconceptions that were found by perf, not just inherently due to the authorship restriction.

[kennytm]

Well it's not like a random user is able to invoke crater and perf, a trusted member must approve first.

PRs not intended for merging can legitimately used for running crater to gauge the effect of a change or gather statistics from the ecosystem, e.g. rust-lang/rust#129604, rust-lang/rust#137044, rust-lang/rust#154887. Also this suggested-change is not restricted to crater/perf either, e.g. one could use draft PRs for testing CI (rust-lang/rust#154969) or bot interactions (rust-lang/rust#155221). So having a "must be valid to merge normally" restriction makes no sense.

[traviscross]

Yes. My motivating use cases are edition testing and lang research, neither of which produce PRs that will ever be merged.

[traviscross]

Suggested change

	- Documentation that is originally authored by an LLM.
	- Documentation that is originally generated by an LLM.

Without changing the semantics, could we search and replace all uses of "authored by an LLM" with "generated by an LLM"? As @xtqqczze has pointed out elsewhere:

Authorship is something that applies to a person, not tools; a LLM can generate text, but it isn’t an author.

Pulling the idea of authorship into this policy seems unnecessarily philosophical for what it's trying to accomplish. The technology is called "generative AI". It'd seem more clear, to me, to stick with "generated".

View changes since the review

[clarfonthey]

jyn specifically clarified authorship as a term here to be specific, so, you can't just change one section and be consistent. It would require at least structural changes to the policy.

[traviscross]

As best I can tell from a careful review, it requires a search-and-replace and then some minor redrafting of the The meaning of "originally authored" section. That section could be redrafted, e.g., as:

This document uses phrases such as "generated by an LLM" to mean "text that was generated by an LLM (and then possibly edited by a human)". No amount of editing can change how the text was originally created; how it was generated originally sets the initial style and it is very hard to change once it's set.

(Of course, there are many ways to redraft it. This is just one way.)

[clarfonthey]

Right, I think it would be fair to change "originally authored" to "originally generated," just, it would have to be a structural change affecting multiple places and not just the one line. For what it's worth, I think that the distinction is pedantic, but will defer to jyn on what the better wording is. I assume that "authored" was chosen intentionally.

[traviscross]

Suggested change

	- Syncing code and documentation into `rust-lang/rust` (e.g., using submodules, subtrees, [josh](https://github.com/josh-project/josh), etc.) from other repositories that do not follow this policy.

It's my understanding this is meant to be allowed, but I don't otherwise see it.

View changes since the review

[jyn514]

Added.

[clarfonthey]

I would say that this is okay since the goal is explicitly not to extend this policy to those repositories, although I would hesitate to just make this a single footnote since there are a few nuances, like needing to resolve merge conflicts. Those specific changes required to merge the subtree should still follow this policy, even if the original repo does not.

[traviscross]

Suggested change

	- ℹ️ This also includes reviewers who use LLMs to discover flaws in unmerged code.
	- ℹ️ This also includes reviewers who use LLMs to discover flaws in unmerged code when they're not entirely certain about those flaws.

Could we narrow this somehow? I routinely run models over the lang nomination queue to prepare for meetings. It's not the main purpose of my process, but nonetheless, the models reliably flag things that I would find anyway, e.g., violations of RFC 0344 lint naming conventions, setting the lint level and report_in_deps in a way inconsistent with our process, etc.

The model is going to have found this flaw in the unmerged code first, and I'm going to have seen that, so I'd read that as falling under this policy. But I'm not looking forward to having to pollute all my comments, e.g.:

declare_lint! {
- UNUSED_FROBINATOR,
+ UNUSED_FROBINATORS,

Please pluralize this name, according to RFC 0344. (An LLM was used in the discovery of this flaw.)

Or could we come up with a scheme for blanket disclosure? Could we disclose once somewhere, e.g., in team, whether all our review comments should be treated as possibly LLM assisted?

I suppose I'd even be OK if triagebot wants to walk around in my shadow posting scarlet letters after each of my comments as long as I don't need to retype this everywhere.¹

View changes since the review

Footnotes

1. That is, I'm OK with this for me. I have deeper concerns about how disclosure affects the Project. See https://github.com/rust-lang/rust-forge/pull/1040#discussion_r3236985179. ↩

[clarfonthey]

I'm a bit confused why you're asking for this clarification since this is under the allowed section. You just don't need to disclose it, from what I can see.

[jyn514]

No, under the rules it currently requires disclosure.

Using an LLM to discover bugs, as long as you [...] disclose that an LLM was used.

[clarfonthey]

Ah, that feels a bit confusing, then. Because the way this list is worded, it looks like it is technically allowed, and probably should be moved down to the caveats section.

I don't think that we should be allowing blanket disclosure. Part of the reason why this policy exists is to avoid complacency, and this is a great way to ensure it.

[traviscross]

The model is going to have found this flaw in the unmerged code first, and I'm going to have seen that, so I'd read that as falling under this policy. But I'm not looking forward to having to pollute all my comments, e.g.:

declare_lint! {
- UNUSED_FROBINATOR,
+ UNUSED_FROBINATORS,

Please pluralize this name, according to RFC 0344. (An LLM was used in the discovery of this flaw.)

Something I should mention here that's probably not immediately obvious to those outside of lang is that (as we do on the lang calls) I'm generally leaving these comments during a live lang meeting. We might go through 25 items in 150 minutes.

That doesn't leave a lot of bandwidth (hence the lang-ops preparation for the calls). That's the context in which I'm asking for a reliable low-cost way to comply with this while being able to leave the comments I need to leave.

[clarfonthey]

I don't think that you need to “pollute” your comments here. I would say, if you're using an LLM to blanket go through all these items, you could easily satisfy this policy by:

1. Having a separate document where all of these details are listed
2. Directly link to the spot that mentions the relevant issue
3. At the top of the main document, make it clear that an LLM was used to generate it

Disclosure still counts if it requires clicking on a link.

[traviscross]

Suggested change

	- An objective concern raised about active harm the policy is having on the reputation of Rust, with evidence, as decided by a leadership council FCP.
	- An objective concern raised about active harm the policy is having on the reputation of Rust, with evidence, as decided by a leadership council FCP.
	- By the leadership council adopting, by FCP, a Project-wide policy and stating in an FCP that it displaces this policy.

In my view, the LC (at any later time) would be within its rights to decide that rust-lang/rust is a shared space and that the drawbacks exceed the benefits, to the Project, of r-l/r having a policy different from that of the Project overall. The LC should be able to do this without having to make an evidentiary finding that the policy is harming the reputation of Rust.

View changes since the review

[clarfonthey]

First, this would be redundant for the first point, since if the LC can just decide whenever to dissolve the policy, then it doesn't matter if they have concerns about active harm.

Second, if you just think the LC can veto any change to how a repository is managed after the teams managing that repo have signed off, without an RFC, I find that at least a little bit concerning.

Third, this is supposed to be a temporary policy, to be replaced by a project-wide policy. (Note that if a project-wide policy sets aside the option for this policy to be tailored by a few specific teams, I would consider that replacing this policy, even if this effectively stays as-is.)

So, from the perspective of this being refined by a larger RFC, I think that it's fair to say that it should only be replaced if there is active and demonstrable harm, not just because the LC doesn't like it. And uh, it's a massive conflict of interest for you to be proposing that to begin with, since you would be one of the people potentially making that decision.

[traviscross]

Third, this is supposed to be a temporary policy, to be replaced by a project-wide policy.

The author has rejected framing this as an interim policy, so I'm not sure this is correct. See #1040 (comment).

[clarfonthey]

I can't comment for what jyn thinks, but to me, the goal here is to not belittle a policy that has already been put in place even if we also agree that policy needs to be put in place elsewhere in the project. The idea isn't that the policy itself is temporary, but rather than the only potential next step would be either:

a) The policy has shortcomings that need to be addressed, or
b) The policy is superceded by a broader one

Neither of these cases are "the LC doesn't like the policy and wants to remove it entirely"

[traviscross]

Suggested change

	- ℹ️ This does not allow reposting the summary publicly. This only includes your own personal use.
	- ℹ️ This does not allow reposting the summary publicly on `rust-lang/rust`.

As written, this could read as a prohibition on posting the summary publicly anywhere. That would be an overreach and is not what I think is meant.

View changes since the review

[clarfonthey]

I think the clarification is fair, but I don't think that removing the point about personal use is fair. The point is to clarify that it's for personal use, and posting it publicly outside the project counts as that still.

[traviscross]

This is a policy for r-l/r, so I'd consider it out of bounds for this to prohibit, e.g., a member of another team using an LLM to construct a summary and then posting that to a Project team space outside of r-l/r for team use. It would stretch the meaning of personal use too far to consider that personal use.

[clarfonthey]

What I meant is:

This does not allow reposting the summary publicly on rust-lang/rust. This only includes your own personal use.

is fine, because posting it publicly outside of rust-lang/rust counts as personal use under this policy, and is therefore fine.

[traviscross]

Suggested change

	- Using an LLM to generate solutions to an issue, learning from them, and then writing a solution from scratch yourself and in your own style.

I'd suggest moving this one to the allowed category (and revising it to mention generating example solutions in the plural, as that's better guidance for what someone should really do).

This is more similar to the other allowed items than to the with caveats items. As with using an LLM to review one's own code, this is a private use. We're requiring that the person write the solution that others will see from scratch. I.e., I read that as asking for independent creation — prohibiting copying in any form (in fact, it might be a good idea to make the language stronger about this, to improve clarity; maybe add "(no copying)" after "from scratch").

Nobody other than the author therefore will ever see these educational materials. Months could pass between the author looking at these examples and writing a solution. We don't demand to know the books or papers the author might have read that contained example code for similar problems.

Demanding disclosure here is a reach into a private space, and so is an overreach.

View changes since the review

[jyn514]

I agree. I've moved this to the "Allowed" section.

[clarfonthey]

This just feels like you failed to take in the rest of the policy and decided to remove a large part of it because you disagree with it. The entire policy effectively clarifies what it considers valid "rewording" of LLM output and what isn't, and so adding a tiny bullet point at the beginning that overrides that and says it's all allowed with no caveats just undermines all that.

[traviscross]

There are no valid rewordings under this policy. As the policy says:

This document uses the phrase "originally authored" to mean "text that was generated by an LLM (and then possibly edited by a human)". No amount of editing can change authorship; authorship sets the initial style and it is very hard to change once it's set.

It then fully bans all contributions that were "originally authored" by LLMs other than those by approved bots, those under the recently added experimental process, machine translation, and trivial code changes. I.e., the other things under the Allowed with caveats section.

The odd one out is this one. The others represent things "originally authored" by an LLM, as the policy defines it. This one is not that. The policy requires that a solution be rewritten from scratch — i.e., it cannot be "originally authored" by an LLM.

[clarfonthey]

So, I guess I wasn't 100% correct on the reasoning here, but the effect is the same: you appear to be failing to take in the policy and deciding to remove a decent portion because you disagree with it.

The point here is that the policy is intentionally conservative when it comes to LLM usage: anything where the LLM could have potentially influenced the output in ways you didn't directly control is included. This is one of those obvious cases. It's like asking a friend to copy their homework; even if you just read a friend's paper and then rewrote your own, it's still fundamentally different than writing your own paper.

And similarly, examples like this exist in code too. For example, there have been multiple cases where code has been leaked and reverse engineers have explicitly refused to read it for legal reasons; simply knowing what was done taints the idea of a "clean-room" implementation. This is an obvious example of that. You can't say that this is "too private;" if we decide that LLMs are our business and you've decided to contribute, you should let us know, just like how you should let us know if you copy-paste code verbatim from another project. Your alternative is to not contribute code, not to hide its source.

[traviscross]

The point here is that the policy is intentionally conservative when it comes to LLM usage: anything where the LLM could have potentially influenced the output in ways you didn't directly control is included.

Included in what the policy allows, without disclosure, are:

• Asking an LLM questions about an existing codebase.
• Asking an LLM to summarize comments on an issue, PR, or RFC.
• Asking an LLM to privately review your code or writing.

Any of these can influence your output — i.e., what you'll send to a reviewer. The LLM can give you broken answers about the workings of the code base, can give you a backward description of what was discussed in the RFC, and in review, can push you away from correct answers and toward wrong ones. I.e., the policy allows you to learn incorrect things from an LLM, without disclosure, that may result in you submitting a very confused PR.

The suggested text says:

• Using an LLM to generate solutions to an issue, learning from them, and then writing a solution from scratch yourself and in your own style.

This, too, is simply an opportunity to learn something from the LLM. You may learn good things, you may learn bad things. That's up to you. You still have to write your own solution.

Learning from things and writing one's own solution is not copying. If it were, then there are many other kinds of learning we would need to prohibit (not just disclose) far more urgently.

[clarfonthey]

The issue isn't that it's copying, though, or whether you learned. The issue is whether an LLM was involved at all. It's a specific case where many people underestimate the effect the tools have on the end result, and so, we ask for disclosure in general to avoid having to litigate the very specific circumstances of what happened to determine whether they need to disclose or not.

It feels like the main issue, which you're explicitly not pointing out, is that you're concerned that people would be judged based upon whether an LLM was involved at all, and therefore, LLM usage should be kept as a "dirty pleasure" in this particular instance. I don't think that it's worth diluting the policy or creating confusion just because people are specifically unwilling to admit they used a particular tool.

[traviscross]

My concern here is that it's inconsistent and an overreach. This would be more consistently placed with the learning rules rather than the original creation rules. It's an overreach in the same way that it'd be an overreach to require disclosure for the other learning rules.

The concerns about disclosure eroding trust in the Project are separate, and I've articulated them elsewhere.

[clarfonthey]

You seem to be intentionally ignoring all of the arguments where LLMs are a very specific case that cannot be generalised. Sure, in a perfectly frictionless vacuum, we would not be asking for disclosure in this particular case. But for a number of different reasons, we're asking this.

Let me come up with an equally hypothetical and unrealistic scenario: imagine if, five years ago, a massive campaign had been undergoing to fill StackOverflow with subtly wrong information to sabotage developers in addition to all the correct information. It would be reasonable to ask all developers to disclose if they had used information on StackOverflow to develop a change in these very specific circumstances. This isn't "overreach," it's a pragmatic desire to correct for potential issues, and simply disclosing that something was involved is not a massive privacy concern.

Of course, I know exactly why people would be uncomfortable disclosing in this scenario, and it's because they don't want to be judged for making a potentially unethical decision by using these tools, although since that line of thought is banned from discussion in this RFC, I will both refrain from making it a part of my argument and insist that you refrain from making it a part of yours.

Simply put, I think that if people wish to not disclose because of its relation to (forbidden topic), then that is an argument that is not suitable for this policy. If you disagree, then you can take a look at my RFC, which explicitly reduces LLM usage even further because of the presence of that argument.

[petrochenkov]

I didn't participate in the LLM-related discussions before, but I'd personally be fine with a wide range of policies, except perhaps most extreme ones like "you must review slop" or "no LLMs at all, even in you free time".
So rubber stamping this policy too.

I'd also be fine with not having a policy and dealing with contributions involving LLMs on case by case basis, using reviewer's judgement, the number of such contributions to rust-lang/rust is not actually large at the moment.

[traviscross]

Suggested change

	If it's clear they've broken the rules, point them to this policy; if it's borderline, report it to the mods and move on.
	Point to this policy, maybe report it to the mods (at your discretion), and move on.

To an outside reader, reporting something to the mods is going to sound like an escalation against the contributor. Given that, it reads a bit backward that the clear violations should only result in the policy being cited while the borderline violations require a report to the mods.

There are many ways this could be redrafted to avoid this. What I'd suggest is leaving whether to report this to the mods to the discretion of the reviewer in all cases and removing the conditional on whether the case is borderline or clear.

View changes since the review

[clarfonthey]

The mods have openly encouraged people to report non-violations so they can keep an eye on things. I don't think that we should be reinforcing the stereotype that communication with the mods is inherently accusatory; we should be breaking that assumption instead.

[traviscross]

Suggested change

	Conversely, lying about whether or how you've used an LLM is considered a [code of conduct](https://rust-lang.org/policies/code-of-conduct/) violation.
	Conversely, claiming to have not used an LLM when you did or stating your use of an LLM as less than it was is considered a [code of conduct](https://rust-lang.org/policies/code-of-conduct/) violation.

I think to be fair to people, this language needs to have a safe harbor. People shouldn't have to worry that they have precisely calibrated their disclosures, i.e., not too high, not too low.

They might have done the work many months ago and may not remember the exact details. It must always be safe to over-disclose — to err on the side of stating that more LLM use happened.

View changes since the review

[kennytm]

the suggested edit is much harder to understand than the original text though

[clarfonthey]

I actually disagree; I would say that lying is a larger burden to meet and would not include people who are mistaken. Instead of weakening the wording, we could maybe clarify that the mods understand the difference between an lie and a misunderstanding.

[traviscross]

the suggested edit is much harder to understand than the original text though

There are many ways to redraft, and I'm OK with one that continues to frame it as lying. E.g.:

Suggested change

	Conversely, lying about whether or how you've used an LLM is considered a [code of conduct](https://rust-lang.org/policies/code-of-conduct/) violation.
	Conversely, lying about not using an LLM when you did, or lying about relying less on an LLM than you did is considered a [code of conduct](https://rust-lang.org/policies/code-of-conduct/) violation.

[clarfonthey]

I mean, I already kind of solved this in my version:

If a user is found to be repeatedly lying about LLM usage (using LLMs in a non-trivial way without disclosing that usage)

It doesn't need to be particularly elegant; parentheses are fine.

[traviscross]

Suggested change

	Using an LLM does **not** mean it's ok to harrass a contributor.
	That a person used an LLM does **not** mean it's OK to harass that contributor.

I think what this means to say is that it's not OK to harass people who used an LLM. It reads to me, though, as saying that using an LLM doesn't excuse the user harassing people, which is also obviously true, though probably too obvious to mention.

View changes since the review

[clarfonthey]

This just feels pedantic, honestly. The new version feels a bit clunky to read, and the original still is understandable imho.

[traviscross]

Suggested change

	- Usages that do not use LLMs for creation and do not show LLM output to another human are likely allowed ✅
	- Usages that use LLMs for creation or show LLM output to another human are likely banned ❌
	- Uses that do not use LLMs for creation and do not show LLM output to another human are likely allowed ✅
	- Uses that use LLMs for creation or show LLM output to another human are likely banned ❌

The more correct word here is uses. The word usage means a customary pattern or habit of use (or a rate or quantity of use).

View changes since the review

[traviscross]

The policy has evolved since this section was written, and I wonder whether it needs to be redrafted in a softer way. It now reads as far starker than the rest of the policy does.

View changes since the review

[traviscross]

I read "without concerns" ambiguously here. Does it mean "it must achieve a simple majority of the members of teams using r-l/r and no concerns must be filed" or that "it must achieve a simple majority of the members of teams using r-l/r without considering concerns to have effect for blocking it."

Either one I could see as having been intended. The first is more similar to how we normally treat concerns, but it's in conflict with wanting only a simple majority, as it actually requires a kind unanimity — any one person can block it. The second would be closer in spirit to a simple majority system, but it's in tension with our usual system of concerns.

Which reading is intended? Probably this should be made more clear.

View changes since the review

[traviscross]

Most of the work lang members do, as a team, is reviewing language proposals made to us in rust-lang/rust issue bodies, PR descriptions, and comments. Here's a random recent one, by way of example:

• Lang proposal: const expressions in #[repr(align)] and #[repr(packed)] and friends. rust#155390

We tend to care about policies for how lang-related issue and PR descriptions are put forward, e.g., the stabilization report template. Though we are not ourselves large contributors of code to r-l/r, we are in the set of maintainers of r-l/r — at least, that's how I see it.

So it surprised me a bit, given that this document sets policy for what's allowed for people making lang proposals to us in r-l/r (e.g., by prohibiting LLM-assisted drafting), that we weren't included on this FCP, though it had been earlier discussed.

I don't know what to do about that. I don't really want to ask here for the hassle of the FCP being restarted. And yet, given what the policy covers, it seems awkward to me that we're not on it — as though I'm commenting here with needs and interests as an outsider. That's how it feels, anyway.

Is there anything we can do about this? Maybe the scope can be narrowed so that it doesn't set policy for lang proposals or documentation items lang owns? Maybe something else? I don't know. @jyn514, what do you think?¹

View changes since the review

Footnotes

1. If you don't mind, I really do want to hear specifically from @jyn514 on this (no hurry). ↩

[jyn514]

I'm happy to narrow the scope so this excludes lang proposals and stabilization reports, so that t-lang can set their own policy. Are there other things in t-lang's purview you would like to see excluded?

[Manishearth]

This is a bit on a tangent, but I think this discussion is also valuable and important to have. I generally agree with @jyn514 about the procedural aspect here, and at the same time, I think 40-checkbox decisions are just ... not really workable.

Feel free to reopen this as a discussion elsewhere if you want to avoid further tangent-ing. Or just hide the comment. I think it's useful for the Rust project to spend some time reflecting on its procedural options here for future such decisions.

Regardless, I do not think the council is the right body to make this decision. Their charter says this:

• Establishing the official position, opinion, or will of the Rust Project as a whole.

  • This helps reduce the need for Project-wide coordination, especially when a long public polling and consensus-building process is not practical - for example, when communicating with third parties who require some understanding of what the Rust Project as a whole “wants”.

We are not communicating with third parties, we are not performing project-wide coordination, and I think you would be hard-pressed to argue that escalating to council would make this whole thing take less time. By contrast, here's the charter for top-level teams:

• Have a purview that is foundational to the Rust Project
• Be the ultimate decision-makers on all aspects of that purview

I think it is very clearly not the intent of the RFC that established the leadership council to allow it to override the decisions of top-level teams.

FWIW, I do think that this is one of the key, deliberate differences between the Leadership Council and Core. Core did make decisions like this, and generally Core was expected to involve everyone in such decisions but not as checkboxes, instead via other mechanisms. I actually think that mostly worked well until the last couple of years of Core where Core was completely swamped with Foundation stuff and other major work items, leading to a bunch of failure modes becoming clear, and people feeling blocked out of these decisions.

Core tended to be "fail closed" for top level decision making: if something felt toplevel, Core expected to handle it by default. LC is "fail open": if something feels toplevel, that doesn't make it LC unless it's one of the explicitly enumerated things, or something everyone is broadly comfortable delegating to LC.

To some extent, I think this change was an overrotation and ended up tearing down a number of Chesterton's Fences: it is valuable to have a team that you can trust to make decisions like this (regardless of whether they make those decisions by default). There's a lot of good in having an entity that is responsible for ratifying good decisions¹, and trusted to do that well, even when it involves heavy compromise. The problem occurs when that trust breaks down, or when the team is broadly unable to do that effectively.

I think this is exactly the type of tricky decision where the Rust project should want to delegate consensusbuilding and final ratification to the toplevel team. I have largely not been participating in the LLM discussions, but from the outside, they look exhausting. From the outside, I am reminded of some of the discussions around the 2018 edition, where everyone and their dog had something to say, the same things got said over and over again, and the conversation ended up being dominated by a handful of very vocal, very passionate people who often ended up talking past each other.

The 2018 edition sucked for the project. The 2018 edition spawned a whole host of governance work, some of which came to fruition, but we were talking about those problems for years.

---

I would love for the Rust project to be in a place where there is a small team that can be trusted to organize an information gathering process², take in all of the input, conduct scoped discussions, iterate on a proposal, and finally make decisions on it, while keeping the process open and legible.

From what I see here, that doesn't seem to be the case currently, but I would urge people to view this as an aspirational goal for project governance and work towards it by building that trust. I don't want to say they should be doing that by default, but I would love to see a situation where when team ratification leads to too many cooks, everyone feels comfortable delegating to a smaller, trusted group. That group doesn't even have to be the council; but having the council be trusted enough to fill that role in 99% of cases is a valuable property.

Lack of trust has been a persistent theme in Rust's governance problems.

Footnotes

1. This blog post is about executives but I think it applies pretty well to toplevel leadership in many kinds of organizations. I like it a lot and recommend folks go through it. ↩

2. I know some of this happened, but I'm noting that the trust isn't there. ↩

[nikomatsakis]

Overall my take is that this policy in broad strokes is good but I find it kind of bewildering and hard to follow. I think people are going to read it and just be overwhelmed. I would really rather we lean into something simple. I think it boils down to this

• LLMs for private use (output not shown publicly) is allowed
• Using LLMs to generate code, documentation or text that you post in a PR or issue is disallowed apart from a specific set of cases.
• We have an experimental program where, if you are you can opt-in to having a reviewer work with you.

I don't think we need to overspecify how the experimental program works. I'd lean into mandatory disclosure (enforced by a bot that puts your PR into a draft state if it's not present, say) and ensure that LLM-authored PRs are not auto-assigned in the normal review queues. Failure to disclosure LLM-authored code is a moderation violation, resulting in a closed PR and rapid escalation for repeated cases.

I'd probably want to agree now that we'll check in periodically on how people are feeling. Maybe we ask compiler + libs-api leads to take in comments in all directions (positive and negative). I don't know, something like that. I would hope that we can update the policy as we gain more experience without a massive FCP.

To me, the list of conditions ("not in safety critical code", etc) is overkill and kind of insulting. It signals a lack of trust in your fellow team members to be able to review code. I realize that it makes others feel comfortable, but I think it comes at a severe cost, and it'll be something that ages fairly poorly.

View changes since this review

[nikomatsakis]

Suggested change

	#### ✅ Allowed
	The following are allowed.
	- Asking an LLM questions about an existing codebase.
	- Asking an LLM to summarize comments on an issue, PR, or RFC.
	- ℹ️ This does not allow reposting the summary publicly. This only includes your own personal use.
	- Asking an LLM to privately review your code or writing.
	- ℹ️ This does not apply to public comments. See "review bots" under ⚠️ below.
	- Writing dev-tools for your own personal use using an LLM, as long as you don't try to merge them into `rust-lang/rust`.
	- Using an LLM to discover bugs, as long as you personally verify the bug, write it up yourself, and disclose that an LLM was used.
	Please refer to [our guidelines for fuzzers](https://rustc-dev-guide.rust-lang.org/fuzzing.html#guidelines).
	- ℹ️ This also includes reviewers who use LLMs to discover flaws in unmerged code.
	#### ✅ Allowed
	- Any usage of an LLM where you are the only one that sees the output, e.g.
	- asking an LLM questions about an existing codebase.
	- asking an LLM to summarize comments on an issue, PR, or RFC (but not reposting that summary)
	- asking an LLM to privately review your code or writing.
	- writing tools and code that you do not attempt to merge into `rust-lang/rust`.
	- Using an LLM to discover bugs, as long as you personally verify the bug, write it up yourself, and disclose that an LLM was used.
	Please refer to [our guidelines for fuzzers](https://rustc-dev-guide.rust-lang.org/fuzzing.html#guidelines).
	- ℹ️ This also includes reviewers who use LLMs to discover flaws in unmerged code.

[nikomatsakis]

Is this equivalent?

[nikomatsakis]

It's not clear to me what this "team" language refers to here, as this is the rust-lang/rust policy

[nikomatsakis]

Using LLMs to generate code, documentation or text that you post in a PR or issue is disallowed apart from a specific set of cases.

I'm actually not sure -- is this what the policy says? If you are an experienced dev, are we saying that in that case it's ok? (I think not.)

e.g., if I was working on a PR with an LLM, presumably I would go through the same experimental process?