Exploring Solutions to Tackle Low-Quality Contributions on GitHub

[moraesc]

Hey everyone,

I wanted to provide an update on a critical issue affecting the open source community: the increasing volume of low-quality contributions that is creating significant operational challenges for maintainers.

We’ve been hearing from you that you’re dedicating substantial time to reviewing contributions that do not meet project quality standards for a number of reasons - they fail to follow project guidelines, are frequently abandoned shortly after submission, and are often AI-generated. As AI continues to reshape software development workflows and the nature of open source collaboration, I want you to know that we are actively investigating this problem and developing both immediate and longer-term strategic solutions.

What we're exploring

We’ve spent time reviewing feedback from community members, working directly with maintainers to explore various solutions, and looking through open source repositories to understand the nature of these contributions. Below is an overview of the solutions we’re currently evaluating.

Short-term solutions:

• Configurable pull request permissions - This has been a long-standing request (since 2016) and is a first step in giving maintainers more tools to customize their PR experience. Repository owners will be able to control pull request access at a more granular level by restricting contributions to collaborators only or disabling PRs for specific use cases like mirror repositories. This also eliminates the need for custom automations that various open source projects are building to manage contributions.
• The ability to delete a PR from the UI - This provides maintainers with the ability to remove spam or low-quality PRs directly from the interface to improve repository organization.

Long-term direction:

As AI adoption accelerates, we recognize the need to proactively address how it can potentially transform both contributor and maintainer workflows. We are exploring:

• Enhanced permission models - Provide maintainers with more tools so they can customize their PR experience:
  • More granular controls for determining who can create and reviewing PRs beyond blocking all users or restricting to collaborators only.
  • Ways for maintainers to define criteria that PRs must meet before they can be opened.
• Improved triage tools - Potentially leveraging AI to evaluate contributions against project guidelines and standards to identify which contributions maintainers should focus on reviewing.
• Transparency in AI-assisted contributions - Improved visibility and attribution when AI tools are used throughout the PR lifecycle.

Next Steps

These are some starting points, and we’re continuing to explore both immediate improvements and long-term solutions. Please share your feedback, questions, or concerns in this thread. Your input is crucial to making sure we’re building the right things and tackling this challenge effectively. As always, thank you for being part of this conversation. Looking forward to hearing your thoughts and working together to address this problem.

[xavidop]

such a great intiative

[Ovod1s]

Только ИИ нужно после обязательно проверить доверять им сильно ещё нельзя...

[evanskofiassafuah]

Только ИИ нужно после обязательно проверить доверять им сильно ещё нельзя...

Translated 👇🏾
Only AI needs to be checked afterwards, you still can’t trust them too much…

[SunShineHead]

To to lessen the complain of clients using the app itself and minimize the log in ratio from 43 different person per account to at least 22 clients may have their own privacy in using the app itself it is awesome for not knowing that everything was being tracked behind the scene, i'm starting to get curious about the tracking issues as thought it's with your games.

[xavidop]

I know this is a pretty ambitious idea and not trivial to implement, but it would be really powerful to have an AI-detection mechanism with a configurable threshold at the repository or organization level. That way, teams could decide what percentage of AI-generated code is acceptable in pull requests.

Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed.

[moraesc]

Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed.

This is definitely something we’re exploring. One idea is to leverage a repository’s CONTRIBUTING.md file as a source of truth for project guidelines and then validate PRs against any defined rules.

In regards to AI-generated code, have you seen cases where the code is AI-generated but still high-quality and actually solves the problem? Or is it always just something you want to close out immediately? I'm curious if an AI-detection mechanism would rule out PRs where AI is used constructively, but interested in investigating this more and understanding what sensible would thresholds look like.

[PaulNewton]

"AI-detection" is a backwards approach to a human problem filled with false positives.
Any system good enough to actually identify AI is a system that can be used to train undetectable AI and that would encourage MORE spam.
QED

The only actual test that ends up mattering is whether the code works and the contributor is allowed to contribute that code.
And that is the problem, code that doesn't work wasting humans time on slop.
The goal is less AI noise not more insisting on itself at every point in the maintainers life.

[who]

Judge work on its merits and its logic, no matter if the source is human, AI, or a combination thereof.

[xavidop]

As of today, I would say that 1 out of 10 PRs created with AI is legitimate and meets the standards required to open that PR. On 28 Jan 2026, at 18:41, Camilla Moraes ***@***.***> wrote:  Another possible approach would be to define a set of rules or prompts and evaluate pull requests against them. PRs that don’t meet those rules could be automatically flagged or potentially even closed. This is definitely something we’re exploring. One idea is to leverage a repository’s CONTRIBUTING.md file as a source of truth for project guidelines and then validate PRs against any defined rules. In regards to AI-generated code, have you seen cases where the code is AI-generated but still high-quality and genuinely solves the problem? Or is it alwaays just something you want to close out immediately? I'm curious because I'm wondering if an AI-detection mechanism would rule out PRs where AI is used constructively, but that's where we'd want to test this thoroughly and understand what sensible thresholds look like. — Reply to this email directly, view it on GitHub<#185387 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABBWEYEKF6WLNDKE376L3GD4JDYFXAVCNFSM6AAAAACS7B7C7OVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNRTGEZTMMI>. You are receiving this because you commented.Message ID: ***@***.***>

[eli-schwartz]

5. Produces a hash-chained evidence bundle - cryptographic proof of what was reviewed, when, and what the models found. Independently verifiable offline.

I think that for extra security you should make sure that this uses the blockchain distributed consensus model. Otherwise how do we know which proofs to trust?

Eager to hear the name of your new cryptocurrency.

[sirosen]

I don't think it's a scam. I think it's a silly.

You are wading into a discussion which is primarily about harm being done by vibe coding.¹ So, uh... pushing a vibe coded project here is super ironic.

Footnotes

1. In the El Reg article about this discussion, a GitHub manager said that this isn't about LLMs, which is so patently dishonest that it's offensive. ↩

[bxb100]

Damn, MF should hire you as their PM.

[tsjensen]

Deleting a PR would be a great feature, I've looked for this since before AI

[rmacklin]

@moraesc As the OP alludes to it's a legitimate flaw that all prior public issues become inaccessible if a public repository decides to disable issues going forward. Are you open to making changes to this? This discussion has been about a potential upcoming feature to disable PRs for a repository, but I want to explicitly ask if GitHub is open to considering improving the behavior for the existing disable-issues feature, too.

[Mossaka]

Hey! I am from Azure Core Upstream and we have a lot of OSS maintainers who mainly maintain repositories on GitHub. We held an internal session to talk about copilot and there is a discussion on the topic where maintainers feel caught between today’s required review rigor (line-by-line understanding for anything shipped) and a future where agentic / AI-generated code makes that model increasingly unsustainable.

below are some key maintainer's pain points:

1. Review trust model is broken: reviewers can no longer assume authors understand or wrote the code they submit.
2. AI-generated PRs can look structurally “fine” but be logically wrong, unsafe, or interact with systems the reviewer doesn’t fully know.
3. Line-by-line review is still mandatory for shipped code, but does not scale with large AI-assisted or agentic PRs.
4. Maintainers are uncomfortable approving PRs they don’t fully understand, yet AI makes it easy to submit large changes without deep understanding.
5. Increased cognitive load: reviewers must now evaluate both the code and whether the author understands it.
6. Review burden is higher than pre-AI, not lower.

[moraesc]

Thanks for sharing @Mossaka ! Great to get more feedback. This looks along the lines of what we've been hearing from maintainers too, so validates what some of the key pain points are. Were there any feature requests mentioned during the session?

[Tibor17]

1. I have a problem with the following statement. It is very leftish. The GitHub has no rights to impose such limitations. You have to configure your repo and TTL of a PR:
   limited timeframe to delete a PR. Maybe a week in case of low activity

2. Regarding the next statement, you can delete a PR when you close it. How different is close and delete?
   Next question, why you mentioned low-quality PRs? I hope you do not want to employ AI to determine what low quality means! We, maintainers, have to determine it.
   The ability to delete a PR from the UI - This provides maintainers with the ability to remove spam or low-quality PRs directly from the interface to improve repository organization.

3. Who is an external contributor?
   Example, you mean a situation when XLibre developers are removed from Xorg?
   Restrict PRs to collaborators - This provides more granular access control, allowing contributions exclusively from existing collaborators while blocking external contributors.

4. This would segregate the maintainers and we don't want it. We have flags on the PRs to indicate what to do with them.
   If the AI is leveraged to this job, we have segregated the maintainers again in the Open Source. But the rule is that we are all equal, and this might be a problem if one is notified and the second guy is not. You may not totally trust the AI due to finally the result may change without the second guy or the quality would be bad without his opinion since he does not attend the review.
   Improved triage tools - Potentially leveraging AI to evaluate contributions against project guidelines and standards to identify which contributions maintainers should focus on reviewing.

[AlanGreene]

Many of the badges can easily be gamed. I've seen plenty of users who've followed some blog post / YouTube video instructing them on how to gain badges in an automated manner through a provided script and pool of user / bot accounts.

[jamisonbryant]

Wow, that sucks!! I had no idea. I have never really chased the badges, just cracked a half-smile whenever I earn one from going about my day-to-day, but IMO they should be harder to obtain than this, especially with botting!!

[eli-schwartz]

Um...last I checked, labeling, assigning, and closing PRs are "write access" permissions. GitHub repo permissions are already very granular, in the places where such things are implemented. I don't think the proposal to include PRs in this is as big a lift as you're thinking it is.

They certainly are not, and I know this because I'm a member of an org where all org members have triage permissions, and only a dedicated mirroring bot has write permissions to push new commits to the repository. All org members must push to our in-house gitolite repository instead, and the reason for only handing out triage roles on GitHub is to prevent anyone from accidentally pushing to the wrong remote and causing the two to diverge (which would mean that GitHub gets force pushed, which would be disruptive to users).

Issues are disabled because we use bugzilla. PRs are allowed for the sake of users who don't wish to email patches to a mailing list.

I can:

• add various labels to PRs, such as the labels that cause our in-house CI system to bump a PR to the top of the queue.
• add assignees,
• add requests for reviewers,
• manage milestones (even if that particular repo doesn't use milestones),
• close them

I cannot:

• edit the PR title,
• edit or delete comments,
• approve workflows,
• mark my own review comment as needing action after the PR author responded to my review by saying "no" and clicking the "resolve" button (in my not so humble opinion, this is a grave usability flaw in GitHub PRs -- the author of a PR must not be permitted to win in a governance dispute with a project member, nor hide reviews by default in the hope that a different maintainer will not notice that the first maintainer has outstanding concerns)

The permission system in question is a toggle that allows administrators to select whether a user has read-only access, "triage", "write", or admin. I am entirely confused, what you seem to be calling triage that isn't what I just described. It is also funny to hear you describing permissions as ,"very granular" when it is a straight line going up at an angle and each level simply adds additional permissions on top of lower levels -- are you seriously suggesting that there's a GitHub repository role that "granularly" permits users to, for example, merge PRs but not add an assignee or a milestone?

[eli-schwartz]

Wow, that sucks!! I had no idea. I have never really chased the badges, just cracked a half-smile whenever I earn one from going about my day-to-day, but IMO they should be harder to obtain than this, especially with botting!!

Badges are something that GitHub implemented to copy Xbox games, are you really that surprised that they are vulnerable to cheating? :)

The literal definition of those badges is that you have e.g. "got X number of PRs merged", but you can get them by creating your own personal repo and submitting PRs to it, last I checked.

A badge that perhaps means a bit more is the one-off badges that you can get if you have had PRs merged by repositories whose code is in particular NASA missions, as you can't game that without knowing it will exist in advance, and the list of repositories whose code is used in some well known government-funded event is unlikely to include outright script-generated cheats.

[chadlwilson]

I think the message was probably ineloquently phrased and comes from a misunderstanding of the term "write access [to code]", but the intent was probably to say that there's no particular reason that GitHub can't consider someone with a triage role also able to submit PRs even if disallowed from fully external folks?

But the triage role certainly is a mess and unsuitable for actually doing triage right now (not being able to edit titles or descriptions is crazy) and very far from granular for public free orgs (as opposed to enterprise orgs, I believe). That shouldn't have been a heavy lift to resolve either, but it has not been 😅

[sirosen]

An option to limit new contributors to one open PR would be nice.

Just today I had to batch-close several AI generated PRs which were all submitted around the same time.

For this protection, defining "new contributor" is probably not possible to do perfectly. But anyone who has no interactions with a project prior to the last 48 hours seems like a good heuristic. The point is to catch such a user at submission time and limit the amount of maintainer attention they can take up.

For a different type of problem, I'd like to be able to close PRs as "abandoned", similar to the issue close statuses. It's a clear UI signal to the contributor that their work isn't being rejected but I'm not going to finish it for them. Several of the low quality contributions I have handled, dating back to before the Slop Era but getting worse, are simply incomplete and need follow through.

[ShaunaGordon]

The limit could be configurable, and maybe default to one? Or have a soft limit of one and a hard limit of three or something? Leaving it open and just nudging doesn't really solve the problem this solution is looking to solve, but there definitely does need to be some flexibility in how it's implemented.

[yurishkuro]

We just implemented & automated the open PR limits for new contributors, but we could only do this with labels. Having an actual hard block for opening PRs enforced by GitHub would be much more useful.

TLDR:

Your limit of simultaneous open PRs is based on your history with this project:

Merged PRs in this project	Max Simultaneous Open PRs
0 (First-time contributor)	1
1 merged PR	2
2 merged PRs	3
3+ merged PRs	Unlimited

There was another harm from these auto-generated PRs - we're in the middle of LFX Mentorship application period and we created a number of "bootcamp issues" for applicants to try contributing to projects. It was extremely disappointing when a couple of people just sent PRs to all those issues in a single batch instead of letting others to try.

[PaulNewton]

But anyone who has no interactions with a project prior to the last 48 hours seems like a good heuristic.

As a setting for big projects maybe.
Otherwise this is a false positive out the gate, that makes contributions harder by literally requiring an introduction system; or finding the secret word in the README(that ai will pick up anyways), or other.
And what counts as an interaction, random comments?, a catch-22 PR they can't submit without previously submitting another PR?, an #intros discussion thread buried somewhere... etc etc

You do not want to line up being a culture of "join discord X and present yourself before contributing." as a default.
Nor should the ladders be pulled up on new learners even more because of AI more than is already happening by baking anti-AI behavior into the processes and tools that will catch lots of curious humans in the nets this would cast

I'd like to be able to close PRs as "abandoned", similar to the issue close statuses. It's a clear UI signal to the contributor that their work isn't being rejected but I'm not going to finish it for them.

Is more UI doodads the solve here,
Isn't marking PR's closed abandoned not already a solved problem by closing the PR, applying labels, comment on the PR, or title edits, etc ?
Does this boil down to 1 button that does some mix of the things above.

---

A theme of alot of these posts seem to be of the "must be this tall" variety which seems more like a platform ranking system than just being able to turn off or gate PRs.

[sirosen]

And what counts as an interaction, random comments?, a catch-22 PR they can't submit without previously submitting another PR?, an #intros discussion thread buried somewhere... etc etc

I was saying that I would like to be able to limit new contributors to 1 open PR. The problem -- as stated -- is that a single user can hammer a project with 5-10 PRs and there's no interaction limit to keep them from doing that.

Being able to say "hey, before you have a merged PR, we only allow you to open 1 PR" reduces the amount of manual cleanup we need to handle.

Is more UI doodads the solve here,
Isn't marking PR's closed abandoned not already a solved problem by closing the PR, applying labels, comment on the PR, or title edits, etc ?

"The solve" makes it sound like there's some singular solution here. I don't think there is.

I try to communicate very clearly with contributors, especially when rejecting their work (or "work").

It takes time and energy to explain why things are being closed. If I could delegate part of that effort to a close-status, I would find it helpful. If you have a well indexed library of response templates, that's another way to make it easier.

Is it the solution to slop? Obviously not. That would be a pretty ridiculous position to hold.

A theme of alot of these posts seem to be of the "must be this tall" variety which seems more like a platform ranking system than just being able to turn off or gate PRs.

I was intentional about phrasing my idea as something which would turn down the volume on interactions with new contributors without forbidding them entirely.

"You haven't contributed to this project before and now you are filling 30 PRs" is not a likely usage mode for a normal human on GitHub. If you open your 5th PR and there aren't even comments on your other 4, it's a near certainty that something is wrong.

I don't see how wanting a protection against this form of abuse is even remotely similar to a reputation/ranking system.

[who]

Giving repo owners the ability to "rate-limit" PRs from contributors is probably a good idea.

You can automate this today using on: pull_request_target GitHub Actions.

[dgoldman0]

For the long term horizon: Implement a reviewer LLM that first does an initial scoring of the PRs? Critique is far easier than creation of a correct result. That automated pre-moderation should give the edge needed to handle. Depending on whether you just use rich prompting or fine-tuning, you can even start building an "oracle vox" for your project, which acts as a reasonably informed, reasonably on point virtual representative for the project/organization.

[chadlwilson]

Yes, I understand and broadly agree with your sentiment - but without going into a long sidetrack into the ethics (and irony) of coding-LLMs actively undermining the very thing that even made them possible in the first place (open source code and contributors) ...I wanted to add a different voice with perhaps some more nuance,

In the sense that different tools can be applied to different jobs with different trade offs. Don't need a Ferrari to get to the grocery store, and all that - and if the tool can bring time back for actually fixing bugs or improving the product, working with well meaning contributors - I'm not as fussed with whether it incorporates AI or not, just as whether I don't mind much whether a tool is built in Python or C or Ruby if it works well relative to its costs/downsides.

[eli-schwartz]

Essentially heuristics defined in clear text rather than a classic deterministic scoring system. I don't think that would be that bad (and perhaps such things already exist) - and neither more or less impersonal as the bots in some OSS projects that nag you to sign CLAs; sign off commits; tell you when a merge conflict has appeared, close stale issues, respond and label based on PR templates /areas of code touched etc.

I will say, that I'm extremely opposed to bots in OSS projects that ''close stale issues" and I've never once seen it work out to the mutual satisfaction of both the maintainers and the community. Sometimes it doesn't even work out to the satisfaction of a single side -- e.g. a single developer forces through the bot and angers other developers.

CLA bots and signoff bots aren't heuristics based, any more than something like os.path.exists() uses heuristics to determine whether a file exists. People are attuned to treating such bots as expressions of inarguable fact.

I haven't actually seen bots that tell you when a merge conflict has occurred, although it's much the same (and effectively just serves as a notification pipeline for when the forge internal flag for displaying a merge conflict in the Web UI, changes status). It sounds incredibly painfully noisy and I would object to such a bot in projects I contribute to, but not because I think it's "impersonal", I just think it's far too prone to generating hundreds of new comments that anyone seeking to read the PR discussion will have to scroll through. Generating a new comment for a frequently changing status flag is not an effective communication channel; better would be a forge native feature to email the author when a merge conflict occurs, and then people can also see the summary at the bottom...

respond and label based on PR templates /areas of code touched

Labeling based on PR templates doesn't require a bot, last I checked. GitHub natively lets you associate templates with labels? Labeling based on paths modified in the diff is not really something I'd call a "heuristic" given it is a simple key/value mapping from paths to labels, manually defined in some script that is then reproducible and idempotent.

So none of this is really anything that I feel would map well to give a hint about how users might respond to a heuristics based LLM review tool.

Especially keep in mind that people who don't like LLMs (such as me) will absolutely flip out any time such a tool erroneously gives them a failing grade and "penalizes" them. And a review tool that doesn't do anything to hide PRs with a failing grade from developer attention, does not accomplish anything to solve the problem of developers being forced to expend their attention on what they consider spam.

and if all comments added have an appropriate disclaimer/disclosure and ability for a contributor to "get help" with a bit of friction if it goes awry - that could perhaps be a useful piece of the toolkit?

A disclaimer/disclosure will do nothing because the victim knows that the project chose to impersonally:

• act without knowledge,
• use the tool that they said is bad, to leap to the incorrect conclusion that the victim is using the exact same tool and penalize them in some manner for using it

An appeal button will not help because people have a natural aversion to challenging authority and this goes quadruple for cases where a first time contributor feels slighted and "on the defensive" from the initial interaction and doesn't have some motivation to get the change in "at all costs and no matter what I have to swallow to do it" (such as a critical work-blocking bug that cannot be worked around in a dependency that cannot be replaced or reimplemented).

[marcindulak]

I will say, that I'm extremely opposed to bots in OSS projects that ''close stale issues" and I've never once seen it work out to the mutual satisfaction of both the maintainers and the community.

There is currently at least one running example of this problem anthropics/claude-code#16497

[chadlwilson]

Yes, I am mainly referring to heuristics where a bot (AI or otherwise) tries to use various aspects of a contribution and the contributor's history to assess signal vs noise for a contribution - not existing deterministic bots doing simple automation, or "rules" for contributions.

I'm aware templates can be used for labels, but some projects use heuristic based bots for this (e.g grpc, many others) depending on how they use labels.

I wasn't making any particular judgment on the merit of all these various bots, nor claiming that an AI agent should be the only tool available - just highlighting that there are many things you might want to do in terms of automation, but they all require setup and configuration, and in some cases exposing your project to supply chain risk for that bot due to the privileges they run with.

If people want to block all AI contributions they should be able to do so, sure, but existing contribution prioritisation is opaque for the vast majority of open source projects anyway, so a signal visible only to maintainers is no worse than the current situation of opacity.

I'm in no way claiming that this could be suitable for all projects - no tool (AI or otherwise) would be.

[PaulNewton]

That assumes you have the resources to evaluate every contribution to determine if it is something you don't need/want.

AHAHAHAAHAHAAH haha ahuehue heu omg I can't.
That's literally the point of this entire discussion, NOT having the resources.

Humans don't have infinite time, and LLM's do NOT have infinite energy resources.
Even if you are the type to let AI go at full rip you will burn time somewhere in some sort of review, backfilling, or defending against security breaches etc etc etc ad nauseum; all the while turing your time into a free training vector for other businesses software.

Productivity gains cannot be infinite in a mortal world using probability distributions.
We need deterministic tools for human to human contribution.
Yeeeesh.

[PraiseTechzw]

This is a very real problem, and I appreciate that it’s being treated as systemic rather than blaming maintainers or contributors individually.

One concern I have with repo-level PR restrictions is that they may disproportionately impact first-time contributors who do want to engage meaningfully but don’t yet have collaborator status.

Personally, I think the most promising direction here is criteria-based PR gating rather than blanket restrictions things like required checklist completion, passing CI, linked issues, or acknowledgement of contribution guidelines before a PR can be opened.

On AI usage specifically, transparency feels more scalable than prohibition. Clear disclosure combined with automated guideline checks could help maintainers focus on high-intent contributions without discouraging responsible AI-assisted workflows.

Looking forward to seeing how these ideas evolve especially solutions that preserve openness while respecting maintainer time.

[moraesc]

One concern I have with repo-level PR restrictions is that they may disproportionately impact first-time contributors who do want to engage meaningfully but don’t yet have collaborator status.

Completely agree with this concern and just want to emphasize that this is just a starting point and not the final solution. The plan is to develop more tools and frameworks for both maintainers and contributors to help address the "slop" problem.

Personally, I think the most promising direction here is criteria-based PR gating rather than blanket restrictions things like required checklist completion, passing CI, linked issues, or acknowledgement of contribution guidelines before a PR can be opened.

Some other ideas we're exploring are requiring an issue to be created first before a user can open a PR or leveraging AI to validate a PR against rules defined in a CONTRIBUTING.md file or something similar. Any thoughts on that?

[robmen]

Some other ideas we're exploring are requiring an issue to be created first before a user can open a PR

Requiring that an issue be assigned to the user before they can post a PR would work well with our process. We already request that users who open a PR also open an issue so it can be triaged into the appropriate milestone. It would be great if GitHub had a way to normalize that workflow.

[taranion]

I like the idea of "issue first".
Allow PRs

• either from project members
• or for existing issues that the maintainer manually put in some kind of "expecting PR" state.

Potentially make the order irrelevant. Have PRs from non-members in some kind of "hold" state, until the issue has been opened and accepted. Close PRs that link to rejected issues. Auto-close PRs in "hold" state from non-members that do not link to an issue after a configurable period.

[funnelfiasco]

Thinking along the lines of the discussion first approach that Ghostty uses, I think one way to create just enough friction would be to have an opt-in where a PR has to be linked to an open issue or discussion topic. So when an unprivileged (i.e. does not have elevated privileges on the repo) user tries to create a PR, there's a required field that takes an issue/discussion number. If that's not provided (or the corresponding issue/discussion is closed), then the PR can't be created.

This could be trivially worked around by throwing in any old issue/discussion (or by creating one), but it may cause just enough friction to help. To guard against this, perhaps maintainers could set a "minimum age" for the issue/discussion (e.g. 12 hours) to prevent creating fake issues to support a spammy PR.

[moraesc]

@jamietanna does the discussion-first approach that you follow require all users to create a discussion first, or only "non-collaborators" (users without write access) ?

[jamietanna]

@moraesc generally non-collaborators. The maintainers and collaborators are able to create an Issue and remove the needs-discussion label that auto-closed an Issue raised by a non-collaborator

That being said, non-collaborators will often go through the Discussion process, as it's pretty good at distilling what we want, and a place to work through ideas before they're in an Issue

[marcindulak]

The problem with github discussions is that they are isolated - they don't automatically create links https://github.com/orgs/community/discussions/2971 like issues do, meaning information is scattered in two places and difficult to find.

[weisdd]

The issue with this approach is that pretty much any popular project would already have enough of pre-existing issues/discussions to choose from.

In our case, we've noticed that a good chunk of AI slop was targeting old issues with the "good-first-issue" label. - They're attractive in a sense that, by definition, it's not something complex to do, so you don't have to spend much time prompting AI and, thus, can generate dozens or even hundreds of PRs a month (we've seen such accounts) across a random list of popular repositories (automated tooling makes it easy to find a target). We ended up removing the label from all issues.

[PaulNewton]

Agree but the permissions names might need a different messaging:

So when an unprivileged

Try this optics nit on for size:
"The unprivileged cannot code on github. ; unless their AI is good enough to pass."
Only being partially sarcastic in order highlight while less privilege from a security standpoint is 100% valid there are some very real problems in how we phrase permissions with AI in the mix reallllly muddying things more and more while also wanting/needing new people to join in.

There is non-zero chance for this slop problem to grow into an economic/cultural divide in haves good AI's vs have no-AI's.
Making under-privelaged excuse for some odd twisted ladder pulling on a "social coding" platform; that could weirdly encourage more AI use by any possible contributor just so they can escalate privilege's making the actual learning seem more and more less useful.
Instead of security/spam prevention tools for maintainers to vet new contributors who do want to learn but can no longer afford the entry fee.
ugh 🤮 .

[4thana]

Maybe something like this could be a starting point: https://github.com/mitchellh/vouch

See also the announcement here: https://x.com/mitchellh/status/2020252149117313349

[dkargatzis]

We’re seeing more repos auto-closing PRs lately, not as a stance against contribution, but as a way to survive review overload.

What keeps coming up—both in OSS and internal platform teams—is that the problem isn’t low-quality code, but low-context change. AI has made it trivial to generate syntactically correct diffs, but the signals maintainers rely on to reason about impact haven’t scaled with that volume.

In high-traffic repositories, that shows up as:

• PRs without a clear linked issue or problem statement
• Diffs that technically pass CI workflows but don’t align with the intent described
• Missing ownership signals (e.g. CODEOWNERS, unclear reviewer expectations)
• Test coverage that exists, but doesn’t actually validate the change being made
• Issue–diff mismatches that force reviewers to reconstruct context manually

At that point, review stops being about correctness and becomes a context-reconstruction exercise, which is where the real cost explodes.

What we’re experimenting with in Watchflow is intentionally defensive, not prescriptive:

• surfacing missing or weak issue linkage early
• checking that PR titles and descriptions align with the actual diff
• reinforcing ownership and reviewer expectations maintainers already rely on
• using deterministic validators for most checks, not more generative AI
• running in analysis mode by default, so nothing is blocked without intent

The goal isn’t to auto-reject contributions or “out-AI” AI-generated noise. It’s to reduce the number of low-context PRs that ever land in a maintainer’s inbox—and make the remaining ones cheaper to reason about before human attention is burned.

There’s a preview setup at https://watchflow.dev where you can try rules in analysis mode before enforcing anything.

For anyone curious, I wrote up the reasoning in more detail here: https://medium.com/@dimitris.kargatzis/the-ai-flood-is-breaking-oss-maintainers-are-hitting-the-limit-30c41247db5a

[MuhammedSinanHQ]

1. Mandatory PR entry gate

Before a PR is even created, require:

Linked issue

Checklist completion

Affected areas

Test evidence

Form-based PR creation + required fields.

If any field missing → PR creation blocked.

2. Zero-trust for first-time contributors

First-time contributor PRs:

Auto-labeled needs-triage

Auto-assigned to bot queue

No maintainer ping

Human review only after automation passes.

3. Automation-first triage

Run a fast bot pipeline:

Lint

Tests

Docs build

License header check

Commit message format

If any fail → PR auto-closed with explanation.

Not commented. Closed.

4. Reputation-weighted permissions

Score contributors by:

Accepted PRs

Reverted PRs

Time-to-fix feedback

Thresholds:

Tier 0 → PRs require passing all gates

Tier 1 → Skip basic checks

Tier 2 → Direct PR + reviewer assignment

This removes blanket collaborator-only models.

5. AI-assisted contributions must self-declare

Checkbox:

“I used AI to generate part of this PR”

If checked:

Require explanation of what was generated

Require manual verification notes

Undeclared AI usage discovered later → contributor cooldown.

6. Cooldown system

Accounts that:

Abandon PRs

Spam low-quality patches

Get:

Temporary PR block

Increasing backoff windows

Automated. No maintainer involvement.

7. Maintainership load dashboard

Expose:

PRs auto-closed by automation

PRs reaching human review

Median review time

Optimize pipeline based on data, not anecdotes.

Bottom line

Quality problems are systems problems, not people problems.
Gates > guidelines.
Automation > moderation.
Reputation > blanket restrictions.

[amkisko]

Formatting is the thing for human beings, for robots I am not sure if it matters. In public space format and style always meant the thing and always led to consequences, so it is authors decision and consequences to use or not exact format and style. The public space resonates accordingly.

[amkisko]

Everyone has a right for apathy, dealing with garbage is just a fact that it belongs to everyone and we have to do something with it — no matter synthesized or produced with typewriter or pen and paper.

[chadlwilson]

More discerning readers will probably find I've specifically defended responsibly disclosed use of AI/LLMs in open source in general, as well as a possible part of a solution to this problem in a number of places [1] [2] [3]; including use for those whom English isn't their native language in [1] as part of my very first comment:

there are many good reasons people might in "good faith" put their messages through an LLM

It's rather difficult to navigate big discussions like this with GitHub's UX and thread folding, but perhaps there is some confusion with someone else; as there are many more critical of AI/LLM contributions than I. 🤷‍♂️


1. https://github.com/orgs/community/discussions/185387#discussioncomment-15685251

2. https://github.com/orgs/community/discussions/185387?sort=new#discussioncomment-15707492
3. https://github.com/orgs/community/discussions/185387#discussioncomment-15701670


[sirosen]

First, @DNYoussef, please. Please. Chill.
I am asking you to take a step back and ask yourself why comments which are not highly emotional are provoking such a strong reaction.

I'm not going to respond to everything you've said but oh wow I disagree with you on so many levels it's hard to even put into text. For example,

I'm sorry you're so easily triggered that you can't separate ideas from the style and format they are presented

I'm sorry but what? You can't always meaningfully separate ideas from the form in which they are presented.
Not everything needs to be prose, but it does need to be clear what the ideas even are. And the burden of making ideas clear falls on the writer or speaker in larger measure than it does on the reader or listener. That's just how communication works.

How do you know he's not an immigrant whom English is his second language and he has to use AI to translate. That's literally the first assumption I made

?

I see no evidence that this was text typed in a non-English language and then fed to an LLM for translation. You seem to be making enormous -- and enormously generous -- assumptions on behalf of one person, and equally enormous -- and enormously prejudicial -- assumptions about someone else.

The original assessment by @chadlwilson seems to be accurate. It's copy-pasted output from an LLM, based on the formatting alone. It's written

• without disclosing that an LLM was used
• without any care to format the content to be readable and digestible
• without any attempt to distill out clear ideas

It's not an attack to suggest that low-effort copy-paste from an LLM is low-effort copy-paste from an LLM.

If there's some hidden gem in the original post, then the poster should have shared that instead of the full output of their tools. Just like I put effort into bug reports and only show the relevant trace, rather than a billion lines of logfile data.

And before you even bring up the impacts of English being a second language, I'll just head you off by noting that I've had plenty of positive interactions mediated by machine translation. That doesn't mean I need to put on blinders and intentionally not identify junk simply because it is also mediated by a machine.

I've been looking all over this thread and you've been commenting everywhere with nothing but negativity and not offering any useful critiques to anyone. I haven't seen a single post for people are like "thank you so much for the interaction you've definitely given me a lot to think about".

I think, based on this comment -- and I mean this genuinely and kindly -- that you have gotten too used to the conversational habits of LLMs. I have had several positive interactions with @chadlwilson in various threads here, but I have never felt the need to give a directly complimentary reply of this kind. The positivity of these interactions is implicit in the earnest and respectful exchanges of ideas.

(And I am still thinking about that thread about reputation systems! I need to think about what I think, but rep systems worry me.)

You don't always need to say, in so many words, "thanks for taking the time and effort to discuss this".
However, you do mention that he has been posting a lot... excellent segue.

---

@chadlwilson, I've unsubscribed from this discussion, and I encourage you to do the same. I came back for one final check before I intentionally forget about it. It seems that most everyone serious has already said their piece, and now it's primarily attracting trolls and spam. I think you're getting XKCD 386-ed into a lot of discussions that aren't worth your time and energy.

[katepolak]

I have skimmed through all the top level comments in here and haven't seen this suggusted yet, sorry if I missed it somewhere.

In addition to requiring a linked issue before creating a PR, there could be a special "label" for issues that marks them as "ready for PR". This way the maintainers could effectively throttle the amount of incoming PRs, by the amount of opened issues marked with said label.
Of course in a way this just moves the issue to the issues tab, where people and "people" would spam those instead, hoping to get approved for PR, but at least this way the PR tab should be cleaned up.

[realaaa]

great discussion ! subscribing and will watch this space

some sort of reputation like scheme is probably needed here

• but in a much better way than say it was implemented on Stack Overflow I remember back in the days how many times I tried to upvote and comment
• and just could not - due to being new account there etc
• so eventually I just gave up - and never contributed much, they made this barrier a bit too high (even though I was a perfect candidate to start getting involved)
• now they are lowering it - but yeah a bit too late

another thing which is most likely needed also

• clear way of showing who is the manual / human account
• and which accounts are bots / tools etc
• this one is tricky yeah - as of course many people are running automations from their main account
• but honestly to be fair - human own input must be clearly visible, as opposed to any automatically generated / assisted input

my 2 cents / kopecks

PS or we can just clone Linus and let him deal with AI bots his way - https://lore.kernel.org/lkml/CAHk-=wgnRQiKqWVrO_uF1btYM2K8r8xL95RGdKU3QLe8B58nrw@mail.gmail.com/ - however we then may need AI therapist to heal their hurt ego + PTSD :)

[pinkforest]

Generally I would love to see new contributors opening an issue first and where the contributor gets assigned to work on something by the maintainer/s.

There are cases where fly-by contributions can be immensely valuable and often it is best done in multiple small PRs - issues would serve the gating where as trying to encourage people to slam one big PR over several small ones will backfire.

Most problematic slop contributors drop multi-thousand PR in your repos blowing everything up through single PR with little thought so I would use gating like getting assigned in the issue to get past the threshold of being known contributor - or saying "hello" to the maintainers, collaborators and other contributors.

Requiring an issue first also fosters a culture where people negotiate a bit first to get things done.

[eli-schwartz]

If slamming one big PR over several small ones is against your contribution guidelines then at least that seems relatively easy to respond to by saying "sorry, this goes against the policy at taps the sign" without further comment.

It is then up to the user to resubmit correctly -- or if it's a slop contributor, giving up because they cannot figure out how to open a correct and rules-abiding PR...

[svrnm]

What a great discussion and I heard and read a lot of good ideas already. One thing that I am missing in all of this, is what plans there are in regards of "Building community, not just walls" as outlined in the blog? The features I see are for building walls, that we as maintainers unfortunately need, at the same time it would be great to get more things to build communities:

• Managing large scale communities at GitHub is painful, people fall back to terraform providers and other tools to manage that.
• Same for understanding who is how active inside your org, and for example might be left out from climbing the ladder, or who is a org member or team member and hasn't been active for an extended period. All solvable by github actions, but something I would love to see out of the box.
• "GitHub still leans heavily toward code authorship" -> YES! Any plans to help with that? As stated in the blog post, having those valued more and providing a trust signal would be valuable.

[Wenbobobo]

Why not give the choice to the users?
If an AI branch could be separated, allowing AI to submit, review, and merge itself, this traffic diversion might ease the conflicts. Then let users vote with their feet, and there would be less arguing.
GitHub hasn't been updated for too long; it should upgrade to adapt to the revolution brought by AI coding agents.

[moraesc]

Hey everyone, thanks for all the feedback you've been providing here. We're reviewing it carefully and really appreciate all your questions and ideas.

I wanted to share an update that we just released two new repository settings to limit pull requests to repo collaborators or disable them entirely. You can checkout the changelog or community discussion for more details.

As always, feel free to drop a comment if you have any questions about how these settings work for feedback on how they impact your projects and community.

[yesoreyeram]

I would like to have optional but configurable rules for PRs such as

• Minimum files changed ( optional: can be tricky for changes such simple typo. Because we don't want to have hello world changes.. )
• Maximum files changed. Because we don't want huge PRs in certain repos
• Maximum lines changed in total. Because we don't want someone to abuse the rules with lots of changes.
• Maximum lines changed per file. Same as above.
• Reject markdown only changes. Sometimes we needed this.
• Reject PRs without new tests for code changes. To maintain code quality.
• Reject PRs without associated issues. Optionally the issues must have certain labels. This will allow maintainers to validate the issue and approve PRs. This will also make sure there are PRs only for valid cases.
• Auto approve certain workflows to run to do basic sanity testing. This will be powerful to make sure certain checks are passed without maintainers to validate things manually. ( Certain repos have settings to disable actions for external contributors )

[mao19901987-lang]

Hey everyone,

I wanted to provide an update on a critical issue affecting the open source community: the increasing volume of low-quality contributions that is creating significant operational challenges for maintainers.

We’ve been hearing from you that you’re dedicating substantial time to reviewing contributions that do not meet project quality standards for a number of reasons - they fail to follow project guidelines, are frequently abandoned shortly after submission, and are often AI-generated. As AI continues to reshape software development workflows and the nature of open source collaboration, I want you to know that we are actively investigating this problem and developing both immediate and longer-term strategic solutions.

What we're exploring

We’ve spent time reviewing feedback from community members, working directly with maintainers to explore various solutions, and looking through open source repositories to understand the nature of these contributions. Below is an overview of the solutions we’re currently evaluating.

Short-term solutions:

• Configurable pull request permissions - This has been a long-standing request (since 2016) and is a first step in giving maintainers more tools to customize their PR experience. Repository owners will be able to control pull request access at a more granular level by restricting contributions to collaborators only or disabling PRs for specific use cases like mirror repositories. This also eliminates the need for custom automations that various open source projects are building to manage contributions.

• The ability to delete a PR from the UI - This provides maintainers with the ability to remove spam or low-quality PRs directly from the interface to improve repository organization.

Long-term direction:

As AI adoption accelerates, we recognize the need to proactively address how it can potentially transform both contributor and maintainer workflows. We are exploring:

• Enhanced permission models - Provide maintainers with more tools so they can customize their PR experience:

  • More granular controls for determining who can create and reviewing PRs beyond blocking all users or restricting to collaborators only.

  • Ways for maintainers to define criteria that PRs must meet before they can be opened.

• Improved triage tools - Potentially leveraging AI to evaluate contributions against project guidelines and standards to identify which contributions maintainers should focus on reviewing.

• Transparency in AI-assisted contributions - Improved visibility and attribution when AI tools are used throughout the PR lifecycle.

Next Steps

These are some starting points, and we’re continuing to explore both immediate improvements and long-term solutions. Please share your feedback, questions, or concerns in this thread. Your input is crucial to making sure we’re building the right things and tackling this challenge effectively. As always, thank you for being part of this conversation. Looking forward to hearing your thoughts and working together to address this problem.

[mao19901987-lang]

Arfu

[mao19901987-lang]

Hey everyone,

I wanted to provide an update on a critical issue affecting the open source community: the increasing volume of low-quality contributions that is creating significant operational challenges for maintainers.

We’ve been hearing from you that you’re dedicating substantial time to reviewing contributions that do not meet project quality standards for a number of reasons - they fail to follow project guidelines, are frequently abandoned shortly after submission, and are often AI-generated. As AI continues to reshape software development workflows and the nature of open source collaboration, I want you to know that we are actively investigating this problem and developing both immediate and longer-term strategic solutions.

What we're exploring

We’ve spent time reviewing feedback from community members, working directly with maintainers to explore various solutions, and looking through open source repositories to understand the nature of these contributions. Below is an overview of the solutions we’re currently evaluating.

Short-term solutions:

• Configurable pull request permissions - This has been a long-standing request (since 2016) and is a first step in giving maintainers more tools to customize their PR experience. Repository owners will be able to control pull request access at a more granular level by restricting contributions to collaborators only or disabling PRs for specific use cases like mirror repositories. This also eliminates the need for custom automations that various open source projects are building to manage contributions.

• The ability to delete a PR from the UI - This provides maintainers with the ability to remove spam or low-quality PRs directly from the interface to improve repository organization.

Long-term direction:

As AI adoption accelerates, we recognize the need to proactively address how it can potentially transform both contributor and maintainer workflows. We are exploring:

• Enhanced permission models - Provide maintainers with more tools so they can customize their PR experience:

  • More granular controls for determining who can create and reviewing PRs beyond blocking all users or restricting to collaborators only.

  • Ways for maintainers to define criteria that PRs must meet before they can be opened.

• Improved triage tools - Potentially leveraging AI to evaluate contributions against project guidelines and standards to identify which contributions maintainers should focus on reviewing.

• Transparency in AI-assisted contributions - Improved visibility and attribution when AI tools are used throughout the PR lifecycle.

Next Steps

These are some starting points, and we’re continuing to explore both immediate improvements and long-term solutions. Please share your feedback, questions, or concerns in this thread. Your input is crucial to making sure we’re building the right things and tackling this challenge effectively. As always, thank you for being part of this conversation. Looking forward to hearing your thoughts and working together to address this problem.