AI Bug Finding in Open Source: HAProxy’s Verdict

Open source maintainer reviewing AI bug finding reports

AI bug finding is already producing real fixes for open source projects, but HAProxy’s lead developer says the tools made for it keep getting in maintainers’ way. The developer, who also sits on the Linux kernel security team, says the problems are refusals triggered by safeguards, restrictive access programs and machine-written reports nobody can act on. The account is one person’s experience, not the project’s official position.

Floods of reports, with no reproducer

Over recent months the lead developer has seen hundreds of AI-assisted security reports, nearly all with the same flaw: a claimed bug with no reproducer and no patch. Reporters often don’t know what the model thinks it found.

That isn’t a model problem, he says. A trusted colleague with access to Anthropic’s Mythos model analyzed HAProxy, then validated the findings and wrote patches. He was impressed by the depth of a few bugs and doubted they’d have surfaced without such tools.

To cut the noise, he documented threat models for the Linux kernel and HAProxy so models can judge severity better. That reduced boring reports on both projects, though less on Linux, where some researchers still target old versions.

Access to the strongest model is gated

The Linux Foundation gave him a six-month Claude Max 20x subscription, which he calls impressive. It didn’t include Mythos. He completed Anthropic’s Cyber Verification Program, which gave him no Mythos access, and says the process restarts with more paperwork each time it changes. He also says OpenAI’s Daybreak program rejected his security team twice without explanation.

The top “Red” tier of the self-service access program requires verified security researcher status, which he says nobody on his team could get. Lower tiers need an identity approval aimed at corporations and, in his account, refuse tasks such as exploit reproduction.

HAProxy still ended up with two batches of Mythos-generated reports, 30 to 50 entries each. He calls them unusable: long single-line paragraphs, no code excerpts and no commit ID for the tested build. His team had Opus 4.8 decode the reports and propose fixes. Without reproducers it was hard to tell real defects from false positives, but about one-third of the proposed fixes were merged after rework.

Safeguards that fire at the wrong moment

He found Fable 5 unusable for low-level C work, saying it refuses whenever a prompt mentions a bug alongside C code. Opus 5 and 5.5 improved a lot and feel like a skilled coworker, and they can find bugs and sometimes write a working reproducer. But they still refuse too often, and sometimes decline to study a root cause even when he already has a reproducer. His workaround is to kill the session and restart, sometimes three or four times.

He says he saw no meaningful change in blocking between his work account and the verified CVP account. GLM 5.3 gave a different picture: on a first pass it found roughly half the candidates Opus 5.5 did, but it caught at least one real bug Opus missed, and it runs faster and cheaper. He also reports good results from Qwen3.8-Flash-Next on a local GPU, and says Mistral’s new Large 4 landed between GLM and Opus 5.5 in a first test.

What maintainers say they need

  • Scan their own code for bug candidates in specific areas
  • Write a reusable reproducer for each bug that proves it is reachable in unmodified code
  • Write several fix candidates from the same session’s context and explain why they will last
  • Drop false positives, which writing fixes tends to expose
  • Write regression tests and look for other bugs in the same class

Weaponized exploits, penetrating running systems, privilege escalation and data exfiltration aren’t on that list. His proposal is that safeguards shouldn’t trigger when the model is working on code whose source is provided. He also suggests a cheap subscription, around $20 a month, for developers of public projects, noting that GitHub has 225 million developers across 800 million projects.

The CLA worry and the backlog

After Anthropic opened an open-source scanner program, he read the contributor license agreement developers must sign. It grants a perpetual, irrevocable, royalty-free copyright and patent license, and asks signers to confirm they’re entitled to grant it. He says that is a concern for open source projects with several contributors and for enterprises with patents.

HAProxy has around 300 to 400 AI-generated reports pending, with two or three looking serious. He notes Anthropic’s own data shows only 0.5% of 7,300 vulnerabilities were prioritized and fixed by maintainers. The upside is that HAProxy now has fewer bugs than older versions, though review work is slowing new features. Whether vendors respond to AI bug finding complaints like these will decide how much of that burden lands on volunteers.

Source: HAProxy Technologies

Written by

Saksham Bhargava

Saksham Bhargava runs Alteroid and has been writing about consumer technology for over five years, covering iPhone, Android, Windows, AI tools and gaming. With an MCA and six years of hands-on web development experience, he tests the features he writes about on his own devices and focuses on clear, practical guides that solve real problems. Away from the screen, he enjoys travelling to places most tourists skip.