AI Security Scanning for Startups: What curl, Microsoft and VulnCheck Found

AI security scanning finds real bugs, but the public record also shows a steep review cost: curl saw five confirmed findings shrink to one, and Microsoft ruled out three of five. Here is the evidence a founder can use to decide whether, and how, to scan.

In May 2026, curl maintainer Daniel Stenberg reported that an AI scan of his project had flagged five “confirmed” vulnerabilities, and that after review only one held up as a real, low-severity flaw. In the same post he wrote that AI analyzers outperform the traditional code checkers that came before them. That split verdict is the honest starting point for any founder asking whether AI security scanning belongs in a startup’s development process: the tools find things, and somebody has to check every finding.

Where it stands: As of Sept. 8, 2026, VulnCheck counted only 202 fixed findings out of the volume Anthropic reported for Project Glasswing, and Anthropic’s own ledger called human triage “the rate limiting step.” The strongest AI security models remain restricted: Anthropic says it does not plan to release Claude Mythos Preview generally.

What do the AI security scanning vendors actually claim?

The claims are large, and almost all are self-reported. Each one comes with a caveat from the company itself or from someone who checked.

Claim Caveat on the record
Anthropic: Mythos found “thousands of high-severity vulnerabilities” VulnCheck: 202 findings fixed in the ledger
OpenAI: Codex Security scanned 30 million+ commits OpenAI says humans review every Patch the Planet finding
Google: Big Sleep found a real SQLite bug Google calls results “highly experimental”
Microsoft: Security Copilot saved about a week 3 of 5 top flags were false positives

Anthropic said on its Glasswing page that “Mythos Preview has already found thousands of high-severity vulnerabilities, including some in every major operating system and web browser .” OpenAI, as reported by SiliconANGLE, said Codex Security “has scanned more than 30 million commits across more than 30,000 codebases,” and that human reviewers had signed off over 70,000 findings as fixed. Those are vendor numbers, not audits.

Google’s Big Sleep team described its 2024 SQLite find as “the first public example of an AI agent finding a previously unknown exploitable memory-safety issue in widely used real-world software,” yet was cautious about it: “we want to reiterate that these are highly experimental results .” It added that “a target-specific fuzzer would be at least as effective.” A fuzzer is an older automated tool that throws large volumes of malformed input at software to make it crash.

What does a false positive cost a small team?

A false positive is a reported vulnerability that turns out not to be one. Each costs the time of whoever knows the code best, and the public cases show the rate can be high.

Microsoft’s threat intelligence team used Security Copilot while auditing open-source bootloaders, in research published in March 2025. It credited the AI with saving “approximately a week’s worth of time,” then wrote: “In our manual review of the five identified issues, we found three were false positives, one was not exploitable, and the remaining issue, which warranted our attention and further investigation, was an integer overflow vulnerability.” Microsoft’s process also used conventional tools, fuzzing and manual analysis alongside the AI.

curl’s result looked similar. Stenberg wrote that the scan “concluded it found five “Confirmed security vulnerabilities”,” and that “The other four were three false positives (they highlighted shortcomings that are documented in API documentation) and the fourth we deemed “just a bug”.” The survivor, he said, “is going to end up a severity low CVE.”

For a startup, the math is about people, not tool fees. If one senior engineer spends a day ruling out three phantom bugs to find one real one, that may still be a good trade. It is a bad trade if nobody budgeted the day.

Why shouldn’t you trust the severity labels?

AI scanners tend to rate findings more severely than the people who maintain the code, according to VulnCheck’s comparison of Glasswing findings. VulnCheck reported that “Claude determined a critical or high severity for 91.5% of findings, while the maintainer determined only 51.3% as critical or high.”

That gap matters because severity drives scheduling. A founder who lets the scanner’s label decide what gets fixed this week may pull engineers onto medium issues dressed up as critical ones. ENISA’s Hans de Vries made a related point at a May 2026 conference, as reported by Infosecurity Magazine: “just one vulnerability doesn’t mean you’re automatically compromised.”

The fix is procedural. Treat every AI severity as a draft and have someone who knows how the code is actually deployed, what is exposed to the internet and what is behind authentication, set the final rating.

Is there a case for scanning anyway?

Yes, and some of it comes from the skeptics. Stenberg, who called the Mythos hype “primarily marketing,” also wrote that “AI powered code analyzers are significantly better at finding security flaws and mistakes in source code than any traditional code analyzers did in the past.” He said earlier AI tools had “triggered somewhere between two and three hundred bugfixes merged in curl” over roughly 8-10 months.

His warning to projects that skip it is direct: “Not using AI code analyzers in your project means that you leave adversaries and attackers time and opportunity to find and exploit the flaws you don’t find.” VulnCheck, which has publicly challenged Glasswing’s numbers, still wrote: “Don’t discount the reality that AI tools like Claude are incredibly valuable and useful tools for discovering vulnerabilities.”

Experts quoted by SecurityWeek also argued that curl is an unusually hard target, saying Mythos’s results there “reflect the maturity and robustness of curl’s codebase, rather than any shortcoming of the model itself.” A young startup codebase may not be that mature, which could mean more real findings, not fewer.

What kind of bugs should you expect AI to find?

Expect familiar bug types rather than new ones. Stenberg wrote: “We have not seen any AI so far report a vulnerability that would somehow be of a novel kind or something totally new.” For a startup, that means an AI scanner is best treated as a faster, broader reviewer for known classes of mistakes, not as a substitute for thinking about how the product could be abused.

Scanning your own code also leaves out what you import. On Oct. 21, Aikido Security reported that LiteLLM, a popular Python proxy for AI model calls, “was found to be compromised by an attacker who injected malicious code into its published Python wheels on PyPI.” No scan of a team’s own repository would have caught that, which is the subject of our look at AI dependency risk after the LiteLLM compromise.

How can a founder set up AI scanning without drowning?

Copy the pattern the large programs already use: AI proposes, a person decides. SiliconANGLE reported that “a human security engineer reviews every Patch the Planet finding before it reaches a maintainer,” and DeepMind says “all patches generated by CodeMender are reviewed by human researchers before they’re submitted upstream.”

  • Name a reviewer first. Someone owns the findings queue before the first scan runs.
  • Start narrow. Scan the code that handles logins, payments and customer data before the whole repository.
  • Re-rate severity. Record the scanner’s label and your own, and schedule by yours.
  • Track the hit rate. Count real findings against dismissed ones for a month; that number tells you whether to expand.
  • Feed fixes into your release process. Our patch day runbook covers ownership once a fix exists.

The same discipline helps on the receiving end, if outsiders run AI tools against your product and send reports, as covered in our guide to running a bug bounty when AI writes the reports.

Access is the last constraint. SiliconANGLE reported that for OpenAI’s GPT-5.5-Cyber, “Access stays restricted to vetted defenders through the company’s Trusted Access for Cyber program,” and Anthropic says Claude Mythos Preview will not be made generally available. As of Sept. 8, VulnCheck counted 202 Glasswing findings fixed and 245 withdrawn.

Frequently Asked Questions

Is AI security scanning worth it for a small startup?

The published evidence says AI scanners find real bugs and also produce false positives that someone has to rule out. curl maintainer Daniel Stenberg calls AI code analyzers significantly better than traditional ones, while Microsoft found that three of the top five issues Security Copilot flagged in its bootloader research were false positives. The value depends on having a person to review results.

How many false positives do AI code scanners produce?

There is no single published rate. Two documented cases: in curl’s Mythos scan, five reported confirmed vulnerabilities became one low-severity flaw after maintainer review, and in Microsoft’s bootloader work, three of five flagged issues were false positives and one was not exploitable.

Can a startup get access to Claude Mythos or GPT-5.5-Cyber?

Not generally. Anthropic has said Claude Mythos Preview is not headed for general release, and SiliconANGLE reports that OpenAI keeps GPT-5.5-Cyber restricted to vetted defenders through its Trusted Access for Cyber program. Broadly available AI coding and review tools are the realistic starting point for most small teams.

Should you trust the severity an AI scanner assigns?

Treat it as a first guess. VulnCheck found that for Glasswing findings with both ratings, Claude called 91.5% critical or high while maintainers rated 51.3% that way. A human who knows how the code is deployed should set the final priority.

Do AI security tools still need human review?

The programs cited here all keep a person in the loop. OpenAI says each Patch the Planet finding gets a review by a human security engineer before any maintainer sees it, and DeepMind says all CodeMender patches are reviewed by human researchers before submission.

Sources

This article is general information for founders, not legal or security-consulting advice. Incident details are as reported by the sources cited on the dates shown and can change; for breach-notification duties, talk to counsel.