🔍 Read the full analysis: Fuzz Tests, Formal Proofs, And The Limits Of Claude’s Work on ThorstenMeyerAI.com
Prime made for students and young adults
- Fast, free delivery for dorm and study essentials
- Prime Video and Amazon Music included
- Member-only deals
TL;DR
Anthropic’s Frontier Red Team and collaborator Theorem report that Claude produced Rocq proofs for a no-crash property covering 239 fuzz-test entry points in 123 open-source libraries. The proofs apply to specific compiled RISC-V binaries under stated assumptions, not every version or platform; the teams also report finding hundreds of bugs and say the cost of producing proofs remains high.
Anthropic’s Frontier Red Team and its collaborator Theorem said on October 22 that Claude produced formal proofs for 239 fuzz-test entry points across 123 open-source libraries, showing that specific compiled programs do not crash on inputs under 64 MB under the teams’ stated assumptions. The project also reported finding hundreds of bugs, but its results concern particular RISC-V binaries and do not establish that the libraries are free of bugs on the platforms most users run.
The teams converted existing fuzz tests into specifications for compiled programs. In the described pipeline, each library and its fuzz-test entry point were built as an instrumented RISC-V executable. AddressSanitizer was used to make certain memory errors cause a crash. Claude then generated proofs in Rocq, a proof assistant whose checker validates whether a proof follows from the specification and formal models.
The property being proved is narrower than general software correctness: for every input under 64 MB, a particular entry point does not crash in the modelled system, including through an AddressSanitizer memory error or failed assertion. The project’s account says the proofs total about 180 million lines of Rocq; a typical proof was about 600,000 lines. Anthropic said the volume of proof text did not need human line-by-line review because the Rocq checker verifies it. The trusted basis still includes the theorem statements, formal models and proof checker.
Anthropic and Theorem said the work uncovered hundreds of bugs, including cases they said fuzzing was unlikely to reach. Examples in the account include a decompressor issue requiring an input of about 20 MB with a rare layout, and an image-decoder issue requiring a 64 MB input and more than 2 GB of memory. They also described a bug involving a sequence of calls across two file handles and a rediscovery of CVE-2026-56132 in libexpat. The source material does not provide a complete public tally or independent assessment of each finding.
Infographic 01 · From test to proof
Existing fuzz tests became specifications. Claude proved them.
Over a few weeks, Claude proved at the machine-code level that hundreds of real fuzz entry points can no longer crash, after patching the bugs it found.
The pipeline
From an existing test to a checked proof
- Compile. The library and its existing fuzz test become a RISC-V executable, instrumented with AddressSanitizer.
- Specify. Theorem’s RISC-V and Linux process semantics plus the no-crash property mechanically yield the specification.
- Prove. Claude writes the proof in Rocq. A failed attempt points at a possible bug; Claude patches it and proves the new spec.
- Check. The Rocq proof checker verifies the proof. Virtually none of the proof needs human review.
Fuzzing vs. proof
Same entry point, different guarantee
| Fuzzing | Proof | |
|---|---|---|
| Inputs | Fuzzing: Generated samples, typically capped at 1 MB | Proof: Every input under 64 MB |
| Paths | Fuzzing: Guided by code coverage, no guarantee | Proof: Every path reachable from the entry point |
| Result | Fuzzing: Bugs found so far | Proof: No crash left to find on the modelled system |
Typical proof: about 600,000 lines for a binary with a median length of 18,000 RISC-V instructions.
“If Claude can prove this spec, the fuzz test on RISC-V becomes superfluous.”Anthropic, 22 Oct 2026
Source: Anthropic, “LLMs are making formal software verification feasible at a much larger scale” (22 Oct 2026).
Powered by Thorsten Meyer AI
What Machine-Checked Proofs Could Change
Formal verification can establish that a stated property holds across all inputs covered by a model, rather than offering the sampling-based assurance of ordinary testing. That makes the reported results relevant to software security: if the specification and model match the program being used, a proved no-crash property can rule out the covered class of failures for that entry point.
The work also illustrates a possible change in who performs the labour. Claude generated proof material, while Rocq performed the checking. That separation matters because the proof-writing system does not have to be trusted in the same way as the checker. It does not remove the need to trust the specification, models, compiler and other components in the verification chain.
The economics remain unresolved. The teams say some proofs took hundreds of millions of tokens, even as newer models and reuse of shared proofs can reduce the burden. The project therefore shows a technical capability at notable scale, but not yet a low-cost, routine method for verifying all software. For maintainers, the immediate value may lie in the bugs found and in a way to turn existing fuzz tests into stronger, machine-checked claims.
As an affiliate, we earn on qualifying purchases.
From Fuzz Tests to Rocq Proofs
Fuzzing feeds many generated or mutated inputs to a program and looks for failures. It can expose serious defects, but the number of tested inputs does not prove that every possible path was exercised. The approach described by Anthropic and Theorem starts with those same existing fuzz-test entry points, rather than trying to define correctness for an entire application.
The teams compiled the code and test to RISC-V, used formal models of RISC-V instructions and Linux processes to frame the property, and asked Claude to produce Rocq proofs. If a proof attempt failed, the failure could point to a possible bug; the account says Claude could patch code, regenerate the specification and try again. The result is a claim about the modelled executable and test, not automatically about source code in every build configuration.
Anthropic framed the project against the difficulty of conventional formal verification, which it says has often required substantial human effort. It described the work as, to its knowledge, the first large-scale use of AI for formal verification of production software at assembly-code level. That characterization is Anthropic’s, rather than an independently established ranking. The company said it is publishing proofs that did not require patching, previously unavailable trusted components and specifications for tests it could not prove.
“To our knowledge, this is the first large-scale use of AI to apply formal verification to production software at the assembly code level.”
— Anthropic, describing the project
formal verification software tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Where the Proofs Stop
The proofs do not establish that the libraries are bug-free. They cover single-threaded, static RISC-V binaries linked against musl, according to the source account. Most users may encounter builds for x86 or ARM linked against glibc, and the reported results do not by themselves prove those builds have the same properties.
The assurance also depends on the specification accurately representing the intended property, and on the formal models and trusted software components being sound. A no-crash guarantee does not rule out defects that do not cause a crash; the account itself mentions a file-access restriction logic bug, which illustrates the distinction between a particular test property and overall security.
The source material does not state the final disposition of every reported bug, whether maintainers have confirmed or fixed them, or the full cost and time breakdown across all proofs. It also describes a section on properties beyond memory safety but cuts off before completing that account. Those details cannot be established from the available material.
As an affiliate, we earn on qualifying purchases.
Publication and Maintainer Follow-Up
Anthropic says it is publishing the proofs that succeeded without code changes, the trusted components not already public, and specifications for fuzz tests it could not prove. Those materials should allow other researchers and maintainers to inspect the claims, reproduce parts of the work and identify gaps in the modelling assumptions.
The next practical test will be whether maintainers confirm and address the reported defects, and whether proof methods can be reused across versions and common platform builds. The teams’ account says newer models and shared proofs can reduce token use, but it does not give a timeline for broad deployment or establish that costs have fallen enough for routine use. Further evidence will be needed to judge how the approach performs beyond these libraries and these specific binaries.
AddressSanitizer memory error detector
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What did Claude prove?
For 239 fuzz-test entry points in 123 open-source libraries, the reported proofs establish that particular compiled programs do not crash on inputs under 64 MB, under stated assumptions and within the modelled system.
Does this mean the libraries are bug-free?
No. The proofs address specific properties of particular RISC-V binaries. They do not cover every platform, build configuration or kind of defect, and a program can have bugs that do not trigger the tested crash conditions.
How were the proofs checked?
Claude generated proof scripts in Rocq. Rocq’s proof checker checked whether those scripts established the stated theorems against the formal models; the proof generator itself was not the final verifier.
Why did the project report bugs missed by fuzzing?
Fuzzing tests a finite set of inputs, while a successful proof must account for all inputs and paths covered by its specification. The teams said some reported defects required unusually large inputs or specific sequences of operations.
What remains to be verified independently?
Researchers and maintainers can examine the published proofs, specifications and trusted components. The available account does not establish that every reported bug has been confirmed or fixed, or that the results carry over to commonly used x86 and ARM builds.
Source: ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
