Ship or Skip — Daily AI Tool Reviews

Nicholas Carlini, a research scientist at Anthropic, published a writeup this week detailing how he used Claude Code to discover a heap buffer overflow in the Linux kernel's NFSv4.0 LOCK replay cache that had gone undetected for 23 years. The bug allows attackers to read sensitive kernel memory over the network by exploiting a 1024-byte owner ID field being written into a 112-byte buffer.

The methodology was deliberately simple: Carlini wrote a script that iterated through Linux kernel source files, giving Claude Code a capture-the-flag framing to identify exploitable vulnerabilities. The AI required minimal human oversight — just pointed at source code and prompted. Claude Opus 4.6, Anthropic's latest flagship model, was substantially more effective at this than its predecessors; Opus 4.1 and Sonnet 4.5 found far fewer bugs in the same codebase.

The implications are significant in two directions. First, AI-assisted vulnerability research is now producing real results — not just CVE farms of minor issues, but remotely exploitable kernel-level bugs. Second, Carlini notes he has hundreds of additional potential findings but the bottleneck is now human validation: reviewers need to manually triage AI-identified candidates before they can be responsibly disclosed. The security community's capacity to absorb AI-generated vulnerability reports may become the limiting factor as model capability continues to improve.

Panel Takes

“This is Claude Code's clearest demonstration that AI agents can do meaningful security work, not just autocomplete. The capture-the-flag prompting technique is immediately reusable for any codebase audit.”

“One impressive find from a dedicated researcher at Anthropic using their own flagship model doesn't tell us about false positive rates, costs, or what happens when less-skilled researchers run the same pipeline. We need systematic benchmarks, not anecdotes.”

“The bottleneck shifting from finding bugs to reviewing them is a landmark moment. AI is producing vulnerabilities faster than humans can validate them. This is the preview of what security research looks like when models get 2x better again.”