Nicholas Carlini - Black-hat LLMs | [un]prompted 2026

Current LLMs can find and exploit zero-day vulnerabilities in critical software without fancy scaffolding — just Claude Code running in a VM with a simple prompt (2:45). This capability emerged only in the last 3-4 months and represents a fundamental shift in the attacker-defender balance that has existed for 20 years.

The entire vulnerability-finding setup is essentially Claude Code with 'dangerously skip permissions' and a prompt saying 'find a vulnerability' (3:10). Adding a single hint line directing the model to specific files enables thorough code review across entire projects, making this accessible to anyone — not just security experts.

Claude found the first-ever critical security vulnerability in Ghost (50,000 GitHub stars, ~20 years old) — a blind SQL injection that could extract complete admin credentials, API keys, and password hashes without authentication (5:30). The model also autonomously wrote a working exploit for the blind injection technique.

LLMs discovered remotely exploitable heap buffer overflows in the Linux kernel's NFS v4 daemon — a bug requiring understanding of two cooperating adversaries sending coordinated packets (7:15). The bug has existed since 2003, predating git itself, and the model even generated the schematic explaining the multi-step attack flow.