Aaron Grattafiori Explains How LLMs Hunt Patch Variants at Scale
From Three Buddy Problem · · 54 min
This plays from the publisher's own server. We do not host a copy.
Description
(Presented by TLPBLACK (https://tlpblack.net): A cybersecurity intelligence platform focused on sharing curated, high-sensitivity threat insights and research with trusted security professionals.)
Three Buddy Problem x Offensive AI Con: Umbriel AI's Aaron Grattafiori breaks down the new offensive-security playbook, using LLMs for patch variant analysis, why the vuln apocalypse hasn't become an exploit apocalypse, and how the death of security through obscurity puts closed-source binaries and firmware within reach.
Plus, we discuss rogue agents escaping sandboxes, the eval-driven loop behind recursive self-improvement, and the brutal cost asymmetry that makes offense cheap and defense almost unaffordable.
Cast: Juan Andres Guerrero-Saade (https://twitter.com/juanandres_gs), Ryan Naraine (https://twitter.com/ryanaraine) and Aaron Grattafiori (https://x.com/dyn___).
Timestamps:
0:00 Introductory banter
1:12 Umbriel AI pitch: "Smart people in a place to cook”
3:08 Weird, exhausting, exciting time in security
4:00 What AI really changed: speed (and a 30-minute reimplementation)
6:36 Incomplete fixes and the variant-analysis pipeline
9:21 Uplifting, abstraction, and representing vulnerabilities
11:48 Experimentation and the new shape of security companies
14:46 The vuln apocalypse vs. the exploit apocalypse
16:36 Triage, verification, and reward hacking
18:08 Do models reintroduce bugs? Is vuln-free code possible?
24:22 Specialized models and the rise of Jev-style classifiers
26:47 Binaries, firmware, and the death of security through obscurity
31:15 Open source as a requirement and corporate contracting
33:54 Rogue agents: Hugging Face, OpenAI, and lab escapes
39:17 Why defense is so expensive, and running the rig
Links:
- Transcript (https://docs.google.com/document/d/1N0RjbBKIT4qUs1ry5GamhXHnGMpfpSUQzXXksAIObZY/edit?usp=sharing)
- Aaron Grattafiori on Twitter (https://x.com/dyn___?lang=en)
- UmbrielAI on Twitter (https://x.com/Umbriel_AI)
- Aaron Grattafiori | LinkedIn (https://www.linkedin.com/in/aarongrattafiori/?isSelfProfile=false)
- Mark Dowd on the zero-day exploit marketplace (https://www.youtube.com/watch?v=NEDlOKHG8nY)
- Becca Lynch | NVIDIA AI Red Team (https://developer.nvidia.com/blog/author/beccal)
- Measuring LLMs’ ability to develop exploits (Anthropic) (https://www.anthropic.com/research/exploit-evals)
- ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents (https://arxiv.org/abs/2605.14153)
- Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models (https://arxiv.org/abs/2312.04724)
- OpenAI: The Hugging Face incident and the road ahead (https://openai.com/index/hugging-face-incident-and-the-road-ahead/)
- What Is Jev? TypeSafe's System One Model Explained (https://aiengineerinsights.com/blog/what-is-jev/)
- OpenAI Daybreak - Trusted Access for Cyber Overview (https://help.openai.com/en/articles/20001258-openai-daybreak-trusted-access-for-cyber-overview)
- Calif: Apple MIE Exploitation Challenge (https://calif.io/research/apple-mie)
- TLPBLACK (https://tlpblack.net/)