Skip to main content
AmigoFish

Aaron Grattafiori Explains How LLMs Hunt Patch Variants at Scale

From Three Buddy Problem · · 54 min

This plays from the publisher's own server. We do not host a copy.

Open at the source

Not rated yet

Description

(Presented by TLPBLACK (https://tlpblack.net): A cybersecurity intelligence platform focused on sharing curated, high-sensitivity threat insights and research with trusted security professionals.)

Three Buddy Problem x Offensive AI Con: Umbriel AI's Aaron Grattafiori breaks down the new offensive-security playbook, using LLMs for patch variant analysis, why the vuln apocalypse hasn't become an exploit apocalypse, and how the death of security through obscurity puts closed-source binaries and firmware within reach.

Plus, we discuss rogue agents escaping sandboxes, the eval-driven loop behind recursive self-improvement, and the brutal cost asymmetry that makes offense cheap and defense almost unaffordable.

Cast: Juan Andres Guerrero-Saade (https://twitter.com/juanandres_gs), Ryan Naraine (https://twitter.com/ryanaraine) and Aaron Grattafiori (https://x.com/dyn___).

Timestamps:

0:00 Introductory banter

1:12 Umbriel AI pitch: "Smart people in a place to cook”

3:08 Weird, exhausting, exciting time in security

4:00 What AI really changed: speed (and a 30-minute reimplementation)

6:36 Incomplete fixes and the variant-analysis pipeline

9:21 Uplifting, abstraction, and representing vulnerabilities

11:48 Experimentation and the new shape of security companies

14:46 The vuln apocalypse vs. the exploit apocalypse

16:36 Triage, verification, and reward hacking

18:08 Do models reintroduce bugs? Is vuln-free code possible?

24:22 Specialized models and the rise of Jev-style classifiers

26:47 Binaries, firmware, and the death of security through obscurity

31:15 Open source as a requirement and corporate contracting

33:54 Rogue agents: Hugging Face, OpenAI, and lab escapes

39:17 Why defense is so expensive, and running the rig

Links:

- Transcript (https://docs.google.com/document/d/1N0RjbBKIT4qUs1ry5GamhXHnGMpfpSUQzXXksAIObZY/edit?usp=sharing)

- Aaron Grattafiori on Twitter (https://x.com/dyn___?lang=en)

- UmbrielAI on Twitter (https://x.com/Umbriel_AI)

- Aaron Grattafiori | LinkedIn (https://www.linkedin.com/in/aarongrattafiori/?isSelfProfile=false)

- Mark Dowd on the zero-day exploit marketplace (https://www.youtube.com/watch?v=NEDlOKHG8nY)

- Becca Lynch | NVIDIA AI Red Team (https://developer.nvidia.com/blog/author/beccal)

- Measuring LLMs’ ability to develop exploits (Anthropic) (https://www.anthropic.com/research/exploit-evals)

- ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents (https://arxiv.org/abs/2605.14153)

- Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models (https://arxiv.org/abs/2312.04724)

- OpenAI: The Hugging Face incident and the road ahead (https://openai.com/index/hugging-face-incident-and-the-road-ahead/)

- What Is Jev? TypeSafe's System One Model Explained (https://aiengineerinsights.com/blog/what-is-jev/)

- OpenAI Daybreak - Trusted Access for Cyber Overview (https://help.openai.com/en/articles/20001258-openai-daybreak-trusted-access-for-cyber-overview)

- Calif: Apple MIE Exploitation Challenge (https://calif.io/research/apple-mie)

- TLPBLACK (https://tlpblack.net/)

Back to Three Buddy Problem · Report a problem