39 specific projects to improve our chances with AGI | Holden Karnofsky, Anthropic
From 80,000 Hours Podcast · · 4 hr 34 min
This plays from the publisher's own server. We do not host a copy.
Description
In light of recent events in AI, we're rereleasing some of older episodes that seem more relevant than ever. This episode with Anthropic's Holden Karnofsky from October 2025 lays out the many concrete things people can do to reduce the risks from increasingly powerful AI — ranging from technical solutions to policy and regulations to improving information security.
If you want more ideas about how you can help steer the trajectory of AI development in a better direction, see our page on the 80,000 Hours website: https://80000hours.org/hugging-face/
---
For years, working on AI safety usually meant theorising about the ‘alignment problem’ or trying to convince other people to give a damn. If you could find any way to help, the work was frustrating and low feedback.
According to Anthropic’s Holden Karnofsky, this situation has now reversed completely.
There are now large amounts of useful, concrete, shovel-ready projects with clear goals and deliverables. Holden thinks people haven’t appreciated the scale of the shift, and wants everyone to see the large range of ‘well-scoped object-level work’ they could personally help with, in both technical and non-technical areas.
In this interview, Holden — previously cofounder and CEO of Open Philanthropy (now Coefficient Giving (https://coefficientgiving.org/research/open-philanthropy-is-now-coefficient-giving/)) — lists 39 projects he’s excited to see happening, including:
- Training deceptive AI models to study deception and how to detect it
- Developing classifiers to block jailbreaking
- Implementing security measures to stop ‘backdoors’ or ‘secret loyalties’ from being added to models in training
- Developing policies on model welfare, AI-human relationships, and what instructions to give models
- Training AIs to work as alignment researchers
And that’s all just stuff he’s happened to observe directly, which is probably only a small fraction of the options available.
All this low-hanging fruit is one factor behind his decision to join Anthropic this year. That said, his wife was also a cofounder and president of the company, giving him a big financial stake in its success — and making it impossible for him to be seen as independent no matter where he worked.
Holden makes a case that, for many people, working at an AI company like Anthropic will be the best way to steer AGI in a positive direction. He notes there are “ways that you can reduce AI risk that you can only do if you’re a competitive frontier AI company.” At the same time, he believes external groups have their own advantages and can be equally impactful.
Host Rob Wiblin and Holden discuss the case for and against those strategies, and much more, in today’s classic episode — originally released October 30, 2025.
Learn more, video, & full transcript: https://80k.info/hk25
Chapters:
- Rob’s intro + cold open (00:00:00)
- An AI Chernobyl we never notice (00:02:10)
- Is rogue AI takeover easy or hard? (00:06:51)
- The AGI race isn't a coordination failure (00:17:12)
- What Holden now does at Anthropic (00:27:42)
- The case for working at Anthropic (00:29:50)
- Can we trust Anthropic, or any AI company? (00:40:42)
- Can Anthropic compete while paying the “safety tax”? (00:49:24)
- What, if anything, could prompt Anthropic to halt development of AGI? (00:56:25)
- Responsible scaling policies retrospective (00:59:17)
- Overrated work (01:14:58)
- Concrete shovel-ready projects Holden is excited about in technical AI safety, AI relationships, and biosecurity (01:17:09)
- How to choose where to work (01:36:51)
- Overrated AI risks: Cyberattacks and persuasion (01:42:51)
- Why AI R&D is the main thing to worry about (01:56:44)
- The case that AI-enabled R&D wouldn't speed things up much (02:08:43)
- AI-enabled human power grabs (02:12:39)
- Main benefits of getting AGI right (02:24:53)
- The world is handling AGI about as badly as possible (02:30:56)
- Learning from targeting companies for…