#250 – Toby Ord on where AGI timelines go wrong
From 80,000 Hours Podcast · · 2 hr 46 min
This plays from the publisher's own server. We do not host a copy.
Description
Both Silicon Valley and the public can’t get enough of ‘AGI timelines.’ But Toby Ord, senior researcher at Oxford’s AI Governance Initiative (https://aigi.ox.ac.uk/) and author of The Precipice (https://en.wikipedia.org/wiki/The_Precipice:_Existential_Risk_and_the_Future_of_Humanity), believes we consistently make big mistakes when thinking about them. He lays out the 14 ways he most often sees people go wrong:
- Assuming AI research is just hill-climbing
- Imagining AI research is just programming
- Forecasting “could” instead of “will”
- Believing the current benchmark is the last one
- Extrapolating trends with no clear finish line
- Assuming inputs keep scaling at the same rate
- Conflating intelligence with capability
- Consuming point estimates and discarding the error bars
- Dismissing dissenting experts
- Forecasting very different things while using the same words
- Assuming capabilities arrive together
- Treating “we don’t know” as permission to carry on as usual
- Choosing a plan that minimises regret rather than maximises impact
- Trusting surface model impressiveness
In this extended conversation with Rob Wiblin, Toby also explains why he thinks:
- AI self-improvement is uniquely dangerous in four ways, but also might not even work
- A ban on superintelligence is possible
- A US-China treaty on superintelligence is also possible
- The case for ‘broad timelines’
- Transformative AI is likely a decade away
- We should just ban unmonitorable chain-of-thought today.
This episode was recorded on July 2, 2026.
Links to learn more, video, and full transcript: https://80k.info/to26
Want to get up to speed on AI? We’ve got a crash course of 10 of our podcast episodes (https://80000hours.org/podcast/on-artificial-intelligence/) designed to help you get to grips with transformative AI — particularly if you’re new to the topic — and what you can do to help shape its trajectory.
Chapters:
- Toby Ord is back — for the 5th time! (00:00:00)
- AI self-improvement might not matter (00:00:14)
- 4 ways AI self-improvement is dangerous (00:12:39)
- A US-China treaty on superintelligence is possible (00:20:47)
- Could we ban superintelligence? (00:37:07)
- We should just ban unmonitorable chain of thought (00:57:46)
- Why Toby thinks AGI is a decade away (01:09:28)
- Even superintelligence needs work experience (01:17:50)
- Is AI coming for mathematicians? (01:32:22)
- The case for broad timelines (01:45:01)
- How should broad timelines change what we do? (02:22:24)
- Are current models all they’re cracked up to be? (02:31:03)
- Coordinating careers for different timelines (02:43:36)
Our production team includes:
- Video editors: Josh Alward, Dominic Armstrong, Ollie Bignell, Andrés Escobar, Jasper Luithlen, Milo McGuire, Luke Monsour, and Simon Monsour
- Producers: Elizabeth Cox and Nick Stockton
- Coordination and support: Katy Moore and Lou Moran
- Camera operator: Jeremy Chevillotte
- Music: CORBIT (https://open.spotify.com/artist/4lWobp6IUcSZ7w5mhnU1c9)