Skip to main content
AmigoFish

Can Open Models Keep Up With OpenAI and Anthropic? | Nathan Lambert

From a16z · · 58 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

a16z Partner Lisha Li sits down with AI researcher and Trillium founder Nathan Lambert to ask how durable the gap really is between frontier AI labs and open models.

Nathan argues that OpenAI and Anthropic have structural advantages in compute, capital, and the ability to pioneer new training paradigms, but open-model labs, particularly in China, have proven remarkably capable of following close behind. They unpack how much of that comes from distillation versus technical talent, why compute remains a major constraint, and how data, RL environments, and post-training are increasingly shaping model progress.

They also discuss Nathan’s new nonprofit research lab, Trillium, and its plans to open up parts of AI research that are increasingly difficult to study from the outside. That includes releasing RL environments and documenting how AI is actually being used to automate AI research, with the goal of demystifying recursive self-improvement and giving researchers and policymakers a clearer picture of what frontier systems can and cannot do.

Timestamps: 00:00 - Intro 00:58 - Compute, data, and talent 08:48 Compute-starved, and the distillation debate 14:57 Is distillation just competition? 19:24 Open and closed on different timescales 25:00 Frontier progress is really about data 31:29 What Trillium will build 38:05 Demystifying RSI 44:32 Can open source pre-train? 49:32 Harnesses and agent swarms

Resources: Learn more about Trillium: https://trilliumlabs.org/ Follow Nathan Lambert on X: https://x.com/natolambert Nathan Lambert’s Newsletter: https://www.interconnects.ai/ Follow Lisha Li on X: https://x.com/lishali88

Stay Updated: If you enjoyed this episode, be sure to like, subscribe, and share with your friends!

Find a16z on X: https://twitter.com/a16z

Find a16z on LinkedIn: https://www.linkedin.com/company/a16z

Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX

Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711

Follow our host: https://x.com/eriktorenberg

Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.

Back to a16z · Report a problem