Skip to main content
AmigoFish

Muon: The Most Underrated Breakthrough in LLMs

From bycloud · · 19 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Try out Solid, agents with their own computers, accounts, and budgets: https://solid.tech/bycloud

For over a decade, AdamW has dominated how we train AI, but a new optimizer called Muon is challenging everything we thought we knew about LLM training. From potentially cutting training compute nearly in half to making previously overlooked AI architectures viable, here's why Muon might be one of the most underrated LLM breakthroughs in the past 2 years.

Your private local AI server needs an upgrade or maintenance support? Book a free audit now at https://nimbuskernel.com/

my latest projects: Intuitive AI Academy We just wrote a new piece on Linear Attention!! https://intuitiveai.academy/ limited time code "BACK2SCHOOL" for 50% off yearly plan

My Newsletter (weekly top research papers) https://mail.bycloud.ai/

My Patreon https://www.patreon.com/c/bycloud

Sources

Original Muon Blog https://kellerjordan.github.io/posts/muon/

Muon is Scalable for LLM Training https://arxiv.org/abs/2502.16982v1

Moonshot's Moonlight https://github.com/MoonshotAI/Moonlight

DeepSeek-V4 https://www.alphaxiv.org/abs/deepseek-v4

Parallax: Parameterized Local Linear Attention for Language Modeling https://arxiv.org/abs/2605.29157

Muon Learns More Robust and Transferable Features than Adam https://arxiv.org/abs/2606.09658

0:00 Intro 0:35 What is optimizer? 4:50 The problem with Adam 6:22 What Muon solves 8:43 The problem with Muon 14:28 How Muon is changing LLM research

Try out my new fav place to learn how to code https://scrimba.com/?via=bycloudAI

This video is supported by the kind Patrons & YouTube Members: 🙏Spam Maj, Alex, Chris LeDoux, DX Research Group, Poof N' Inu, Deagan, Robert Zawiasa, Ryszard Warzocha, Tobe2d, Louis Muk, Akkusativ, Kevin Tai, Mark Buckler, NO U, Tony Jimenez, Ângelo Fonseca, jiye, Anushka, Asad Dhamani, Binnie Yiu, Calvin Yan, Clayton Ford, Diego Silva, Etrotta, Gonzalo Fidalgo, Handenon, Hector, Jake Disco very, Michael Brenner, Nilly K, OlegWock, Daddy Wen, Shuhong Chen, Sid_Cipher, Stefan Lorenz, Sup, tantan assawade, Thipok Tham, Thomas Di Martino, Thomas Lin, Richárd Nagyfi, Paperboy, mika, Leo, Berhane-Meskel, Kadhai Pesalam, mayssam, Bill Mangrum, nyaa, Toru Mon, Lame Plane, Matej Macak, Len Mo, saylikhapekar, ZyanSheep, THEVIERAOS, Ricardo Raphael Corona-Moreno, superchordate

[Discord] https://discord.gg/NhJZGtH [Twitter] https://twitter.com/bycloudai [Patreon] https://www.patreon.com/bycloud [Business Inquiries] bycloud@smoothmedia.co [Other Inquiries] bycloudai@gmail.com [Profile & Banner Art] https://twitter.com/pygm7 [Video Editor] @aduckchicken2 Manim Animations created by https://x.com/miyashita_03 [Ko-fi] https://ko-fi.com/bycloudai [Bitcoin (BTC)] 3JFMJQVGXNA2HJE5V9qCwLiqy6wHY9Vhdx [Ethereum (ETH)] 0x3d784F55E0bE5f35c1566B2E014598C0f354f190 [Litecoin (LTC)] MGHnqALjyU2W6NuJSSW9fTWV4dcHfwHZd7 [Bitcoin Cash (BCH)] 1LkyGfzHxnSfqMF8tN7ZGDwUTyBB6vcii9 [Solana (SOL)] 6XyMCEdVhtxJQRjMKgUJaySL8cGoBPzzA2NPDMPfVkKN

Back to bycloud · Report a problem