Skip to main content
AmigoFish

NVIDIA Made 4-Bit AI Work. It’s a Nightmare.

From bycloud · · 19 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Check out Crusoe Intelligence Foundry that made inference & finetuning much easier, with $5 in free credits on sign-up: https://console.crusoecloud.com/request-foundry?utm_source=bycloud&utm_medium=influencer&utm_campaign=mi-su-credits

What is this "NVFP4" precision that NVIDIA is pushing so hard for, especially for their upcoming Vera Rubin GPUs? In today's video, I'll be explaining how NVIDIA made FP4 actually work for LLM training, and how it is much harder than simply using fewer bits. I will be breaking down NVFP4, the training tricks that make it stable, and why NVIDIA is choosing to design its newest GPUs around low-precision AI.

my latest project: Intuitive AI Academy We just wrote a new piece on Linear Attention!! https://intuitiveai.academy/ limited time code "BACK2SCHOOL" for 50% off yearly plan

My Newsletter (weekly top research papers) https://mail.bycloud.ai/

My Patreon https://www.patreon.com/c/bycloud

Sources:

Pretraining Large Language Models with NVFP4 https://arxiv.org/abs/2509.25149

Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery https://research.nvidia.com/labs/nemotron/files/NVFP4-QAD-Report.pdf

NVIDIA Nemotron 3 White Paper https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-White-Paper.pdf

Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3-Ultra-Technical-Report.pdf

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence https://www.alphaxiv.org/abs/deepseek-v4

Try out my new fav place to learn how to code https://scrimba.com/?via=bycloudAI

This video is supported by the kind Patrons & YouTube Members: 🙏Spam Maj, Alex, Chris LeDoux, DX Research Group, Poof N' Inu, Deagan, Robert Zawiasa, Ryszard Warzocha, Tobe2d, Louis Muk, Akkusativ, Kevin Tai, Mark Buckler, NO U, Tony Jimenez, Ângelo Fonseca, jiye, Anushka, Asad Dhamani, Binnie Yiu, Calvin Yan, Clayton Ford, Diego Silva, Etrotta, Gonzalo Fidalgo, Handenon, Hector, Jake Disco very, Michael Brenner, Nilly K, OlegWock, Daddy Wen, Shuhong Chen, Sid_Cipher, Stefan Lorenz, Sup, tantan assawade, Thipok Tham, Thomas Di Martino, Thomas Lin, Richárd Nagyfi, Paperboy, mika, Leo, Berhane-Meskel, Kadhai Pesalam, mayssam, Bill Mangrum, nyaa, Toru Mon, Lame Plane, Matej Macak, Len Mo, saylikhapekar, ZyanSheep, THEVIERAOS, Ricardo Raphael Corona-Moreno, superchordate

[Discord] https://discord.gg/NhJZGtH [Twitter] https://twitter.com/bycloudai [Patreon] https://www.patreon.com/bycloud [Business Inquiries] bycloud@smoothmedia.co [Other Inquiries] bycloudai@gmail.com [Profile & Banner Art] https://twitter.com/pygm7 [Video Editor] @Booga04 Manim Animations created by https://x.com/miyashita_03 [Ko-fi] https://ko-fi.com/bycloudai [Bitcoin (BTC)] 3JFMJQVGXNA2HJE5V9qCwLiqy6wHY9Vhdx [Ethereum (ETH)] 0x3d784F55E0bE5f35c1566B2E014598C0f354f190 [Litecoin (LTC)] MGHnqALjyU2W6NuJSSW9fTWV4dcHfwHZd7 [Bitcoin Cash (BCH)] 1LkyGfzHxnSfqMF8tN7ZGDwUTyBB6vcii9 [Solana (SOL)] 6XyMCEdVhtxJQRjMKgUJaySL8cGoBPzzA2NPDMPfVkKN

Back to bycloud · Report a problem