What Is Quantization? How AI Models Get Smaller & Faster
From IBM Technology · · 8 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
Learn more about Quantization here → https://ibm.biz/~NdqpU881V
Huge AI models come with huge memory requirements.Grace Ableidinger explains how quantization compresses LLMs using lower-precision formats like INT8 and INT4. Learn how model size, GPU memory, speed, and accuracy trade off during AI inference.
00:00 – Understand What Quantization Does 00:23 – See Why LLMs Need Quantization 00:51 – Connect AI Parameters to Memory Requirements 01:31 – Calculate the Memory Cost of a 70B LLM 02:13 – See How Quantization Reduces Precision 02:53 – Follow Quantization Layer by Layer 04:21 – Compare FP16, FP8, INT8 & INT4 05:34 – Balance Memory Savings with Model Accuracy 06:33 – Compare PTQ vs Quantization-Aware Training 07:17 – Choose the Right Quantization Tradeoff
AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~SLKUNTFaQ
AI was used in the creation of the transcript and metadata for this video.
#llm #aimodels #aiinference #machinelearning #aiengineering
--------------------------------------------------------------------------------------------------------- Find us on YouTube: 🔵 IBM Technology: https://www.youtube.com/@IBMTechnology 🔵 IBM: https://www.youtube.com/@IBM 🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates