Skip to main content
AmigoFish

What Is Model Distillation? How Smaller AI Models Learn

From IBM Technology · · 13 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Learn more about Knowledge Distillation here → https://ibm.biz/~9OAE6ZHmW

Small AI models can learn from much larger ones. Aadam Mulla explains model distillation, where student models learn from a teacher’s soft labels and dark knowledge. Learn how distillation balances AI capability, speed, cost, and deployment.

00:00 – Understand What Model Distillation Does 00:36 – See Why Smaller AI Models Matter 01:15 – Connect Model Parameters to Memory and Speed 02:24 – Meet the Teacher and Student AI Models 02:50 – See How LLMs Predict the Next Token 04:59 – Train the Teacher Model with Hard Labels 07:26 – Train the Student with Soft Labels 09:12 – Understand Why Soft Labels Teach More 10:04 – Explore Dark Knowledge in Model Distillation 12:09 – Balance AI Capability with Model Efficiency

AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~xkl5jbfCz

AI was used in the creation of the transcript and metadata for this video.

#aimodels #llm #machinelearning #aiengineering --------------------------------------------------------------------------------------------------------- Find us on YouTube: 🔵 IBM Technology: https://www.youtube.com/@IBMTechnology 🔵 IBM: https://www.youtube.com/@IBM 🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates

Back to IBM Technology · Report a problem