Skip to main content
AmigoFish

Why AI Models Run Faster on Some Chips: The KV Cache Effect

From IBM Technology · · 10 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Learn more about KV Cache here → https://ibm.biz/~6LPSw4Y8N

The same AI model can run differently across chips. Kaoutar El Maghraoui explains how GPUs, TPUs, and AI accelerators store and move KV cache during LLM inference. Learn why memory architecture and data movement can shape AI performance.

00:00 – Understand Why LLMs Need a KV Cache 00:46 – See How KV Cache Reuses Keys and Values 01:39 – Balance KV Cache Capacity and Fast Memory 02:41 – Follow KV Cache Through GPU Memory 04:41 – Compare TPU Memory and Compiler Orchestration 06:44 – Explore Dataflow Accelerator Memory 08:34 – Compare How AI Accelerators Move KV Cache 09:05 – Connect KV Cache Movement to LLM Performance

Continue your education in the course Memory Systems in Dataflow Architectures on Coursera here → https://ibm.biz/~I7KAjOCAb

AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~modQneKWV

AI was used in the creation of the transcript and metadata for this video.

#kvcache #llm #aimodels #aiaccelerator --------------------------------------------------------------------------------------------------------- Find us on YouTube: 🔵 IBM Technology: https://www.youtube.com/@IBMTechnology 🔵 IBM: https://www.youtube.com/@IBM 🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates

Back to IBM Technology · Report a problem