Why AI Models Run Faster on Some Chips: The KV Cache Effect
From IBM Technology · · 10 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
Learn more about KV Cache here → https://ibm.biz/~6LPSw4Y8N
The same AI model can run differently across chips. Kaoutar El Maghraoui explains how GPUs, TPUs, and AI accelerators store and move KV cache during LLM inference. Learn why memory architecture and data movement can shape AI performance.
00:00 – Understand Why LLMs Need a KV Cache 00:46 – See How KV Cache Reuses Keys and Values 01:39 – Balance KV Cache Capacity and Fast Memory 02:41 – Follow KV Cache Through GPU Memory 04:41 – Compare TPU Memory and Compiler Orchestration 06:44 – Explore Dataflow Accelerator Memory 08:34 – Compare How AI Accelerators Move KV Cache 09:05 – Connect KV Cache Movement to LLM Performance
Continue your education in the course Memory Systems in Dataflow Architectures on Coursera here → https://ibm.biz/~I7KAjOCAb
AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~modQneKWV
AI was used in the creation of the transcript and metadata for this video.
#kvcache #llm #aimodels #aiaccelerator --------------------------------------------------------------------------------------------------------- Find us on YouTube: 🔵 IBM Technology: https://www.youtube.com/@IBMTechnology 🔵 IBM: https://www.youtube.com/@IBM 🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates