Skip to main content
AmigoFish

What Is LLM Inference? How AI Models Generate Responses

From IBM Technology · · 7 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Learn more about LLM Inference here → https://ibm.biz/~CCvvB5TvB

Every LLM response starts with much more than a prompt. Grace Ableidinger explains LLM inference, from tokenization and GPU batching to KV cache, prefill, and decode. Learn how AI models turn static weights into real-time responses.

00:00 – Understand What LLM Inference Actually Does 01:15 – See How LLM Inference Engines Serve AI Models 01:43 – Follow an LLM Request from API to Model 02:15 – Turn Prompts Into Tokens with Tokenization 02:59 – Route LLM Requests to GPU Hardware 03:30 – Increase LLM Throughput with Continuous Batching 04:23 – Understand LLM Prefill and KV Cache 05:24 – See How Decode Generates Tokens 06:18 – Measure LLM Latency with TTFT and ITL 06:59 – Connect the Full LLM Inference Pipeline

AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~LqLIqIARH

AI was used in the creation of the transcript and metadata for this video.

#llm #aimodels #aiinference #ai #aiengineering --------------------------------------------------------------------------------------------------------- Find us on YouTube: 🔵 IBM Technology: https://www.youtube.com/@IBMTechnology 🔵 IBM: https://www.youtube.com/@IBM 🔵 IBM Developer: https://youtube.com/@IBMDeveloperAdvocates

Back to IBM Technology · Report a problem