Skip to main content
AmigoFish

How llm-d scales AI inference across GPU clusters

From Red Hat · · 57 sec

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Single-server AI inference only gets you so far before cluster-scale operational burdens take over. Red Hat CTO for AI Brian Stevens breaks down how llm-d provides the missing link for managing AI inference across hundreds or thousands of GPUs. #llmd #vLLM #RedHat #EnterpriseAI #Kubernetes

➡️ Scale your inference: https://red.ht/future-of-inference

Back to Red Hat · Report a problem