Skip to main content
AmigoFish

How AI Agents Work: Inside Agentic Inference

From NVIDIA Developer · · 7 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

What happens when an AI agent starts working, and how do we measure its performance?

In this episode of Inside the Architecture, we break down how AI agents use models, tools, and context to complete tasks. Follow the work through animated diagrams, then explore how benchmarks and inference infrastructure shape the experience.

You’ll learn: How agentic workflows differ from a single prompt and response How tool calls, subagents, and growing context change the workload Why fixed-length tests and recorded-session replay measure different things How to interpret interactivity, latency, concurrency, and throughput Where NVLink, NVIDIA Dynamo, NVFP4, and TensorRT-LLM fit into the system No prior experience with inference infrastructure required.

#AgenticAI #nvidia

Benchmark results and methodology: https://developer.nvidia.com/blog/nvidia-vera-rubin-and-blackwell-set-a-new-standard-for-agentic-ai-performance-per-watt/ NVIDIA Dynamo architecture: https://developer.nvidia.com/blog/introducing-nvidia-dynamo-a-low-latency-distributed-inference-framework-for-scaling-reasoning-ai-models/

00:00 From Chat to AI Agents 00:48 How AI Agents Work 02:28 How Agentic Benchmarks Work 03:37 Which Metrics Matter? 04:32 Where Do Performance Gains Come From? 05:22 Understanding Benchmark Results 06:35 The System Behind the Agent 06:52 Where to Learn More

Back to NVIDIA Developer · Report a problem