Skip to main content
AmigoFish

Trace AI agents with OpenTelemetry | AgentOps Unlocked

From Red Hat · · 5 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.

If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.

Watch on YouTube

Not rated yet

Description

Your AI agent failed? See exactly why in minutes. Learn how AI agent observability and OpenTelemetry tracing capture tool calls, latency, and token costs without changing your code.

When an autonomous AI agent misbehaves or returns an unexpected answer, debugging a non-deterministic system without trace data is guesswork. While sandboxing controls what an agent can do and identity controls what it can call, tracing provides the definitive record of what it actually did.

In this video, Legare Kerrison demonstrates how to add production observability to autonomous agents without rewriting application code. Using an MLflow tracking server backed by the OpenTelemetry (OTel) software development kit (SDK), we capture full reasoning chains, system prompts, tool inputs, outputs, and token counts as structured spans. Learn how this data flows directly into standard enterprise observability pipelines using the OpenTelemetry Protocol (OTLP) environment variable.

00:00 The problem with black box AI agents 00:21 Sandboxes, identity, and tracing 00:43 Adding tracing without changing agent code 01:02 Configuring MLflow and OpenTelemetry export 01:40 How automated callback tracing works 02:20 Invoking the agent from the user interface 02:37 Inspecting traces in the MLflow tracking interface 03:09 Breaking down spans, prompts, and tokens 03:53 Auditing tool errors and unexpected behavior 04:38 OpenTelemetry integration for enterprise pipelines

Explore open source AgentOps and Red Hat security tools: 📂 Access the AgentOps GitHub repository → https://red.ht/github-agent-ops 🤖 Learn about the OpenTelemetry Protocol → https://red.ht/open-telemetry

Watch the previous episode of AgentOps Unlocked to learn about managing zero-trust AI agent identity → https://www.youtube.com/watch?v=2M3SByYGuZk

#AgentOps #OpenTelemetry #MLflow #AIAgents #OpenSource #DevOps #RedHat

Back to Red Hat · Report a problem