Beyond the Voice Bot: Engineering Real Time Customer Conversations
From Docker · · 50 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
Voice AI is changing how businesses answer calls, qualify leads, and support customers. But building a voice bot that actually works in production takes much more than connecting an AI model to a phone line.
In this episode of Ship Happens, host Per Krogslund talks with Seth Wright from CTM (formerly Call Tracking Metrics) about the evolution from call tracking and marketing attribution to configurable telephony platforms powered by AI.
CTM's platform helps businesses manage calls, texts, routing, call trees, voice bots, and handoffs to human teams. AI adds another layer of intelligence: enriching call data, identifying high-intent leads, triggering follow-ups, and building customer context that helps systems make better decisions about what should happen next.
Seth explains why modern large language models have changed what's possible in voice systems—and why the engineering challenge is far from solved. A voice interaction has to feel immediate. That means minimizing latency, preparing data before it's needed, and designing workflows that can handle real-time decisions such as booking appointments or escalating urgent issues.
The conversation goes beyond the voice interface to explore the work required to make these systems dependable: testing realistic conversations, replaying failures, improving observability, explaining AI-driven decisions, and building evaluation pipelines that catch problems before they reach customers.
Because when a voice AI system is slow, unreliable, or confidently wrong, customers don't care how impressive the demo looked.
Building better voice AI isn't just about making machines talk. It's about engineering systems that listen, understand context, act quickly, and know when to bring in a human.
What You'll Learn: • How CTM evolved from call tracking and marketing attribution into a configurable communications platform • How AI can enrich call data, identify hot leads, and trigger follow-up actions • Why voice bots need clear pathways between automation and human support • How LLMs changed the architecture of voice AI • Why latency is one of the defining challenges of real-time voice systems • How preloading customer and appointment data can make conversations faster • How customer memory records can improve routing and personalization • Why explainability matters for lead scoring, sentiment analysis, and automated decisions • How to test, replay, debug, and evaluate AI-driven conversations • Why observability and evaluation pipelines are essential for reliable production systems • How engineering teams can adopt AI without sacrificing quality or burning out • What new engineers should understand about building software in an AI-driven world
Chapters: 00:00 Answer Fast or Lose 01:01 Meet the Guest 01:33 What CTM Does 03:54 AI for Call Insights 05:18 IVRs and Voice Bots 06:18 Why Voice Changed 09:35 Voice Value and Speed 12:03 Engineering Voice Systems 13:54 Voice UI Primitives 17:46 Explainable Lead Scoring 20:54 Decision Models and Sentiment 23:06 Model Choices and Latency 27:32 Real-Time Data Loading 29:09 Testing and Debugging 33:33 Evaluation Pipelines and Teams 42:08 Engineering Craft in AI 44:27 Advice for New Engineers 48:44 Future Assumptions 49:55 Wrap Up and Goodbye
Key Takeaways:
• A great voice bot depends on the whole system. The conversational model is only one part of the experience. Routing, data access, integrations, escalation paths, and operational reliability all matter.
• Latency is a product decision. In a live conversation, even a technically correct response can feel broken if it takes too long. Architecture and data preparation directly shape the customer experience.
• Context makes automation more useful. Enriched call data and customer memory can help systems route conversations, recognize opportunities, and respond more appropriately.
• Human handoffs are part of good automation. Reliable systems need to recognize when a situation calls for human judgment…