Sovereign AI Starts at the Traffic Layer - Andjelko Iharos, HAProxy Technologies (ASL)
From CNCF [Cloud Native Computing Foundation] · · 5 min

The player loads from YouTube only when you click. It uses YouTube's privacy-enhanced mode.
If the video does not play, the owner may have turned off embedding. Watch it on YouTube instead.
Description
Connect with the cloud native community and explore CNCF events: https://kubecon.io
Sovereign AI Starts at the Traffic Layer - Andjelko Iharos, HAProxy Technologies (ASL)
Cloud native teams spent a decade building traffic infrastructure they control: open source, decentralized, observable, and running on Kubernetes. Then AI arrived. Inference, the most sensitive layer in the stack, became a call to a vendor API. Your cluster is sovereign. Your AI isn't.
You already run what's needed to reclaim that layer. An inference request is plain HTTP, with the model name in the JSON body. The ingress and Gateway API patterns in the cluster today can route and balance that traffic, and apply policy at the boundary. One data plane, instead of a fragmented traffic stack.
This session covers consistent-hashing across GPU backends and model-aware routing from request inspection, with inference staying inside the cluster. Everything runs on HAProxy, the open source, model-agnostic load balancer and ingress controller built for high throughput and low latency. You keep sovereign control of AI traffic with no performance penalty.
Back to CNCF [Cloud Native Computing Foundation] · Report a problem