- Subject
- Microsoft 365 & SharePoint
- Filed
- Source
- Joseph Davis
- Loans
- 522
AI Agents Still Need Kubernetes Infrastructure That Teams Can Trust
AI can generate services, but enterprises still need controls for permissions, capacity, costs and failures. Kubernetes remains one way to provide them.

AI agents may generate applications and configuration, but they don’t remove the need to run those applications safely and affordably. That gap helps explain why Kubernetes still matters: enterprises need infrastructure that controls access, allocates capacity and contains failures, even when software increasingly handles the setup.
The question isn’t whether developers want to spend less time on deployment configuration. They do. An agent that takes repetitive setup work off their hands can be useful. The harder question is who makes sure the resulting service has the right permissions, enough capacity and a reliable way to recover when something goes wrong.
At KubeCon, which attracted more than 10,000 attendees last week, that operational work remained in view. Linux Foundation CEO Jim Zemlin put the distinction plainly: “Code is cheap [but] trust is expensive.” AI changes how quickly teams can create software. It doesn’t, by itself, establish that software is safe to run in a business.
AI adoption doesn’t remove operational work
Cloud-native teams are already adopting AI tools. A CNCF and SlashData report found that 75% of cloud-native developers use AI coding tools, compared with 55% of developers who aren’t cloud-native. Cloud-native developers aren’t all Kubernetes operators, but the figures offer little support for the idea that this community is simply rejecting AI.
Enterprise use is also real, though the degree of autonomy remains limited. Zapier’s November 9 survey found that 71% of respondents’ organizations had agents in production. The survey covered 513 AI-experienced managers in the US at companies with at least 1,000 employees, so its results may not represent every organization. Among leaders at organizations using agents, only 30% said the agents completed more than 60% of assigned tasks without human intervention.
Those figures describe adoption, not a transfer of operational responsibility. Enterprises can use agents while still requiring applications to be secure, recoverable and affordable. An agent can create a service, but someone still needs to decide which customer records it can access, how much capacity it can consume, and what happens when demand competes for limited resources.
Capacity controls affect AI costs
AI workloads make those decisions especially visible because accelerator capacity is expensive and demand changes. Zemlin said KubeCon discussions included security and improving performance with less hardware and power. The challenge is balancing predictable resources for training with responsive capacity for customer-facing inference.
China Merchants Bank described that challenge in a CNCF case study published in September. The bank supports AI training and inference across nearly 10,000 accelerator cards. It needed training capacity to be predictable while inference services responded to changing customer demand. Holding capacity for peak traffic can leave expensive resources idle when demand falls; scaling too slowly can leave users waiting.
The bank combined Kubernetes with tools for scheduling, monitoring, scaling and moving model data. It reported that average accelerator compute utilization rose from 35% to more than 60%. For comparable models and services, inference cost per million tokens fell by over 60%. Those results, as reported by the bank, show why infrastructure management remains part of the AI cost equation. Generating a working service is only one step; operating it efficiently matters too.
A deployment can succeed and still use more resources than it needs. It can also leave broad permissions in place for another team to discover later. Faster configuration generation doesn’t prove that the configuration is appropriate. Platform teams still have to provide controls without taking away the developer agility those platforms are meant to support.
Agent platforms need isolation and clear limits
Agents introduce infrastructure needs of their own. They run code, wait for model responses and maintain state. Keeping dedicated compute running while an agent waits can waste resources, while running generated code without adequate isolation can expose systems to risk.
On November 5, Google announced Agent Substrate 1.0 and its general availability on Google Kubernetes Engine. The company also donated Agent Substrate, gVisor and OpenRL to CNCF. Google’s design separates persistent agent state from reusable execution environments and uses gateways to control access. The underlying needs are familiar to platform teams: agents require boundaries, resource controls and a place for state that persists beyond a single execution.
Nvidia contributed OpenShell to the CNCF Sandbox on November 10. The runtime is designed to set boundaries around agent access to data, secrets and networks. Zemlin described it as “critical sandbox software that needs peer review and collective development.” Open source doesn’t guarantee that a control is safe, but shared code gives teams a common place to inspect and improve it rather than relying on separate implementations they can’t easily assess.
Give agents a platform they can operate
Enterprises remain accountable for the work they delegate to agents. In practice, that means permissions must be enforced, state preserved, resources allocated and failures contained. Agents can eventually take on more configuration and troubleshooting, but the platform still defines what they’re allowed to do and provides the feedback they need to act reliably.
That doesn’t mean every project needs Kubernetes. A managed service that hides infrastructure may be the right fit. The practical choice is to use capabilities a team knows how to operate, rather than create another set of systems simply because an agent can generate them.
Let agents build, but give them clear access limits, spending boundaries and a way to undo a bad deployment. The next step for IT teams is to check whether their platform can enforce those controls and show what agents actually do. KubeCon’s relevance rests on that operating work: making shared infrastructure safer and less costly while teams build on top of it.