Is a Pod the right deployment unit for an AI agent?

When we first started building kagent, we didn’t run every agent in its own Kubernetes Pod, Service, and ServiceAccount. Instead, agents were simply executed inside the kagent runtime. It was the simplest architecture possible: one runtime... When we first started building kagent, we didn’t run every agent in its own Kubernetes Pod, Service, and ServiceAccount. Instead, agents were simply executed inside the kagent runtime. It was the simplest architecture possible: one runtime hosting many agents. It worked well for demos and proofs of concept. As the number of agents grew, however, fundamental questions started to emerge. How do we isolate one agent from another? How does each agent get its own identity? How do we enforce access and network policies? How do we understand what an individual agent is doing? Who owns an agent, and how do we support multi-tenancy? These aren’t Kubernetes questions. They’re agent platform questions. The Pod as the Deployment Unit Our first answer was straightforward: run every agent in its own Pod, Service, and ServiceAccount. That decision immediately solved many of our problems. A Pod provides process and container isolation. A ServiceAccount gives every agent its own Kubernetes identity, allowing us to integrate naturally with authentication and authorization mechanisms. Existing network policies, admission policies, and security controls continue to work without modification. Observability systems can attribute logs, metrics, and traces to individual agents. Scheduling and resource management also became Kubernetes-native. As the architecture evolved, we introduced stronger isolation mechanisms such as agent-sandbox in kagent, allowing agents to execute with tight security boundaries. For a while, this felt like the right abstraction. But Should Agents Be Best Represented as Pods? The more we thought about agents, the more we realized they are quite different from traditional microservices. Most services are expec

When we first started building kagent, we didn’t run every agent in its own Kubernetes Pod, Service, and ServiceAccount. Instead, agents were simply executed inside the kagent runtime. It was the simplest architecture possible: one runtime... When we first started building kagent, we didn’t run every agent in its own Kubernetes Pod, Service, and ServiceAccount. Instead, agents were simply executed inside the kagent runtime. It was the simplest architecture possible: one runtime hosting many agents. It worked well for demos and proofs of concept. As the number of agents grew, however, fundamental questions started to emerge. How do we isolate one agent from another? How does each agent get its own identity? How do we enforce access and network policies? How do we understand what an individual agent is doing? Who owns an agent, and how do we support multi-tenancy? These aren’t Kubernetes questions. They’re agent platform questions. The Pod as the Deployment Unit Our first answer was straightforward: run every agent in its own Pod, Service, and ServiceAccount. That decision immediately solved many of our problems. A Pod provides process and container isolation. A ServiceAccount gives every agent its own Kubernetes identity, allowing us to integrate naturally with authentication and authorization mechanisms. Existing network policies, admission policies, and security controls continue to work without modification. Observability systems can attribute logs, metrics, and traces to individual agents. Scheduling and resource management also became Kubernetes-native. As the architecture evolved, we introduced stronger isolation mechanisms such as agent-sandbox in kagent, allowing agents to execute with tight security boundaries. For a while, this felt like the right abstraction. But Should Agents Be Best Represented as Pods? The more we thought about agents, the more we realized they are quite different from traditional microservices. Most services are expected to be continuously available. Agents are not. An agent may wake up only when assigned a task, execute for a few seconds or minutes, and then become completely idle. Keeping a dedicated Pod alive for every potential agent quickly becomes wasteful. Agents also have execution patterns that don’t resemble long-running services: An agent may dynamically create multiple subagents to perform certain subtasks in parallel. An agent may impersonate a user or execute on behalf of a human. An agent may pause while waiting for human approval before continuing. An agent’s lifetime may be measured in seconds or minutes rather than days. These characteristics naturally lead to a question: Are Kubernetes Pods the right lifecycle abstraction for short-lived, bursty AI agents? Pods are excellent execution environments. But that doesn’t necessarily mean they should also be the right abstraction for AI agents. Enter Agent-substrate Instead of treating every agent as a first-class Kubernetes workload, agent-substrate introduces an additional control plane above Kubernetes. Kubernetes continues to manage Pods, Services, networking, storage, and compute resources, while agent-substrate manages the lifecycle and placement of AI actors onto execution workers. Agent-substrate introduces a set of abstractions that are similar to the Kubernetes concepts we are already familiar with. A WorkerPool is analogous to a NodePool, Workers are analogous to Nodes, and ActorTemplates correspond to the declarative specification of a Pod. Let’s look at what this abstraction looks like in practice. A WorkerPool defines a collection of execution workers that can host Actors. Example of the default WorkerPool in kagent: apiVersion: ate.dev/v1alpha1 kind: WorkerPool metadata: labels: app.kubernetes.io/instance: kagent app.kubernetes.io/name: kagent name: kagent-default namespace: kagent spec: ateomImage: ghcr.io/kagent-dev/substrate/ateom-gvisor:v0.0.6 replicas: 3 An ActorTemplate defines how an Actor should execute, much like a PodTemplate defines how a Pod should be created. Below is an example of a simple ActorTemplate in kagent. Note that it includes the runsc configuration, which serves as the execution entrypoint for gVisor. I omitted several kagent-specific fields, including the agent’s name and additional configuration details. apiVersion: ate.dev/v1alpha1 kind: ActorTemplate metadata: labels: app.kubernetes.io/managed-by: kagent kagent.dev/sandbox-agent: hello-substrate name: hello-substrate namespace: kagent spec: containers: - command: - /app ... env: ... image: cr.kagent.dev/kagent-dev/kagent/golang-adk@sha256:e01479b52280b0eae9e2808cc68392ba98fd737782496ff256847257e6bb8ed1 name: kagent pauseImage: gcr.io/gke-release/pause@sha256:bcbd57ba5653580ec647b16d8163cdd1112df3609129b01f912a8032e48265da runsc: amd64: sha256Hash: efd12935f6654c91a1389710eb8dfa4d12b6b9be00db87526dc2eb584ad00119 url: gs://gvisor/releases/nightly/2026-06-02/x86_64/runsc arm64: ... snapshotsConfig: location: gs://ate-snapshots/kagent/hello-substrate workerPoolRef: name: kagent-default namespace: kagent The Worker or Actor is not represented as a custom resource in Kubernetes. Kubernetes only sees WorkerPools and ActorTemplates. Agent-substrate, however, sees Workers and Actors. This separation allows the cluster to manage a fixed number of execution Pods while agent-substrate manages a much larger number of logical agents. You can use the substrate CLI or API to view them directly. Each Worker is mapped to a single unique Pod. $ kubectl-ate get workers NAMESPACE POOL POD STATUS ASSIGNED ACTOR kagent kagent-default kagent-default-deployment-ddfcfbdd7-54pb7 FREE <none> kagent kagent-default kagent-default-deployment-ddfcfbdd7-jmjl5 FREE <none> kagent kagent-default kagent-default-deployment-ddfcfbdd7-z2mmh FREE <none> $ kubectl-ate get actors NAMESPACE TEMPLATE ID STATUS ATEOM POD ATEOM IP VERSION kagent hello-substrate a786a0c4-c2c8-44e5-9ea5-67b6

When we first started building kagent, we didn’t run every agent in its own Kubernetes Pod, Service, and ServiceAccount. Instead, agents were simply executed inside the kagent runtime. It was the simplest architecture possible: one runtime... When we first started building kagent, we didn’t run every agent in its own Kubernetes Pod, Service, and ServiceAccount. Instead, agents were simply executed inside the kagent runtime. It was the simplest architecture possible: one runtime hosting many agents. It worked well for demos and proofs of concept. As the number of agents grew, however, fundamental questions started to emerge. How do we isolate one agent from another? How does each agent get its own identity? How do we enforce access and network policies? How do we understand what an individual agent is doing? Who owns an agent, and how do we support multi-tenancy? These aren’t Kubernetes questions. They’re agent platform questions. The Pod as the Deployment Unit Our first answer was straightforward: run every agent in its own Pod, Service, and ServiceAccount. That decision immediately solved many of our problems. A Pod provides process and container isolation. A ServiceAccount gives every agent its own Kubernetes identity, allowing us to integrate naturally with authentication and authorization mechanisms. Existing network policies, admission policies, and security controls continue to work without modification. Observability systems can attribute logs, metrics, and traces to individual agents. Scheduling and resource management also became Kubernetes-native. As the architecture evolved, we introduced stronger isolation mechanisms such as agent-sandbox in kagent, allowing agents to execute with tight security boundaries. For a while, this felt like the right abstraction. But Should Agents Be Best Represented as Pods? The more we thought about agents, the more we realized they are quite different from traditional microservices. Most services are expected to be continuously available. Agents are not. An agent may wake up only when assigned a task, execute for a few seconds or minutes, and then become completely idle. Keeping a dedicated Pod alive for every potential agent quickly becomes wasteful. Agents also have execution patterns that don’t resemble long-running services: An agent may dynamically create multiple subagents to perform certain subtasks in parallel. An agent may impersonate a user or execute on behalf of a human. An agent may pause while waiting for human approval before continuing. An agent’s lifetime may be measured in seconds or minutes rather than days. These characteristics naturally lead to a question: Are Kubernetes Pods the right lifecycle abstraction for short-lived, bursty AI agents? Pods are excellent execution environments. But that doesn’t necessarily mean they should also be the right abstraction for AI agents. Enter Agent-substrate Instead of treating every agent as a first-class Kubernetes workload, agent-substrate introduces an additional control plane above Kubernetes. Kubernetes continues to manage Pods, Services, networking, storage, and compute resources, while agent-substrate manages the lifecycle and placement of AI actors onto execution workers. Agent-substrate introduces a set of abstractions that are similar to the Kubernetes concepts we are already familiar with. A WorkerPool is analogous to a NodePool, Workers are analogous to Nodes, and ActorTemplates correspond to the declarative specification of a Pod. Let’s look at what this abstraction looks like in practice. A WorkerPool defines a collection of execution workers that can host Actors. Example of the default WorkerPool in kagent: apiVersion: ate.dev/v1alpha1 kind: WorkerPool metadata: labels: app.kubernetes.io/instance: kagent app.kubernetes.io/name: kagent name: kagent-default namespace: kagent spec: ateomImage: ghcr.io/kagent-dev/substrate/ateom-gvisor:v0.0.6 replicas: 3 An ActorTemplate defines how an Actor should execute, much like a PodTemplate defines how a Pod should be created. Below is an example of a simple ActorTemplate in kagent. Note that it includes the runsc configuration, which serves as the execution entrypoint for gVisor. I omitted several kagent-specific fields, including the agent’s name and additional configuration details. apiVersion: ate.dev/v1alpha1 kind: ActorTemplate metadata: labels: app.kubernetes.io/managed-by: kagent kagent.dev/sandbox-agent: hello-substrate name: hello-substrate namespace: kagent spec: containers: - command: - /app ... env: ... image: cr.kagent.dev/kagent-dev/kagent/golang-adk@sha256:e01479b52280b0eae9e2808cc68392ba98fd737782496ff256847257e6bb8ed1 name: kagent pauseImage: gcr.io/gke-release/pause@sha256:bcbd57ba5653580ec647b16d8163cdd1112df3609129b01f912a8032e48265da runsc: amd64: sha256Hash: efd12935f6654c91a1389710eb8dfa4d12b6b9be00db87526dc2eb584ad00119 url: gs://gvisor/releases/nightly/2026-06-02/x86_64/runsc arm64: ... snapshotsConfig: location: gs://ate-snapshots/kagent/hello-substrate workerPoolRef: name: kagent-default namespace: kagent The Worker or Actor is not represented as a custom resource in Kubernetes. Kubernetes only sees WorkerPools and ActorTemplates. Agent-substrate, however, sees Workers and Actors. This separation allows the cluster to manage a fixed number of execution Pods while agent-substrate manages a much larger number of logical agents. You can use the substrate CLI or API to view them directly. Each Worker is mapped to a single unique Pod. $ kubectl-ate get workers NAMESPACE POOL POD STATUS ASSIGNED ACTOR kagent kagent-default kagent-default-deployment-ddfcfbdd7-54pb7 FREE <none> kagent kagent-default kagent-default-deployment-ddfcfbdd7-jmjl5 FREE <none> kagent kagent-default kagent-default-deployment-ddfcfbdd7-z2mmh FREE <none> $ kubectl-ate get actors NAMESPACE TEMPLATE ID STATUS ATEOM POD ATEOM IP VERSION kagent hello-substrate a786a0c4-c2c8-44e5-9ea5-67b6