Background Agent Task Execution Patterns

Background agent execution runs on a handful of structural patterns: async dispatch, durable state, checkpointing, and a couple of planning models that decide how work gets broken up. Whether an agent survives contact with production, or falls apart the first time a tool call fails, comes down to how these pieces get put together. Not the model. Not the prompt. The plumbing.

Start with the definition, because "background agent" gets thrown around loosely. It's an agent that runs a multi-step task on its own, outside the request-response cycle that kicked it off. A caller hands it a goal, gets back a run identifier or a simple acknowledgment, and moves on with their day. The agent plans, calls tools, checks what came back, adjusts course, and either saves a result somewhere or stops to ask a human a question.

Four traits mark a good fit for this pattern. The task repeats on a regular schedule, weekly is common. It needs information pulled from more than one system. Someone can look at the output afterward and tell if it's right. And a bad first draft doesn't cost much. Miss those traits and the architecture probably isn't worth the trouble.

Three situations rule it out entirely. Deterministic workflows are cheaper to test and run without an agent layer at all, so don't bother. Real-time conversations need speed and back-and-forth that async execution can't give you. And decisions with real consequences but no clean way to measure success should stay with a person making the call, full stop. The right workflow, when it fits, tends to save somewhere between 5 and 30 hours a month without forcing anyone to reorganize how a team works. That's the whole pitch. Everything past this point is about how to actually build the thing so it holds up.

Why the gap between prototype and production is widening faster than most teams expect

Gartner's numbers tell the adoption story bluntly: fewer than 5% of enterprise applications used AI agents in 2025, and that's projected to hit 40% by 2026. That's not gradual. That's a cliff.

The governance side is where it gets uncomfortable. Gartner warned that large enterprises could go from fewer than 15 agents running in 2025 to more than 150,000 by 2028. Only 13% of organizations said they believe they have the right governance in place for that. Do the math on that gap and it stops looking like a rounding error.

McKinsey's data adds a second axis. Regular generative-AI use across business functions climbed from 65% in 2024 to 71% in 2025. So usage is scaling steadily, while the infrastructure underneath it hasn't caught up to standard patterns yet. Two curves, moving at different speeds, headed for the same intersection.

Gartner also projects that by 2028, at least a third of enterprise software will depend on agentic AI, but getting there means clearing an 85% failure rate that current deployments run into. That number is the whole pivot of this piece. Failure at that scale isn't a "the model wasn't smart enough" problem. It's an execution pattern problem: agents dying mid-task, losing state, or getting stuck because nobody built a way for them to recover.

There's a structural wrinkle worth sitting with. Frontier models have been extending their task-completion horizons steadily, and the tasks they can now attempt are longer than most teams' infrastructure can reliably support. The AI got ambitious faster than the plumbing did.

How agent execution differs structurally from the request-response model most infrastructure was built for

Most infrastructure was built on an assumption: work is short, stateless, and interchangeable. Containers get bin-packed, autoscaled on CPU and memory, and if something breaks, the scheduler just evicts it and starts over. Fine for a web server. Not fine for an agent.

Agent execution breaks every one of those assumptions, one at a time. A single workflow can run for minutes or hours, not milliseconds. It's stateful, meaning a decision three steps in depends on something that happened two steps earlier, not just the current input. And it touches a dozen external systems along the way, generating intermediate outputs that later steps depend on. Restart from scratch and you don't just lose time, you lose the thread of the whole task.

Recent research on production agentic systems put a name to the resulting mess: highly fragmented execution. Host-side orchestration and tool calls put the CPU on the critical path, and each task involves repeated handoffs between compute layers. A task used to be one clean tensor computation running on a GPU. Now it's a sprawling, data-dependent graph: many model calls, many tool invocations, many control decisions, all stitched together by orchestration code running across several host-side components. It's less "one machine doing one job" and more a relay race where the baton keeps changing hands mid-stride.

Kubernetes noticed. In March 2026, Kubernetes SIG Apps published an introduction to something called Agent Sandbox, a new CRD-based abstraction built specifically for singleton, stateful agent workloads. That's a tell. Kubernetes maintainers didn't tell teams to cobble one together from existing resources, they built a dedicated primitive. When the people who run the world's container orchestration standard decide the old toolkit doesn't cut it, that's not a minor complaint.

What agents actually need boils down to four things. Isolated execution environments that spin up in milliseconds, not minutes. Durable state that survives across the whole task lifecycle, not just one function call. Coordination primitives for multi-agent work, spawning sub-agents, routing tasks between them, pulling results back together. And the ability to pause and pick back up later without re-initializing everything from zero. Each pattern in the rest of this piece answers one or more of those four needs.

Async dispatch and the run-identifier contract that replaces blocking calls

The mechanic is simple to describe, even if it's not simple to build well: a caller submits a goal and gets a run identifier back right away. The agent then does its work independently of whatever session or process started it. Close the laptop, walk away, doesn't matter.

That run identifier is doing more work than it looks like. It's the contract that lets a caller poll for status, subscribe to events, or just come back later and check. It decouples the caller's clock from the agent's clock, which sounds obvious until you remember that most software was built around the opposite assumption: that someone's waiting on the other end of the line.

This matters structurally for a plain reason. An agent that runs for twenty minutes can't hold a synchronous connection open that whole time, nobody's browser tab survives that. The caller doesn't need to be present while the work happens. And results land in a review queue instead of a response body, which changes the whole shape of how a person interacts with the system.

OpenAI's Codex app automations are a concrete example of the pattern in practice. An automation bundles an instruction with optional skills and a trigger, and when the run finishes, results show up in a review queue for someone to check on their own time. That's async dispatch taken to its logical end: the agent doesn't care if you're home.

Scheduled or cron-style execution is really the same pattern with a different trigger, time-based instead of user-initiated, but the run-identifier contract still applies underneath it. Daily issue triage, CI failure summaries, release briefs, automated bug scans: all of it fits this shape well.

Decoupling execution from the caller solves one problem and immediately exposes another. Once the agent is running unattended, its internal state has to survive on its own, with nobody watching over its shoulder. That's durable execution's job.

Durable state and checkpointing as the difference between an agent that recovers and one that restarts from zero

Async dispatch is what exposes the failure modes in the first place. Orchestration can fail partway through. LLMs are probabilistic, so they sometimes wander into a branch nobody expected. Tool calls time out or return garbage. Processes crash, clusters get rescheduled, and the task that was mostly done is suddenly gone.

Standard retry logic doesn't help here. It assumes operations are stateless and idempotent, meaning running them twice causes no harm. Agent steps are neither of those things. Retry a step that already sent an email or already wrote to a database, and now there are two emails and a duplicate row.

Durable execution is the fix: a programming model that guarantees a piece of code finishes despite failures along the way, through automatic state persistence, automatic retries, and the ability to resume a workflow where it left off. Inference.sh breaks it into four working parts. State checkpointing saves the agent's complete state after every meaningful step, every LLM call, every tool return, every decision point. Resumability means restarting from the most recent checkpoint instead of the beginning when something stops. Retry logic handles the transient failures with sensible backoff instead of hammering a broken API. And long suspension support lets an agent pause for minutes, hours, or until a human actually responds.

Not every framework that claims checkpointing is built the same way, and the difference matters more than the marketing suggests. Diagrid points out that LangGraph, CrewAI, and Google ADK all implement some form of checkpointing, but they're not built toward true durable execution. In something like Dapr Workflows, every await point becomes a checkpoint automatically, no explicit save call required. A durable reminder gets created before each step, and if the process crashes, if Dapr crashes, even if the whole cluster goes down, that reminder wakes the workflow back up and retries until it succeeds.

The infrastructure world has caught up to this in the last year. AWS announced Lambda Durable Functions in December 2025, with durable execution capabilities baked into the platform. Microsoft updated its Durable Task offering for AI agents between March and May 2026, with the Consumption SKU going GA on March 31, 2026, positioning the Durable Task Scheduler as checkpointing and coordination infrastructure that agent frameworks can plug into. Cloudflare Workflows does the same job for durable multi-step execution on Workers. According to Inngest, durable execution crossed into the early majority in 2025, pushed mostly by the needs of AI agent infrastructure, with AWS, Cloudflare, and Vercel all shipping offerings around the same window.

Framework-level support has gotten more specific too. LangGraph has one of the stronger agent-native checkpointing setups going, saving graph state at each superstep and organizing runs by thread when a checkpointer is attached. OpenAI's Agents SDK, updated in April 2026, points toward externalized agent state, snapshotting, sandbox-aware orchestration, and rehydrating a paused agent into a fresh container. AutoGen supports saving and loading agent and team state, including full message threads and group-chat manager state.

For long-horizon workloads running hundreds of turns, checkpointing stops being a nice-to-have and becomes an economic necessity. A crash with no checkpoint means re-running everything that came before it, which is expensive in both time and API cost. Spot or preemptible compute instances are cheaper but can get pulled out from under a running task with almost no warning, so checkpoint and restore is what keeps that progress from evaporating.

Open-source platforms that deploy inside a team's own infrastructure, OpenHands is one example, let teams pick their own checkpointing strategy and durable execution layer rather than being stuck with whatever a closed platform decided for them. Model-agnostic, infrastructure-agnostic deployment means the checkpointing guarantees are actually visible and auditable, not a black box someone has to trust blindly.

Sequential chaining and orchestrator-worker decomposition as the two load-bearing planning patterns

Sequential chaining is the simpler of the two, and it's exactly what it sounds like. A complex task gets broken into a fixed sequence of steps, and each output feeds into the next step as input. The model only has to do one thing well at a time, which cuts the cognitive load per step way down. The developer knows the sequence ahead of time, and the agent just follows predefined code paths through it.

This fits well for context that has to carry across multi-turn pipelines: customer support agents, structured document processing, anything where the shape of the task is known before it starts. It breaks down the moment the task's structure can't be fully mapped out in advance, because there's no fixed sequence to follow.

Orchestrator-worker, sometimes called plan-and-execute, is a different animal. A central orchestrator LLM breaks the task down dynamically, and the subtasks come out of the orchestrator's reasoning about the specific input in front of it, not from code someone wrote in advance. The orchestrator hands each subtask off to a specialized worker, one might query a database, another might call an external API, and then pulls the results back together.

Google Cloud's architecture documentation describes it plainly: the coordinator analyzes and breaks down the request, sends sub-tasks out to expert agents, and uses an AI model to route work dynamically. That routing is the whole distinction from chaining. Nothing is fixed ahead of time.

There's a side benefit worth calling out: because the orchestrator has to state its plan before executing, that plan becomes something a person can actually review. Clearer audit trail, and stakeholders get a chance to catch a bad plan before it turns into a bad action, instead of finding out during a tool call that already fired.

None of this is free. Orchestrator-worker adds real upfront computational overhead just for the planning step, and that cost has to earn its keep against how complex or high-stakes the task actually is. As a rule of thumb, sequential chaining is the right default when the task shape is known and stable. Orchestrator-worker earns its cost when the task structure itself is uncertain or depends heavily on the input. And however these break down, both patterns benefit from the durable execution infrastructure described earlier, which can wrap individual steps to preserve progress across failures.

Parallelization and iterative refinement as patterns that trade latency and cost against quality

Parallelization splits a big task into independent pieces and runs them at the same time across multiple agents or model calls. It fits naturally where sub-problems genuinely don't depend on each other: code review, evaluating job candidates, A/B testing, building guardrails around another system. The obvious payoff is speed, cutting time to resolution by running things concurrently instead of one after another. There's a second, less obvious payoff too: when several agents evaluate the same piece of work independently and their results get aggregated, consensus can surface disagreements that a single pass might not surface.

The constraint is strict, though. This only works when the sub-tasks truly have no data dependency on one another. Force parallelization onto tasks that secretly depend on each other's outputs, and the result is coordination bugs that are miserable to debug, because everything looks like it ran fine in isolation.

Iterative refinement runs the opposite way: generate, evaluate, revise, and loop until a quality bar is hit or an iteration limit runs out. It can produce results that are genuinely hard to get in one shot, which makes it useful for polishing something complicated. But the tradeoff is straightforward: every extra loop adds latency and operating cost, so the exit conditions, a quality check or a hard iteration cap, have to be designed carefully or the loop just runs wild and burns money.

There's a variant worth naming separately: reflection, where one agent generates and a second one critiques against defined criteria, then the first revises based on that critique. It's a two-agent quality loop rather than one agent stuck evaluating its own homework. Codebridge.tech points to this as the right fit for regulated or high-stakes work where mistakes are genuinely expensive: legal contract review, medical reasoning checked against clinical guidelines, security audits run on generated code before it hits CI/CD.

The practical rule here is simple, and worth holding onto: only add a refinement loop once failure data actually shows that single-pass generation isn't hitting the bar. Without that evidence sitting in front of someone, the extra cost isn't justified, it's just spending compute to feel thorough. Parallelization trades more compute for less wall-clock time. Refinement trades more time and money for higher quality. Neither is free, and pretending otherwise is how a 5-hour-a-month time saver turns into a compute bill nobody wants to explain in the next budget meeting.