I have spent the last couple of years watching teams bolt LLM calls onto existing systems, and one confusion keeps coming up in almost every architecture review: engineers use the word "AI" to describe two fundamentally different execution models. One is a single-turn request-response call, similar in shape to any other API call your infrastructure already handles. The other is a multi-step agentic workflow, where the model plans, calls tools, observes results, and decides what to do next, sometimes for minutes at a stretch. If you design your infrastructure for one and deploy the other, you will hit timeouts, runaway costs, or silent failures that are hard to debug because nothing actually crashed.
What It Is
Single-turn AI is exactly what it sounds like: you send a prompt, the model returns a completion, and the interaction is over. There is no loop, no memory of intermediate steps beyond what you pass in the context window, and no autonomous decision-making about what to do next. A classification endpoint that labels support tickets, a summarization service that condenses a log file, or a chatbot reply that answers one question — these are all single-turn, even if the underlying conversation has history attached. The defining trait is that the model does not take action between the request and the response. It reasons, it outputs text, and control returns to your application code.
Multi-step agentic workflows are different in kind, not just degree. Here the model is given a goal, a set of tools (functions, APIs, shell access, database queries), and permission to decide its own sequence of actions until it either completes the goal or hits a stopping condition. The model calls a tool, receives the result, reasons about what happened, and calls another tool, possibly dozens of times, before producing a final answer. In my experience, the mental model that clicks fastest for infrastructure engineers is this: single-turn AI is a function call, multi-step agentic AI is a process with its own control flow, and that process is running code you did not write line by line — it was generated dynamically by the model at each step.
How It Works
A single-turn call has a predictable shape at the infrastructure level. You have one inbound request, one outbound request to the model provider (or your self-hosted inference endpoint), and one response. Latency is bounded by the model's generation time plus network overhead. You can put this behind a standard load balancer, apply normal timeout and retry logic, and reason about capacity the same way you would for any REST API.
POST https://api.solvethenetwork.com/v1/completions
Content-Type: application/json
{
"model": "internal-llm-v2",
"prompt": "Summarize the following incident log...",
"max_tokens": 512,
"timeout_ms": 8000
}
HTTP/1.1 200 OK
{
"completion": "Summary: disk pressure on sw-infrarunbook-01 caused...",
"tokens_used": 340
}A multi-step agentic workflow replaces that single request with a loop that your orchestration layer has to own. The pattern usually looks like this: the model receives the goal and a list of available tools, it emits a tool call instead of a final answer, your system executes that tool call against a real system (an API, a database, a shell), the result is appended to the conversation, and the model is invoked again with the updated context. This repeats until the model emits a final answer or you hit a step limit.
state = { goal: "Diagnose high latency on sw-infrarunbook-01", steps: [] }
while not state.done and len(state.steps) < MAX_STEPS:
response = call_model(state.goal, state.steps, tools=AVAILABLE_TOOLS)
if response.type == "tool_call":
result = execute_tool(response.tool_name, response.args)
state.steps.append({tool: response.tool_name, result: result})
elif response.type == "final_answer":
state.done = True
state.answer = response.text
if len(state.steps) >= MAX_STEPS:
raise WorkflowTimeoutError("Agent exceeded step budget")Notice what changed. You now own a state machine, a step budget, a tool execution sandbox, and error handling for every tool in that sandbox. Each loop iteration is itself a full model call with its own latency, so a ten-step workflow might take 90 seconds where a single-turn call takes 3. This is the part teams underestimate: the infrastructure cost isn't just "more model calls," it's an entirely new category of long-running, stateful process that behaves more like a CI job than an API request.
Why It Matters
The distinction matters because it changes almost every infrastructure decision you make downstream. Timeout strategy is the first casualty. A reverse proxy configured with a 30-second timeout will happily kill an agentic workflow that's 80% of the way through a legitimate multi-step task, and the user just sees a generic error with no indication that real work was in progress. I've debugged this exact failure mode at a client running an internal ops assistant — the workflow itself was fine, but nginx was configured for chatbot-style latency and kept truncating longer agent runs at exactly 30 seconds, every time.
Cost modeling is the second casualty. Single-turn systems have a cost per request that's easy to forecast: tokens in, tokens out, multiply by request volume. Agentic workflows have variable cost per task because the model decides how many steps it needs. A workflow debugging a network issue might resolve in three tool calls on a good day and spiral into fifteen on a bad one, especially if a tool returns an ambiguous result and the model starts second-guessing itself. Without a hard step budget and per-workflow cost ceiling, you can burn through your inference budget on a handful of stuck agents.
Failure isolation is the third, and in my view the most operationally dangerous. In a single-turn system, a bad output is contained — it's one wrong summary, one wrong classification. In an agentic workflow, a bad decision at step 3 propagates: the model acts on a wrong tool result, that action changes real system state, and every subsequent step reasons from a now-incorrect world model. I have seen an agent granted write access to a monitoring config repeatedly "fix" an alert threshold based on a misread metric, compounding the mistake across several tool calls before a human noticed. This is why any multi-step agentic workflow with side effects needs the same rigor you'd apply to an automated deployment pipeline: dry-run modes, approval gates on destructive actions, and full audit logging of every tool call and its arguments.
Real-World Examples
A single-turn pattern shows up in things like automated log summarization triggered by a cron job, ticket triage that tags incoming support requests with a category and severity, or a documentation search assistant that answers one question against a knowledge base per request. Each of these fits cleanly into existing request-response infrastructure: an API gateway, a rate limiter, a timeout of a few seconds, and standard horizontal scaling.
Multi-step agentic workflows show up in incident remediation assistants that pull metrics from a monitoring system, cross-reference recent deploys, check relevant logs on hosts like sw-infrarunbook-01, and propose (or in more mature setups, execute) a fix. They also show up in infrastructure-as-code review agents that read a pull request, run a policy linter, query a cost estimation API, and post a consolidated review comment — three or four distinct tools chained by the model's own judgment about what to check next. Another common one is automated capacity planning: an agent queries historical usage from a metrics store, calls a forecasting function, checks current budget against a finance API, and drafts a scaling recommendation. None of these can be expressed as a single prompt-response pair because the next action genuinely depends on what the previous tool call returned.
A useful test I apply when a team asks "is this agentic or not": if you can write down the full sequence of steps in advance and it never changes based on intermediate results, it's not truly agentic — it's a fixed pipeline with LLM calls in it, which is really just a chain of single-turn calls glued together by your own code. Real agentic behavior requires the model to choose the next step based on what just happened, not just execute a predetermined script.
Common Misconceptions
The first misconception is that adding a loop automatically makes a system "agentic" in the sense that matters for infrastructure planning. A fixed three-step pipeline — fetch data, summarize, post to a chat channel — is not agentic even if it involves three separate model calls, because the sequence and branching are decided by your code, not the model. Calling it agentic and provisioning for open-ended, variable-length workflows when your actual system is a fixed pipeline leads to over-engineered orchestration layers for something that would run fine as a simple job queue.
The second misconception, and the more dangerous one, is treating multi-step agentic workflows as safe by default because "the model is smart." Model capability and system safety are separate concerns. An agent with tool access to production systems needs the same guardrails as a human operator with production access: scoped credentials per tool, rate limits on destructive actions, and a hard ceiling on how many actions can happen without a checkpoint. I have seen teams skip this because the demo worked fine in a sandbox with mocked tools, then deploy against real infrastructure and discover the model's occasional bad tool-call argument now hits a real API with real consequences.
The third misconception is assuming multi-step agentic workflows are strictly better or more advanced than single-turn calls, and therefore the direction every system should move in. That's backwards. Single-turn calls are cheaper, faster, easier to test, and easier to reason about in production. You should only reach for a multi-step agentic architecture when the task genuinely requires branching based on intermediate results that you cannot predict ahead of time. If a fixed pipeline solves the problem, use a fixed pipeline — it will be more reliable, cheaper to run, and far easier for the next engineer to debug at 2am.
The last one worth flagging: teams sometimes assume that because a workflow is "multi-step," retries and idempotency don't need special thought — the same way they might for a stateless single-turn call. That assumption breaks fast. If step 4 of an agentic workflow fails and you naively retry the whole workflow, you may re-execute steps 1 through 3, including any tool calls with side effects, a second time. Idempotent tool design, or explicit checkpointing of workflow state, isn't optional once real actions are involved — it's a prerequisite, the same way it would be for any distributed job orchestration system you already run.