What Breaks First When You Scale an Agent From a Demo to Production

A demo agent working well on the handful of inputs you tried it on is a real signal — and it predicts almost nothing about production, because demos and production differ on exactly the dimensions that break agents. Here’s the order failures actually show up in, based on where each of the earlier pieces in this series applies.

First: the context window fills up for real

A demo runs for a handful of turns. Production runs run for as many turns as a real user needs, which is a number you didn’t pick. The first thing that breaks at scale is context management — a sliding-window or naive-summarization approach that was invisible in a 5-turn demo starts dropping load-bearing facts the moment real sessions run to turn 40 or 50. If you didn’t deliberately choose a compaction strategy, you inherited whatever your framework does by default, and you don’t know what that is until it fails on you.

Second: tool result shapes stop being consistent

A demo calls each tool a handful of times, usually against clean, well-behaved test data. Production calls the same tool against real data — empty results, malformed records, paginated responses that sometimes have one page and sometimes have forty. This is exactly the second most common cause in the hallucinated tool-call field guide: a tool’s result shape that was consistent in testing turns out not to be, and the model’s reasonable inference from a clean demo response breaks against messy real data.

Third: the tool list grows past what any prompt can disambiguate

A demo ships with the handful of tools the feature needed. Production accretes tools over months — someone adds search_v2 next to search, get_user next to fetch_user, and nobody goes back to consolidate. The disambiguation failure mode that barely mattered with 6 tools becomes a real, frequent source of wrong-tool calls once a production agent has 30.

Fourth: something the demo never exercised becomes reachable

Demos are walked through by people who know what the agent can do and mostly stay on the happy path. Production is used by people who don’t, which means the permission-boundary tiers you assigned casually during the demo phase — “eh, this one’s probably fine autonomous” — get tested against real attempts to do things you didn’t anticipate, including adversarial ones via prompt injection through tool results. An action that was never actually reachable in the demo becomes reachable the moment real, uncontrolled input flows through the same paths.

Fifth: cost and latency stop being an afterthought

A demo run costs cents and nobody’s watching a clock. A production agent running continuously, or at real user volume, turns the cost math from a rounding error into a line item somebody asks about — and turns “it feels fast enough” into a real p95 latency number that either meets a bar or doesn’t.

The pattern underneath all five

Every one of these is the same root cause wearing a different symptom: a demo’s input distribution is narrow and cooperative; production’s is wide and uncooperative. Nothing about the model got worse between demo and production — the inputs it’s now facing are just genuinely different from the ones it was validated against. The fix isn’t a single change, it’s treating “what does production input actually look like” as a question to answer before scaling up, not a series of incidents to discover it through afterward: stress-test context length with real long sessions, feed tools intentionally messy data, count your actual tool list, and audit permission tiers against what’s really reachable — before the input distribution audits you for free in production.