AI Agents in Production: What Works, What Breaks, and What It Costs
The demo-to-production gap is bigger than people expect
An agent that books a meeting in a scripted demo and an agent that reconciles invoices against a live ERP are separated by about six months of unglamorous work. The model is rarely the bottleneck. What breaks is everything around it: stale tool schemas, partial API failures, permissions that differ per user, and the agent confidently retrying an action that already succeeded. We now assume that roughly 70% of an agent project is integration and guardrails, and only 30% is prompt and model work. Budgeting the other way around is the single most common reason these projects stall at pilot.
Give the agent fewer tools, not more
Teams often hand an agent twenty tools and hope planning sorts it out. Accuracy drops sharply once tool count climbs, because the model has to disambiguate between things that look similar in a schema. The pattern that holds up is a narrow agent per workflow — a claims-triage agent, a lead-enrichment agent — each with four to seven well-named tools and a hard-coded fallback to a human. If you need breadth, route between narrow agents rather than widening one. It is also far easier to debug: when something goes wrong, you know which agent owns the mistake.
Every write action needs an idempotency key
This is the detail that separates agents you can sleep through from agents that page you. Any tool that creates, updates or sends something must accept an idempotency key generated per logical task, so a retry after a timeout does not create a duplicate refund, duplicate ticket or duplicate email. Pair that with an append-only action log — who asked, what the agent decided, which tools fired, what came back. When a client asks why the system did something in March, that log is the answer. Without it, you are guessing at a non-deterministic system after the fact.
What it actually costs to run
For a mid-volume internal agent — say 5,000 tasks a month, each involving four or five model calls with retrieved context — inference typically lands somewhere between a few hundred and a couple of thousand dollars monthly, depending on model tier and how much context you stuff into each call. That is usually not the expensive part. Observability, evaluation runs, and the engineering time to keep tool integrations current cost more over a year than tokens do. Price the total cost of ownership when you build the business case, not the API line item.
Human-in-the-loop is a design choice, not a failure
The most successful deployments we've run put a human approval step on anything irreversible: money moving, contracts sending, records deleting. The agent does the tedious 90% — gathering, drafting, cross-checking — and a person clicks approve. Teams sometimes see this as an admission the agent isn't good enough. In practice it is what makes the rollout politically survivable, and it generates a labelled dataset of approvals and rejections that you can use to tighten the system later. Remove the human step only once the rejection rate has been near zero for months.
How to scope a first agent honestly
Pick a workflow that is high-volume, rule-heavy, currently done by people who complain about it, and reversible if it goes wrong. Instrument the manual process first so you have a baseline handling time and error rate. Then build for one team, measure for six weeks, and only expand if the numbers moved. If you cannot name the baseline metric today, the project is not ready to start — and that is a cheaper thing to discover in a scoping call than in month four.
Have a project that needs this kind of thinking?
Let's talk through what you're building — free consultation, no commitment.
Get in Touch