# Production AI Is an Operations Problem, Not Just a Model Problem A model can produce a useful answer in a prototype and still be nowhere near ready for production. That distinction matters because a successful proof of concept usually answers a narrow question: Can the model perform this task? Production asks a much larger set of questions: What happens when an upstream dependency is unavailable? Who owns the system after deployment? What data can the model access? What happens when a tool action succeeds but the response is lost? How do operators understand why a particular result occurred? When should automation stop and escalate to a human? How is the system updated, rolled back, and monitored? The model is one component. Production AI is the operating system around it. ## The production boundary starts where the demo stops Consider a simple prototype: User request ↓ LLM ↓ Answer This can demonstrate genuine capability. But once the system becomes part of a real workflow, the architecture changes. User / Application ↓ Application Layer ↓ Identity + Authorization ↓ AI / Model Layer ↓ Policy + Validation ↓ Tools / APIs / Data ↓ Downstream Systems Telemetry + Audit + Evaluation ↓ Operators and Human Escalation. Now the engineering problem is no longer just model quality. It includes identity, permissions, state, dependencies, telemetry, failure handling, cost, and operational ownership. That is why production AI should be treated as a systems problem. ## A good answer is not the same as a reliable operation Suppose an AI agent can create a support ticket. The prototype looks successful: A user describes a problem. The model classifies it. The agent calls the ticketing API. A ticket appears. But production introduces failure states between those steps. What if the API creates the ticket but the network response times out? The caller may interpret the operation as failed and retry it. The retry can create a duplicate ticket. The important question is no longer whether the model selected the right tool. The important question becomes: What state does the system know with certainty after an uncertain failure? This is where ordinary distributed-systems concerns become part of AI engineering: operation IDs, idempotency, durable state, retry policies, reconciliation timeout behavior, audit records Adding an LLM does not remove these problems. In some workflows, it adds another decision-making component to an already distributed system. ## The model should not be the authorization system A common architectural mistake is allowing an agent to decide both what should happen and whether it is allowed to happen. Those are different decisions. For example: Agent proposes: "Refund this customer $500" Policy layer checks: * Is this agent allowed to request refunds? * Is the amount within its permitted limit? * Is the transaction reversible? * Does this customer qualify? * Is human approval required? The model can participate in deciding what action appears appropriate. The system should independently enforce whether the action is permitted. This distinction becomes especially important when an AI system can call tools or interact with downstream services. OWASP's guidance on excessive agency highlights risks created by unnecessary functionality, excessive permissions, and excessive autonomy. A safer production pattern is: Model proposes action ↓ Structured request ↓ Schema validation ↓ Authorization check ↓ Policy evaluation ↓ Approved action OR human escalation The control boundary should remain deterministic even if the component making the recommendation is probabilistic. ## Observability has to include the AI decision path When an ordinary service fails, an operator may already have metrics, logs, and traces. An AI-enabled workflow creates additional questions: Which model or configuration handled the request? Which tools were available? Which tool was selected? What external dependencies were called? Did validation reject the proposed action? Was the result escalated? Did the workflow retry? What happened downstream? Without this context, an incident can become difficult to reconstruct. OpenTelemetry defines its role around generating, collecting, and exporting telemetry such as traces, metrics, and logs; it is not itself the storage or visualization backend. For production AI, the useful question is not simply: Do we have AI monitoring? It is: Can we reconstruct the path from request to model decision to tool execution to downstream outcome? That usually means connecting AI-specific events to the wider service trace. For example: Trace ID: 84f... Request received ↓ Context retrieved ↓ Model invoked ↓ Action proposed ↓ Policy rejected ↓ Human escalation created The value is not the individual events. The value is the relationship between them. An operator investigating a failure should be able to understand what happened across the system boundary rather than inspecting isolated AI logs. ## Production ownership cannot be an afterthought Another difference between a prototype and a production system is that somebody has to own the behavior tomorrow. That includes decisions such as: Who responds when the model provider is unavailable? Who updates a dependency? Who reviews changes to tool permissions? Who investigates a harmful action? Who owns evaluation regressions? Who controls the rollback? Who receives alerts at 2 AM? Google's SRE guidance on production readiness explicitly treats operational readiness and service ownership as part of the work required before taking production responsibility. This principle applies to AI systems as well. A production AI system should not become an ownership gap between the application team, the data team, the platform team, and the people operating the underlying service. The ownership model does not need to be identical everywhere. But it should be explicit. ## The fallback path is part of the architecture AI systems will encounter situations where the preferred path should not continue. Examples include: confidence or evaluation thresholds are not met, a required dependency is unavailable, retrieved data is stale or incomplete, a policy check fails, a tool action is irreversible, a request exceeds an autonomy limit A production architecture should define what happens next. One possible pattern is: Automated request ↓ AI decision ↓ Risk / policy evaluation ↙ ↘ Allowed Not allowed ↓ ↓ Execute Human review ↓ ↓ Verify Approve / reject The human path is not necessarily evidence that automation failed. It can be an intentional control boundary. For low-risk and reversible operations, more autonomy may be acceptable. For high-impact, irreversible, or security-sensitive operations, the system may need stronger controls and explicit approval. The important point is that escalation should be designed before the incident that makes it necessary. ## Production changes require controlled rollout AI systems change in more ways than conventional applications. A production behavior change might result from: application code prompts model versions retrieval data tool definitions policies evaluation thresholds external APIs Treating all of these as harmless configuration can create operational risk. Progressive rollout, monitoring, and rollback remain useful engineering patterns. Google's SRE guidance recommends staged rollouts and emphasizes monitoring changes so unexpected behavior can be detected and rolled back. The exact rollout mechanism depends on the architecture, but the operational question remains stable: If this change makes the system worse, how do we know, and how do we stop it? A deployment process that cannot answer that question is incomplete. ## A production AI readiness model Instead of asking whether the model is "ready," it can be more useful to review several system layers. ### 1\. Capability Can the model perform the required task at an acceptable level for the intended workload? ### 2\. Data Are the inputs, retrieval sources, schemas, freshness requirements, and data boundaries understood? ### 3\. Control Are permissions, validation, policy checks, autonomy limits, and approval gates explicit? ### 4\. Reliability What happens during timeouts, partial failures, duplicate requests, dependency outages, and retries? ### 5\. Observability Can operators connect requests, model activity, tool calls, dependencies, failures, and outcomes? ### 6\. Operations Who owns incidents, changes, evaluation regressions, maintenance, and recovery? ### 7, Recovery Can the system degrade safely, fall back, escalate, reconcile state, or roll back a harmful change? A weakness in any one layer can matter more in production than an incremental improvement in model quality. ## The engineering decision is bigger than model selection Model evaluation is still important. But a system does not become production-ready merely because it selects the right model. The production question is broader: Can this capability continue to operate safely when dependencies fail, inputs change, permissions matter, operators need answers, and a human has to take responsibility for the outcome? That is the point where AI engineering becomes operations engineering. The model may be the most visible component. It is rarely the only component that determines whether the system can run for real.