Artificial intelligence is increasingly becoming part of the application itself, shaping how users search, learn, receive guidance and complete work. That shift creates a different engineering problem.
That shift creates a different engineering problem.
A model that performs well in a demonstration still has to operate inside a real product. It must work with users, permissions, application data, cost limits, latency expectations and failure modes. In that environment, the question is not only whether the model can produce a strong answer. It is whether the system around the model can make the answer reliable, secure and useful.
That is the focus of Virat Gohil, a senior software architect specializing in generative AI and large-scale platforms. His book, Beyond the Prompt: Building Real-World GenAI Systems That Actually Work, argues that production AI depends less on isolated prompt improvements and more on the architecture surrounding the model.
“The model is one component,” Gohil says. “The system is the product.”
The model is only one layer
Generative AI discussions often start with the model: benchmarks, reasoning ability, context windows and output quality.
Those choices matter, but they do not solve every product problem.
A capable model cannot decide whether a user is authorized to see a document. It cannot guarantee that retrieved information is current. It does not decide when an answer needs human review or how an application should behave when a dependency fails.
In Beyond the Prompt, Gohil groups those responsibilities into model, context and control: what the system generates, what information it receives and how authorization, validation, observability and oversight are handled.
That distinction matters because many AI failures do not begin with the model. A poor result may come from the wrong document being retrieved, permissions being applied too late or an evaluation process that failed to catch the issue before release.
Prompting can shape an answer. It cannot replace the application architecture around that answer.
AI features need a shared product core
Gohil’s perspective on AI platforms was shaped in part by large-scale platform work before the current generative AI cycle.
The platform later needed to support a second, substantially different audience: a global education community.
One option would have been to build another application, duplicating infrastructure, code, operating processes and future development. Instead, the system was transformed into a multi-tenant platform. Common capabilities were separated from tenant-specific functions, allowing different audiences to share the product core without forcing every experience to be identical.
That lesson applies directly to enterprise AI. The first AI feature can often survive with dedicated integrations. The second and third begin repeating the same needs: model access, retrieval, permissions, evaluation, monitoring, fallback behavior and cost controls.
At that point, the organization is not just building features. It is building a platform.
Reusable systems without forced uniformity
The goal of a shared platform is not to make every product behave the same way.
Different AI applications may need different prompts, datasets, interaction models, risk tolerances and review requirements. A customer-support assistant, an internal search tool and an agent that takes action inside a workflow should not be governed identically.
But they may still benefit from a shared foundation.
That foundation can provide model routing, retrieval, evaluation, observability, policy enforcement and deployment workflows. Individual products can then vary where they need to, without rebuilding the same infrastructure each time.
A strong platform makes variation controlled and deliberate, not accidental.
Tenant context is part of AI safety
Multi-tenancy becomes more consequential when an application includes AI.
An AI feature may retrieve documents, summarize sensitive material, recommend an action or invoke a tool. That means user identity, role and tenant boundaries are not just application details. They become part of the AI system’s behavior.
An assistant should not receive information simply because it exists somewhere in a shared platform. Its context must be restricted by the same tenant, audience, role and access rules that govern the rest of the application.
Filtering after generation is too late if unauthorized material has already entered the prompt.
That makes access control a core part of retrieval-augmented generation. The system must decide what the model is allowed to see before the model generates an answer.
For enterprise AI, security cannot be treated as a final review step. It has to be built into context selection, tool access, orchestration and monitoring.
AI performance is application performance
AI capabilities add more moving parts to a product interaction.
A single request may involve retrieval, ranking, prompt construction, model calls, validation, policy checks and sometimes tool execution. Users do not experience those as separate systems. They experience one product.
That means AI performance is application performance.
A production system may need caching, fallback models, bounded retries, circuit breakers, asynchronous workflows or a conventional non-AI path for critical functions. The most capable model is not automatically the right model for every request.
A slower, more expensive model may be justified for a high-value task. A simpler model may be better for routine classification, extraction or routing.
The engineering challenge is to match the model and supporting infrastructure to the task, rather than treating one model as the answer to every problem.
Reliability lessons from before generative AI
Many of the reliability questions around AI are not new.
Before joining Apple, Gohil worked at Airvana, a wireless infrastructure company whose systems supported carrier networks. His work included scaling an element-management system from about 800 to 2,000 network nodes while maintaining high availability.
Those systems were different from today’s AI applications, but the core engineering questions were familiar: What happens under peak load? What happens when a dependency fails? Can operators detect degradation before users experience an outage?
Generative AI adds another layer of difficulty because an application can remain technically available while the quality of its answers declines. Every service may be running, even as retrieval relevance drops or a model update changes behavior.
That is why evaluation and observability have become production infrastructure for AI. Teams need to monitor not only whether an application responded, but whether the response was useful, grounded, policy-compliant and appropriate for the task.
Build the platform before agents arrive
Agentic AI will place even more pressure on these foundations.
An agent that can take action requires more application engineering than an assistant that only generates text. It needs authenticated identity, bounded permissions, dependable data access, controlled tool use, audit trails, failure recovery, spending limits, evaluation and escalation paths when confidence is low.
Giving a model more autonomy does not reduce the need for identity, reliability and control. It makes those requirements more important.
That is why platform engineering belongs at the center of enterprise AI strategy. A well-designed foundation should make the next AI capability faster to deliver, easier to operate and safer to change.
What lasts beyond the model
Models will continue to improve. New agent frameworks, multimodal systems, orchestration tools and evaluation methods will continue to appear.
But organizations should not confuse the pace of model innovation with the durability of product architecture.
A demonstration proves that a model can do something under selected conditions. A production system must prove that the application can do it repeatedly, securely, economically and at the level of quality its users require.
The model will change. The system is what has to last.
©2026 Cox Media Group
