A working PoC is not a business case
Large general-purpose models and coding agents have made it easier than ever to build an impressive AI proof of concept. Making the same solution ready for production is far harder. It has to be tested on the right data, scale to full volume, integrate with the rest of the company's systems and have its performance tracked once it is live. Each of these steps adds cost. A working PoC opens the investment discussion; it does not settle it.
A proof of concept is the cheapest moment in the life of an AI project. It needs a sample of data, a small team and a few weeks. What comes after costs more, and not only because the volume grows. Production adds work the PoC never had to do: data pipelines, integration with core systems, monitoring, retraining, human review and security controls, each with a running cost of its own.
Many projects stop at that jump. In research conducted with Lenovo, IDC found that for every 33 AI proofs of concept a company launched, only 4 reached production.1 IDC did not blame the models. It pointed to “the low level of organizational readiness in terms of data, processes and IT infrastructure.” The ratio is unlikely to improve by itself, because PoCs keep getting cheaper. With large general-purpose models and coding agents, a small team can now build a convincing AI PoC in days, which makes the distance to production easier to underestimate.
A PoC answers a narrower question than it seems
A PoC is designed to answer one question: can this idea work? A clean sample of data, a few willing users, a controlled environment and a team that fixes problems by hand are the right conditions for that question. Production asks different ones. Will the system work on all the data, for every user, every day? Can it run at a cost the business case can carry? Will anyone notice when its performance drops?
| What the PoC shows | What production also requires |
|---|---|
| Works on a sample of data | Works on all data, including edge cases and data that changes over time |
| Runs for a few users | Runs at full volume within response-time limits |
| Stands on its own | Integrates with core systems, access rights and security controls |
| The team fixes errors by hand | Monitoring catches errors and drift, and someone owns the fix |
| Costs are small, often subsidised | Running costs are part of the business case |
| Works for the team that built it | Works for the people whose jobs change around it |
Foundation models and coding agents make the gap easier to miss
Two developments have made PoCs cheaper at the same time. The first is large general-purpose models such as GPT and Claude. With a few well-written prompts, they often produce acceptable results on the first attempt, without a custom model or weeks of data preparation, and for a PoC that is usually good enough. Whether the same model gives consistent answers across thousands of different cases, avoids errors on the edge cases and runs at full volume at a reasonable cost is a separate question. Only systematic testing on real data can answer it.
The same models are not always the right choice for production. A large model called through an external service adds response time that a real-time process may not tolerate, its cost grows with every request, and sending company data to an outside provider raises questions of privacy and vendor dependence. Many production systems therefore run on a smaller model trained or tuned for the specific task, or on classical machine learning, and reserve large models for the cases that need them. Gartner likewise expects the systems that stand out to send routine work to small, cheap models and keep large models for complex tasks.2 A PoC built on the largest available model tests whether the task can be done at all, not whether it can be done that way at scale.
The second is coding agents and AI-assisted development, which let a team build the interface and connections around the model in days. In Google Cloud’s DORA 2025 research, based on about 5,000 technology professionals, 90 percent of respondents said they use AI at work. The research found that AI adoption goes together with higher software delivery throughput but lower delivery stability, and its central conclusion is that AI amplifies a team’s existing strengths and weaknesses.3
For executives, the implication is plain. A working PoC has become cheap evidence. It shows that a team can build something, not that the organization can run it. “We already have a working PoC” should open the investment discussion, not close it.
The model is the small part of the system
In 2015 a group of Google engineers showed, in a paper titled Hidden Technical Debt in Machine Learning Systems, that the model code is only a small part of a real machine learning system. The data, configuration, monitoring and serving infrastructure around it is far larger and more complex.4 The observation predates generative AI and applies to every kind of AI system in use today. A PoC mostly tests the small box in the middle of the picture.
What an AI system needs after the PoC
- Data collection
- Data validation
- Data pipelines and features
- Integration with core systems
- Model
- Serving at full volume
- Monitoring and drift detection
- Retraining
- Human review of uncertain cases
- Security, access and compliance
- Configuration and versioning
- Process and role changes
Three systems that worked in testing and struggled in use
The gap between PoC and production rarely appears as a single failure. It usually surfaces in one of three places.
When the model meets new data. Epic’s sepsis prediction model was widely used in US hospitals. When researchers at Michigan Medicine tested it independently on 38,455 hospitalizations, the results fell well short of what the developer had reported. The model missed two out of every three patients who developed sepsis, and clinicians would have had to assess eight flagged patients to find one real case.5
When the system meets the real setting. Google Health’s system for detecting diabetic retinopathy reached more than 90 percent accuracy in the lab. In 11 clinics in Thailand, more than a fifth of the images were rejected as too low in quality, slow internet connections delayed results, and nurses and patients had to work around the system.6 The model was accurate. The setting was not the one it had been built for.
When the model meets scale and money. Zillow used its home price forecasts to buy and resell homes through Zillow Offers. In November 2021 it announced that it would close the business. Its homes segment lost $421.6 million in the quarter, and the company cut about a quarter of its workforce. Chief executive Rich Barton said that “the unpredictability in forecasting home prices far exceeds what we anticipated.”7 An error margin that is tolerable in a price estimate became a balance-sheet risk once the company was buying homes at volume on the strength of it.
Running costs belong in the business case
Production costs differ from PoC costs in kind, not only in size. For generative AI, Gartner expects the cost per token to fall by more than 90 percent by 2030, yet agentic tasks use 5 to 30 times more tokens than a standard chatbot, so total spending is expected to rise.2 In customer service, Gartner predicts that the generative AI cost per resolution will exceed $3 by 2030, more than many offshore human agents cost.8 Classical machine learning has running costs of its own: data pipelines, retraining, monitoring and human review of the cases the model is unsure about. None of them appears in a PoC budget.
For executives and innovation leads, the practical change belongs at the gate between PoC and scale in the stage-gate model we described earlier. Four questions are worth asking before approval. Has the model been tested on unseen data from the setting where it will run? Who monitors it, and what happens when performance drops? What does a completed task cost at full volume, including human review? Which process changes around it, and who owns that change? A PoC that cannot answer them may have shown that the idea can work. It has not yet shown that the organization can run it.
Footnotes
-
CIO.com, 88% of AI pilots fail to reach production, but that’s not all on IT, reporting IDC research with Lenovo, 25 March 2025. ↩
-
Gartner, Gartner Predicts That by 2030, Performing Inference on an LLM With 1 Trillion Parameters Will Cost GenAI Providers Over 90% Less Than in 2025, 25 March 2026. ↩ ↩2
-
Google Cloud, Announcing the 2025 DORA Report, 23 September 2025. ↩
-
D. Sculley et al., Hidden Technical Debt in Machine Learning Systems, NeurIPS 2015. ↩
-
A. Wong et al., External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients, JAMA Internal Medicine, 2021. ↩
-
MIT Technology Review, Google’s medical AI was super accurate in a lab. Real life was a different story., 27 April 2020. ↩
-
CNBC, Zillow says it’s closing homebuying business, cutting 25% of workforce, 2 November 2021. ↩
-
Gartner, Gartner Predicts GenAI Cost Per Resolution for Customer Service Will Exceed Offshore Human Agent Costs by 2030, 26 January 2026. ↩