What Johnson & Johnson learned from 900 AI experiments
Most AI projects stall because of how they are managed, not because the models fail. Companies that turn AI into results run it as an innovation portfolio: stage-gates for decisions, sprints for learning, and stop criteria agreed before a pilot starts.
After three years of encouraging employees to experiment with generative AI, Johnson & Johnson counted nearly 900 use cases across the company. When its leaders reviewed them, they found that 10 to 15 percent of the use cases produced about 80 percent of the value.1 The company stopped funding most of the rest and concentrated on a small set of priorities, among them a sales copilot and tools that anticipate supply chain risk. It also moved ownership of those priorities from a central team to the business units that would use them.
Johnson & Johnson, generative AI portfolio
The lesson has little to do with the technology. Innovation management has taught it for decades: early experiments are cheap and numerous, a few of them carry most of the value, and the discipline lies in deciding which ones to stop. Many companies have yet to apply it to AI. McKinsey’s The State of AI in 2025 found that nearly two thirds of organizations had not yet begun to scale AI across the enterprise, and that only about 6 percent qualified as high performers.2 S&P Global reported that companies abandoned 46 percent of their AI proofs of concept before production, and that the share of companies scrapping most of their AI initiatives rose from 17 percent to 42 percent in a single year.3
AI projects stall on management decisions, not on models
When RAND interviewed experienced data scientists and engineers about AI projects that failed, the most common root cause was neither the model nor the data. It was a misunderstanding between leadership and the team about which problem the project was meant to solve.4 BCG reaches a similar conclusion from another direction. In its survey of 1,000 executives, about 70 percent of the difficulty in scaling AI sat with people and processes, 20 percent with technology and only 10 percent with algorithms.5
Both findings describe the same kind of uncertainty. Before an AI project starts, nobody knows whether the model will be accurate enough on the company’s own data, whether the redesigned process will improve the measure the business cares about, or whether employees and customers will adopt it. These are the uncertainties of a new product, not of an infrastructure upgrade. Yet most AI work is still governed like an IT project.
Why neither a fixed plan nor Scrum fits AI work
A fixed-scope project plan with a go-live date assumes that feasibility is known and only delivery remains. In an AI project, feasibility is the first thing in question. The plan then either absorbs the uncertainty through repeated delays or pushes a weak model into production because the date has arrived.
Scrum handles changing requirements far better, but it rests on a different promise: every sprint produces a working increment that could be shipped. An AI experiment often ends with a finding rather than an increment. The model may misclassify a quarter of the contracts from one region, or the data needed for the next step may not exist. That finding is progress, yet it has no place in a sprint review built around delivered features. Velocity and burndown charts then show a busy team while the question that matters, whether the bet should continue, stays open.
| Fixed project plan | Scrum alone | Stage-gates with sprints | |
|---|---|---|---|
| Assumes | Feasibility is known | Every sprint yields a shippable increment | Feasibility and value have to be proven |
| Unit of progress | Milestones delivered | Features delivered | Evidence gathered |
| Who decides to go on | Steering committee, at the end | Product owner, sprint by sprint | Executives, at each gate |
| Typical failure | Late discovery, sunk cost | Busy pilots that never scale | Gates that turn into paperwork |
Gates for decisions, sprints for learning
Innovation management has a tested answer to this mix of uncertainty and speed. Robert Cooper, who developed the Stage-Gate model for new product development, later described with Anita Friis Sommer a hybrid in which agile sprints run inside each stage while the gates between stages remain go or stop decisions.6 The two mechanisms work at different levels. Teams use short, time-boxed sprints to learn as fast as possible. Executives use the gates to decide, on the evidence those sprints produced, whether a bet deserves the next tranche of money and people.
Adapted to AI, five stages cover the path from an idea to a system in daily use:
- 01
Is this problem worth solving?
Sprints: map the current process, measure the baseline, write the value hypothesis
Gate 1
Pass with: a business owner, a measured baseline and agreed stop criteria
Stop if: no one owns the outcome or the baseline cannot be measured
- 02
Does it work on our data?
Sprints: build evaluation sets from real cases, compare approaches and their cost
Gate 2
Pass with: evaluation results above the agreed threshold at an acceptable cost
Stop if: accuracy or unit cost stays below the threshold
- 03
Does the redesigned process move the measure that matters?
Sprints: pilot with real users in a redesigned process, track cost and quality together
Gate 3
Pass with: the target measure improves without a loss of quality
Stop if: the gain disappears once the work runs end to end
- 04
Can we run it reliably and affordably?
Sprints: integrate with core systems, set up monitoring, train users, manage the change
Gate 4
Pass with: running cost, monitoring and support are in place
Stop if: the cost at scale erodes the value case
- 05
Is it still working?
Sprints: monitor quality, drift and adoption, and improve
Periodic review
Pass with: keep, retrain or retire, decided on current evidence
Two features make the model work. First, the evidence required at each gate is agreed before the stage starts, so the decision cannot be rewritten to fit whatever the team happened to find. Second, funding follows the gates. A project receives enough budget to answer its next question, not the full budget for a scaled system it has not yet earned.
Morgan Stanley’s assistant for its wealth management advisors shows what this looks like in practice. Before rolling out the GPT-4 based tool, the firm built a set of evaluations: advisors and engineers graded the model’s summaries for accuracy, and daily regression tests with sample questions caught weaknesses before they reached clients. More than 98 percent of advisor teams went on to use it.7 The evaluations were the evidence for the feasibility gate, and adoption followed because the decision to scale rested on that evidence.
AI projects are innovation bets, and they respond to the same discipline as any other portfolio of new products. Learning happens in sprints, where a team is free to discover quickly that an approach does not work. Money moves at gates, where executives decide on evidence agreed in advance. J&J arrived at this discipline after 900 experiments. Most companies can adopt it with far fewer.
For executives and innovation leads, the first step is to place every current AI initiative on the five stages and ask two questions of each one: which stage it is really in, and what evidence the next gate will require. In a follow-up piece, we look at where the model breaks in practice: the gate most companies skip, how to write stop criteria, and who should own each decision.
Footnotes
-
The Wall Street Journal, Johnson & Johnson Pivots Its AI Strategy, April 2025, summarised in DeepLearning.AI, The Batch, May 2025. ↩
-
McKinsey, The State of AI in 2025: Agents, Innovation, and Transformation, November 2025. ↩
-
S&P Global Market Intelligence, Voice of the Enterprise: AI & Machine Learning, Use Cases 2025, as reported by CIO Dive, March 2025. ↩
-
RAND, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed, August 2024. ↩
-
BCG, AI Adoption in 2024: 74% of Companies Struggle to Achieve and Scale Value, October 2024. ↩
-
R. G. Cooper and A. F. Sommer, Agile–Stage-Gate Hybrids: The Next Stage for Product Development, Research-Technology Management 59(1), 2016. ↩
-
OpenAI, Morgan Stanley uses AI evals to shape the future of financial services. ↩