Many AI Projects Die Before Anyone Picks a Model
Whether an AI use case can work is largely settled before a model is chosen. Two things matter most: how much judgment it asks of the AI, and whether the company’s data can support that.
When an enterprise AI project stalls, the post-mortem usually blames the technology: the model hallucinates, accuracy isn’t high enough, responses are too slow. In the cases I’ve come across, the more common cause lies in the problem itself. The project asks the AI to make heavy judgment calls, and the company doesn’t have the data to support them. A project like that is hard to make work with any model.
A useful first look at a use case asks how much judgment it expects of the AI, then how deep into the company’s own data it needs to go. Drawn as two axes, these make a four-quadrant map. Most failed POCs end up in the square with heavy judgment and thin data, the top-left.
The two axes
The horizontal axis is how deep into the company’s own data a use case needs to go. The question isn’t whether the company has data; every company does. It comes down to three things: whether the data is structured (anything scattered across emails, spreadsheets and personal laptops doesn’t count), whether it’s reliable (a customer with three names in three systems doesn’t count), and whether it’s detailed enough (if decisions are made customer by customer but the data stops at the region, it doesn’t count). The further right, the more a use case depends on data built up over years. Data like that is hard to come by, and it’s the part nobody can copy.
The vertical axis is how much judgment is handed to the AI, in five levels from the bottom: generate a draft that a person reads in full; retrieve the relevant material from a pile; recommend a course of action that people are free to ignore; make the default decision, with people stepping in only when they disagree; act directly and leave a record. The higher up, the more a mistake costs.
The four quadrants
Use cases in the bottom-left need little data and little judgment: general-purpose Q&A, writing assistants, translation, meeting notes. They don’t use the company’s data and don’t decide anything for anyone. The products on the market are already mature, so buying them and rolling them out to everyone works fine. The risk here is small; the main trap is treating this as the whole AI strategy.
Use cases in the bottom-right use the company’s own documents and data, but only to retrieve and generate: product knowledge assistants, role-play visits, pre-call briefings. Mistakes are cheap. Buy a platform, configure it with the company’s own content, and it pays off quickly. The usual problem is that nobody maintains the content, and within six months the answers are out of date.
The top-left is a trap: heavy judgment with no data underneath. A model won’t say “I don’t know”; it will give an answer that sounds entirely reasonable. Demos in this square tend to look the best, so these projects are the easiest to get approved, and this is also where the most POCs fail.
In the top-right, the AI proposes the default plan, allocation or adjustment. It’s the hardest square and touches the most parts of the business, but it’s also the most valuable, because it changes the decisions themselves. There’s usually nothing to buy off the shelf here; it has to be built.
Same use case, different square
Pharma companies have spent the last few years building NBA, Next Best Action, where the system suggests which doctor a rep should see next and what to discuss. By name it belongs in the top-right. But at most companies the doctor-level data can’t support it: CRM entries are typed in by reps, prescription flow data reaches the hospital but not the individual doctor, and master data doesn’t match across systems. The same NBA sits in the top-right at a company with solid data, and ends up in the top-left at a company without it.
So which square a use case falls in depends on the company’s own data. The name alone says little.
A top-left project can still be rescued, in one of two ways. One is to spend the time filling in the data, pushing it to the right. The other is to lighten the judgment and pull it down: from “decide” to “recommend,” and if that’s still too much, to a pre-call briefing. The real pity is doing neither and just tuning the model.
How I use the map
When I use this map myself, I usually place each AI project on it one by one and see where they land.
Projects in the top-left generally need a clear decision: fill in the data, downgrade them, or stop them. Projects in the bottom-right are often where extra investment pays off soonest, and they tidy up the data along the way. If nothing sits in the top-right, the company hasn’t started on real decisions yet.
Within a square, one way to set priorities is by whether a mistake can be undone. A week’s visit plan that’s wrong can be fixed the next week. A wrong annual target affects a whole year of someone’s income, and their trust in the company. The reversible ones are good places to start.
I ran this map over the AI applications common in pharma commercial, and wrote up the results in Ten Kinds of AI Apps. Eight Still Stop at Answering and Suggesting.
