Performing AI, or AI-Native?
Once an AI project is live, has any decision actually been handed to the system? It’s the simplest way to tell a performance from AI-native work.
Over the past two years I’ve looked at a lot of corporate AI project lists. The long ones run to fifty or sixty items, the short ones to about ten. After seeing enough of them, I got into the habit of running each list through one question: since the project went live, has any decision that a person used to make been handed to the system?
The decision can be small. Which customer to see first today, who should handle this email, whether this order needs a second check. If even one decision now starts with an answer from the system, the project is really using AI. If none does, then however big the model and however slick the interface, it’s still a performance. Not many projects make it through that question.
Same feature, two designs
Pharma companies often build something called NBA, Next Best Action: the system tells a sales rep which doctor to see next and what to talk about.
More than one company has built it like this. The suggestions sit in a sidebar on the CRM customer page, and reps can look if they want to. In the first month a few people click out of curiosity; after that, fewer and fewer do. Since nobody follows the suggestions, there are no results to check them against, and nobody can say whether they were any good. The project quietly fades away.
When I built my own product, I tried something different. The system drafts each rep’s weekly visit plan. Reps can change it, but when they do, they pick a reason. Managers stop reviewing every plan and look only at what was changed. The suggestions that get followed now have results that can be checked, the ones that get changed come with a reason, and it becomes much clearer where the system went wrong. Managers spend their time on exceptions instead of reports.
Both designs can run on the same model and the same data. The only difference is who writes the first draft of the visit plan.
Common kinds of performance
Among projects that haven’t handed a single decision to the system, four kinds are especially common.
One is a steady stream of POCs. Twenty a year, each demoed at a review meeting and well received, but few of them get used in the business for three full months. When one POC ends, the next one starts.
Another is renaming an old system. Someone adds a chat box to an existing tool and calls it “Smart Something,” while the process and the data stay much as they were.
A third is adding a step at the end. Nothing upstream changes, and a final step appears: “AI generates a summary.” With or without it, everyone before that step works the way they always have.
The fourth is retrieval without summaries. Legal worries the AI will say something wrong, so it may only return passages from the original documents. Used this way, a large language model does about as much as a search engine.
Why this happens
It’s hard to pin this on any one person. A few things come together and push people here.
The first is incentives. The rewards for launching a project arrive the same year: the budget gets approved, an innovation case gets written up, the year-end review has something in it. Business results take a year or two to show, and by then the project lead has usually moved to another role. Under those incentives, launching lots of projects and demoing fast is a natural response.
Then there’s the sign-off. If a decision goes to the system, someone has to answer for it when it’s wrong, and the business, IT and legal each have their own reasons for not wanting that job. With nobody signing, whatever the AI produces stays a reference.
Data is another hurdle. The use cases that matter need the company’s own data, clean, connected and detailed down to the individual customer. Most companies aren’t there yet, so they start with tools that don’t need their data, like general-purpose Q&A, writing and translation assistants. These are worth having, but competitors can buy the same ones, so they don’t set anyone apart.
What AI-native means
AI-native is a phrase people use a lot these days, and they don’t all mean the same thing. When I use it, I mean a process designed on the assumption that judgment is cheap.
Old processes assume judgment is expensive, so they batch decisions up, make them once a quarter and leave them to a few people. In an AI-native process, the system gives the first answer to a decision, and people correct what’s wrong and deal with the unusual cases. The rhythm changes too: what used to happen once a quarter can happen every week. So does the way results are measured. Hours saved and active users matter less than whether the business metric improved once people acted on the system’s answers. Every decision the system makes is recorded, can be reversed, and can be traced back to whoever approved letting it decide.
Of these changes, the first matters most: the system provides the default answer. The others mostly follow from it.
Starting with one decision
So a good place to start is one specific decision; the model and platform can be chosen later.
A good candidate usually comes up dozens or hundreds of times a week, so there’s enough feedback. The information needed to make it is already in the company’s systems and mostly reliable. And a wrong call can be undone.
I put together a way to choose in Many AI Projects Die Before Anyone Picks a Model. Once a decision is chosen, the harder part is getting the organization comfortable letting go, which I write about in AI Has to Earn Its Decision Rights.
