95% of AI pilots fail. The 5% that work do three simple things differently.

The short version

• MIT NANDA’s August 2025 report found about 95% of enterprise generative AI pilots show no measurable profit-and-loss impact, with only 5% of integrated tools reaching production.
• McKinsey’s State of AI 2024 found 65% of firms report no EBIT impact from generative AI; BCG found 74% of firms struggle to achieve and scale value, and only 4% are future-built, and that 4% earns above-average returns.
• The failures are almost all setup failures. The tool is asked to work inside a workflow no one has written down, with knowledge no one has given it, and a handoff no one has changed.
• The 5% that work do three things in order: they fix the boring paperwork, they hand over real knowledge instead of hoping, and they keep a person at the approval point.
• None of the three requires a better model. All three are available today, and the only cost is doing the unglamorous part first.

If you have ever run an AI pilot that quietly died, you probably remember the shape of it. There was a demo. People were impressed. There was a budget, a timeline and a lot of goodwill. Then, about four months in, no one could point to anything that had changed, and the renewal quietly did not happen.

Here is the thing most people get told at that point: the technology was not ready. Almost every time, that is not what happened. In August 2025 the MIT Media Lab’s NANDA initiative published The GenAI Divide: State of AI in Business 2025 and looked at what separates the pilots that pay off from the ones that do not. The barrier it names is not model quality. It is the “learning gap&rdquo: tools that do not learn from feedback and do not fit how work flows.

The technology is ready. The recipe is the part most people skip. This article is that recipe.

The honest number

We are not going to talk around the headline figure. MIT NANDA found roughly 95% of custom generative AI tools built by firms, and pilots of purchased ones, produced no measurable impact on profit and loss. Only 5% of integrated AI tools reached production with any return at all. That is the finding that travelled furthest, and it deserves its weight.

It is also not one lonely study. McKinsey’s State of AI 2024 found 65% of respondents reporting no EBIT impact from generative AI. BCG’s Where’s the Value in AI (2024) found 74% of firms struggle to achieve and scale value from AI. Gartner predicted at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, on cost, unclear value and risk.

Three independent research firms, three different methods, the same wall. But every one of those reports also contains its own way out.

Look at the last row of that BCG finding and the whole picture turns. Four per cent of firms did it. Same models, same year, same publicly available tools. They were future-built, and BCG found they earn above-average returns.

So the honest reading of the 95% is not “AI does not work.” It is “AI works for the people who set it up properly, and almost no one is taught how.” That is a far better problem to have. Weak technology you cannot fix. A bad recipe you can fix this month.

A setup problem, not a technology problem

AI demos well and deploys badly. Show a model a task and it impresses. Put that same model inside the company, connected to the CRM, the inbox, the diary, the accounting software, the phone, and something breaks. The teams that get through that gap tend to break in one of three ways.

The first is the queue. A marketing team fine-tunes a writing model. It produces drafts. The drafts sit in a queue, because reviewing and posting them is still someone’s job. The review step, the handoff between software and person, was always the bottleneck, and no one touched it. The pilot worked. Nothing changed.

The second is work that moves instead of disappearing. The drafting gets faster, so the human’s job becomes reviewing drafts. Harder work, invisible work, unmeasured work. The pilot reports a big efficiency gain; the team reports exhaustion. Both are telling the truth about different halves of the same deployment.

The third is the boundary. The software takes an action it should not have: a refund promised outside policy, a quote sent at old prices, a message sent to the wrong customer. The response is human and expensive. Permissions get pulled. Everything goes back to people. The pilot is filed under lessons learned.

Almost nobody walks away from AI saying “it got the answer wrong.” They walk away saying “nothing really changed”, and that is a setup problem, not a technology one.

In all three cases the model did its part. What was missing was everything around it: a written-down process, real knowledge to work from, and an agreed line for where the software stops and a person decides. Those are the three things the 5% do.

One: fix the boring paperwork first

Every business runs on a layer of work that appears in no process document at all. The CRM holds the record; a person has to notice the record changed. The invoice was paid; a person has to see the money land and chase the next step. The booking came in; a person has to copy it from one system into another.

This is Operational Drag: the invisible tax on work that only moves when someone remembers to push it. It survives because each individual push is trivial and the total is enormous. No one writes a workflow document for “remember to chase the quote on Thursday.” It is not a process. It is a person, being conscientious, every single day.

And it is where naive pilots fail. A tool connected to the CRM can only do what the CRM records. The drag lives in the gaps between the records, and those gaps are held together by people, because nothing else bridges them.

So before anything is switched on: choose one recurring workflow. Write it down end to end. Where does it start, what does it touch, who decides, what does finished look like. It takes an afternoon. It is the single highest-return hour-to-hour investment in the whole project, and it is the one nearly everyone skips because it does not look like AI.

Two: give it real knowledge, not hope

The second mistake is subtler. Teams give the system a job and hope it works the rest out. Hoping is expensive.

MIT’s report is specific that the tools which succeed are integrated into existing systems than bolted on as a new destination, and that they learn from human feedback. Translated into a Monday morning: give the system the actual material. Your current prices. Your refund policy, including the exceptions. Your standard replies, and the ones you rewrote by hand because they came out wrong. Your tone, written down in plain words.

This is the unglamorous part that makes people feel like it is not working. It is also the whole difference. A capable system with your knowledge behaves like an experienced colleague. The same system without it behaves like a clever stranger, confident, plausible, and in need of checking every time. Guess which one a team keeps using.

The good news: this compounds. Every correction you feed it is a correction it does not need again. The setup is front-loaded, and the returns arrive every week afterwards.

Three: keep a human at the approval point

The third thing sounds like a limitation. It is not. It is the reason the 5% get to keep going.

Draw the Authority Line before you switch anything on. Below the line, the software acts on its own. Above the line, a person decides. Every action type gets classified once, by someone who understands the business, not by the software guessing in the moment.

This is not the same as a human reviewing everything forever. It is a human at the point where judgment lives: the refund outside policy, the price that moved, the message going to a hard customer. Everything below the line happens, without anybody watching.

Worth saying plainly, because the boundary failures are what kill pilots: the answer to “it did something it should not have” is not to take the software away. It is to move that one action above the line. That is a five-minute fix on day one and a cancelled project on day ninety.

Then measure the right thing. Count staff-hours removed, not messages sent. And count the checking too, so the saving is real than relocated. That honesty is what separates a pilot that survives its second year from one that quietly does not.

None of these three needs the frontier model of the month. They need the boring engineering of integration, the discipline of boundaries, and the willingness to spend an afternoon on paperwork before buying anything. This is why the failure rate stays flat while the models get better every quarter. The hard part was never the technology.

Where Twinthos comes in

Digital employees are changing how work gets done. Repetitive work goes to software. People get to focus on being human: the judgment, the relationships, the decisions only a person should make. That is the whole point, and it is why the setup matters so much.

Twinthos is built as the answer to this exact failure mode. We do not sell a model, a chat interface or a transformation programme. We take one workflow, the one where work stops unless someone remembers to push it, write it down with you, wire a managed digital employee into the systems it already touches, draw the Authority Line together before anything switches on, and count the staff-hours that come back.

The £999 Operations Audit is the starting point: we measure the drag, quantify it in hours and money, and hand you the diagnosis whether or not you ever work with us. If the number is not big enough to be worth acting on, we will say so.

Next step

Start with the £999 Operations AuditBook a discovery call

Twinthos is a managed digital employee service, not software you install. It works across the email, diary and accounting systems your business already runs. Below the Authority Line it acts on its own; above the line a person decides.

Common questions

Where does the 95% figure come from?

MIT Media Lab NANDA, The GenAI Divide: State of AI in Business 2025, published August 2025. It surveyed 300 public deployments, 52 interviews and 153 executives, and found roughly 95% of custom or pilot generative AI tools with no measurable P&L impact.

So does that mean AI does not work?

No. The same research points at setup, not intelligence. The barrier MIT names is the learning gap, tools that do not learn from feedback and do not fit how work actually flows. The models pass the demos. The deployments miss the recipe.

What do the 5% that succeed do differently?

Three things, before they switch anything on. They write down and fix the boring workflow first. They give the system real knowledge, prices, policies, past replies, instead of hoping it works it out. And they keep a person at the approval point, where judgment lives.

Why does a pilot stall even when the AI is right?

Because the next step in the process is still a person’s job. The draft gets written and then sits in a queue waiting for someone to notice it. The software was never the slow part; the handoff was, and nobody changed the handoff.

What does “fix the paperwork first” actually mean?

Choose one recurring workflow and write it down end to end: where it starts, what it touches, who decides, what “done” means. Most small businesses have never written these steps down, because a conscientious person has been holding them in their head.

What does Gartner’s 30% abandonment prediction mean?

Gartner predicted at least 30% of generative AI projects would be abandoned after proof of concept by end of 2025, citing poor data quality, inadequate risk controls, escalating costs and unclear business value.

How do I know if my business is ready for this?

Count the hours. If a named person spends repeatable hours each week moving work between systems, there is something to fix. Twinthos’s £999 Operations Audit measures the drag, quantifies it in hours and money, and tells you plainly if the number is too small to be worth acting on.

Sources

1. Why most AI pilots fail, MIT Sloan, 2025 Coverage of the MIT NANDA GenAI Divide report, including the 95% figure and the stated barriers.
2. The state of AI, McKinsey, 2024 Primary source for the 65% no-EBIT-impact finding and the state-of-AI survey methodology.
3. Where’s the value in AI?, BCG, 2024 Primary source for the 74% struggle-to-scale figure, the 4% future-built cohort, and the failure-mode analysis.
4. Gartner press release, 29 July 2024 Primary source for the 30%-abandonment prediction and the reasons cited.