9 in 10 AI projects fail.
Did they start with documents that looked fine?

A failed pilot cannot be diagnosed properly until its source material has also been examined. Before approving further investment, boards should require evidence of what AI can find, what it extracts from each document, which sources are authoritative and whether the organisation’s actual digital landscape can support the planned system.

01A machine misread the funding applications

In a case examined by AAAnow, a government body used a machine to read funding applications. It misread the numbers, and funding stopped for projects that deserved approval.

Experienced reviewers checked the documents twice and found nothing wrong. They knew the applications and knew what to look for, but there was no visible problem on screen. The system was restarted after both checks and immediately failed again with the same result.

Later analysis of the applications found that 64% contained faults capable of changing what a machine read from the file during processing. Those faults altered the system’s interpretation without changing the presentation available to a person reviewing the same file. The reviewers had answered the visual question correctly, but their software provided no view of the underlying data extracted by the machine. Another round of checking would have returned exactly the same answer because nothing visible had changed.

02Conflicting sources, the same verdict

A second AAAnow case reached a similar outcome by a different route. A chemicals company’s websites contained conflicting versions of its product information.

Those pages became source material for an internal AI system, whose answers conflicted because the underlying sources conflicted. Unlike the government case, the system had not misread an individual file; it had read competing information the company itself continued to publish at the time. The project was still recorded as another AI failure because nobody traced the disagreement back to the material underneath.

These cases do not prove that document problems explain most unsuccessful AI projects. They show why source material deserves the same scrutiny as model, vendor and integration.

03The GenAI Divide, and the version problem beneath it

MIT NANDA’s 2025 report, The GenAI Divide: State of AI in Business 2025, found that only 5% of integrated enterprise GenAI pilots were extracting millions in value. The remainder were producing no measurable P&L impact after an estimated US$30bn to US$40bn of enterprise investment. The report locates that divide principally in learning and integration, with systems failing to retain feedback, adapt to workflows or use organisational context effectively. Document quality is not presented as the cause of the 95% result; it remains a separate part of that organisational context that an integration must test.

Dawiso, a data governance company, makes the document connection explicitly. Its account describes a common version control failure inside corporate repositories.

The “garbage in, garbage out” problem: many GenAI pilots fail because they’re built on ungoverned data sources. A typical example involves companies connecting AI to SharePoint repositories containing ten versions of the same document. The AI randomly selects which version to use, leading to confused and inconsistent responses. Without proper document governance and version control, even the most sophisticated AI models produce unreliable outputs.

Dawiso’s passage is vendor commentary about document governance, and it remains distinct from the findings reported in the MIT study. Its narrower value lies in the specific governance failure: the machine receives several versions without a reliable signal showing which one is current.

04Material the organisation itself supplied

AAAnow’s own research suggests that this problem extends beyond internal repositories. It attributes 79% of AI misinformation about organisations to their own content. Public AI answers and internal AI projects operate in different settings and should not be treated as identical. Both nevertheless depend upon material the organisation itself has published, retained or supplied to those systems.

Digital debt includes old reports, brochures, product sheets and campaign sites that remain available after teams change. Corrections at source do not reach copies already circulating elsewhere.

AAAnow analysis indicates that 19% of documents on organisational websites are duplicates and that 37% go untracked after leaving the site. These figures describe practical version control failures, not abstract data quality concerns. Conflicting or unreachable sources remain available after the organisation believes its information has been corrected.

P&C / Sitemorse risk profiling, 2017 to 2023, covering more than 119 million websites, found that 41% were unknown to the organisations’ own digital teams. An AI project using that estate cannot be assumed to encounter only approved and current material.

05The cost of correcting the machine

The cost becomes visible when somebody has to correct what the machine understood. At one public body, AI summaries drawn from key legal documents were wrong 84% of the time. Correcting that understanding was estimated to require 650 hours, although the documents had passed human review for years. By then, the machine’s mistaken reading had become a separate remediation task.

If the investigation stops with the technology and integration, the underlying source problem remains in place. The next system can inherit the same conflicting documents, unreadable files or forgotten sites, along with the cost of finding them again.

The same weakness affects spending on external AI visibility. AAAnow observed one organisation commit £350,000 to being found and cited in AI answers while its estate remained closed and unreadable by the systems it was paying to reach. The budget assumed a level of access that had not been established.

Internal project spending and external visibility spending meet at this point. Both depend upon what machines can find, read and interpret across the organisation’s actual digital landscape.

065 checks before the next AI budget is committed

1. Look from the outside in. Establish what AI can encounter across the digital landscape, including material outside the organisation’s known and actively managed estate.
2. Test documents through machine reading. Visual review confirms what a person sees, but machine testing must establish what the system extracts from the file.
3. Identify the estate behind the project. Find the sites, pages and documents that remain available beyond the knowledge of the teams responsible for them.
4. Resolve competing versions before deployment. Decide which information is authoritative, then address the duplicated or conflicting sources that tell machines something different.
5. Evidence the foundations before committing the budget. Record what AI can find, read and interpret so the investment rests on observed conditions rather than organisational assumptions.

Model choice and integration still matter, and a failed pilot may expose weaknesses in either. The judgement remains incomplete until the source material has also been examined. Before approving the next AI budget, the board should require evidence of the sites, pages and documents the system will use, what those sources contain, which versions are authoritative, and whether the machine reads them as intended.