AI fills pitch decks with promise, then reappears in post-mortems when reality intervenes. Models perform flawlessly in demos, stumble in production, and leave expensive lessons. This article examines the most visible failures, explains what went wrong, and offers a practical playbook to prevent the next one.
Why ambitious AI projects still fail
- Data and reality diverge. Models are trained on yesterday’s data while operations run on today’s. When the gap widens, performance collapses.
- Objectives get misaligned. Optimizing for speed or engagement rarely optimizes for truth or trust. Proxy metrics are not outcomes.
- Black-box pipelines block accountability. If leaders cannot reconstruct a system’s reasoning, they cannot correct it. Opaque AI is ungovernable AI.
- Context beats cleverness. Edge cases and human nuance break clean abstractions. The more regulated the environment, the more this matters.
Case 1 – IBM Watson Health
Watson for Oncology promised AI-assisted cancer treatment plans. Internal documents later showed it sometimes recommended “unsafe and incorrect” options.[1]In 2022, IBM sold its healthcare data and analytics assets to Francisco Partners, which relaunched the business as Merative.[2]
What failed:
– Overreach: marketing outpaced clinical evidence.
– Data fit: fragmented hospital data broke model assumptions.
– Missing feedback: clinicians had little way to refine outputs.
Lesson:
In medicine, progress demands proof at every step. Build an audit trail that shows how each recommendation is produced.
Case 2 – Zillow offers
Zillow used algorithms to buy homes directly from consumers. In November 2021, it announced the program’s shutdown after massive write-downs.[3]Bloomberg later reported that the home-flipping business had “racked up losses” and would be discontinued.[4]
What failed:
– Market regime shift: rising rates and supply swings invalidated the model.
– Incentives: “buy more” is not “buy profitably”.
– Blind spots: local repair costs and timing risks were under-modelled.
Lesson:
Models that touch volatile assets need buffers, field inspection, and the right to say no.
Case 3 – Amazon recruiting engine
An internal hiring model learned to penalize resumes associated with women. Amazon quietly shut it down.[5]
What failed:
– Historical bias: past data encoded discrimination.
– Weak governance: bias testing arrived after harm.
Lesson:
Assume that bias exists, and prove where it does not. Publish tests, invite auditors, then pilot under supervision.
Case 4 – Computer vision mislabeling
In 2015, Google Photos mislabeled black people as “gorillas”. Google apologized publicly[6]The company later removed the gorillas category entirely rather than risk repetition.[7]
What failed:
– Sparse representation: under-sampled data produced harmful errors.
– Temporary fix: blocking terms hid the symptom instead of solving it.
Lesson:
Diverse datasets and continuous evaluation are mandatory for global vision systems.
Case 5 – Hallucinations in the courtroom
In 2023, a U.S. judge sanctioned lawyers who filed a brief containing fake citations generated by an AI chatbot.[8]
What failed:
– Misuse: generative models create fluent text, not verified facts.
– Missing verification: no human check before filing.
Lesson:
Treat AI output as a draft, never a source. Verification must be built into every workflow with legal or financial consequence.
The scale of the problem
This is no longer anecdotal. Gartner projects that more than 40% of agentic AI projects will be canceled by 2027 as costs outpace value.[9]MIT research summarized by Tom’s Hardware found that 95% of enterprise generative-AI pilots show no measurable profit-and-loss impact.[10]
For leaders:
– Budget for verification, not just innovation.
– Fund monitoring as seriously as training.
– Reward accuracy and accountability, not launch volume.
The playbook that prevents déjà vu
Start with real workflows. Design around human tasks, not headlines.
Set true objectives. Align optimization with the result that actually matters.
Prove guardrails. Bias tests and model cards are operational essentials.
Monitor drift. When context changes, the system must know or pause.
Make audits easy. Every decision should be traceable.
Scale slowly. Pilot, measure, iterate, and earn trust before growth.
The bottom line
AI did not fail in these stories. Governance did. Incentives did. Integration did.
The fix is not greater speed. The fix is greater truth. Truth is a design choice that must live inside data, objectives, testing, and audits.
References
[1] STAT News, “IBM’s Watson recommended ‘unsafe and incorrect’ cancer treatments,” July 25 2018
[2] IBM Newsroom, “Francisco Partners to acquire IBM’s healthcare data and analytics assets,” Jan 21 2022
[3] Zillow Group Investor Site, “Q3 2021 results and plan to wind down Zillow Offers,” Nov 2 2021
[4] Bloomberg , “Zillow shuts home-flipping business after racking up losses,” Nov 2 2021
[5] Reuters, “Amazon scraps secret AI recruiting tool that showed bias against women,” Oct 10 2018
[6] WIRED, WIRED, “When it comes to gorillas, Google Photos remains blind,” Jan 11 2018
[7] The Verge, “Google ‘fixed’ its racist algorithm by removing gorillas from image categories,” Jan 12 2018
[8] Reuters, “New York lawyers sanctioned for using fake ChatGPT citations in brief,” June 22 2023
[9] Reuters, “Over 40 percent of agentic AI projects will be scrapped by 2027, Gartner says,” June 25 2025
[10] Tom’s Hardware summary of MIT research,“ 95 percent of enterprise GenAI implementations have no measurable P and L impact,” Sept 2025
(Mark Jennings-Bates, BIG Media Ltd., 2025)
