Amazon blames human employees for an AI coding agent’s mistake

Amazon Web Services suffered a 13-hour outage to one system in December as a result of its AI coding assistant Kiro’s actions, according to the Financial Times. Numerous unnamed Amazon employees told the FT that AI agent Kiro was responsible for the December incident affecting an AWS service in parts of mainland China. People familiar with the matter said the tool chose to “delete and recreate the environment” it was working on, which caused the outage.

While Kiro normally requires sign-off from two humans to push changes, the bot had the permissions of its operator, and a human error there allowed more access than expected.

Amazon described the December disruption as an “extremely limited event” that pales in comparison to a major outage in October, which took down online services, like Alexa, Fortnite, ChatGPT, and Amazon for hours. An outage that didn’t trap anyone in their smart bed is something of a lucky escape.

It is not the only time AI coding tools have caused problems for Amazon. A senior AWS employee said the December outage is the second production outage linked to an AI tool in the last few months, with another linked to Amazon’s AI chatbot Q Developer. The employee described the outages as “small but entirely foreseeable.” Amazon said the second incident did not impact a “customer facing AWS service.”

Amazon blames human error for the problems, not the rogue bot, and said it has “implemented numerous safeguards” like staff training following the incident. The company said it’s a “coincidence that AI tools were involved” and insists that “the same issue could occur with any developer tool or manual action.” That’s true, and though I’m not an engineer, I’d guess one wouldn’t deliberately scrap and rebuild something to make a change in all but the most dire of circumstances.