A brief history of AI executives calling for regulation

Over the past few days, a lot of people who stand to make a lot of money from AI all publicly agreed that it’s time to make everyone slow down before we lose control — including OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofounder Demis Hassabis, Microsoft CEO Satya Nadella, and X CEO Elon Musk.

When people who profit from something declare that it’s dangerous and needs to be regulated, there’s always reason to be skeptical. But whatever their reason, this is far from the first time AI thought leaders have sounded the alarm.

Charles Darwin’s theory of evolution prompted author Samuel Butler to warn about intelligent self-replicating machines replacing humans back in 1863 — and in his follow-up 1872 book Erewhon, he describes visiting a fictional society that has already regulated machines out of existence to avoid humanity’s collapse. Computing pioneer Alan Turing famously warned in a 1951 lecture that artificial intelligence would “meet with great opposition,” because undying machines “would not take long to outstrip our feeble powers” and would eventually “take control.”

Calls for intervention, too, aren’t new. In 2000, Sun Microsystems cofounder and chief scientist Bill Joy wrote that self-replicating robots could be more dangerous than nuclear weapons, partly because they’re being developed “within the now-unchallenged system of global capitalism” rather than in tightly controlled government labs. And in 2009, Microsoft researcher Eric Horvitz, who now serves as the company’s chief scientific officer, brought leading AI researchers together to consider “creating policies that might constrain or bias the behaviors of autonomous and semi-autonomous systems so as to address concerns.”

But modern AI bigwigs warning that their investments should be controlled? That trend appears to have begun with Elon Musk, who invested earlier than most.

By the time Elon Musk told a gathering of US governors that they needed to regulate AI right away, the SpaceX and Tesla CEO already had skin in the game. He’d already invested $38 million in OpenAI, as well as stakes in at least two other AI companies, and he’d publicly spoken about the dangers of AI for three years.

In June 2014, he suggested AI research could produce a real-life Terminator, called AI “potentially more dangerous than nukes” that August, and suggested that working on AI was “summoning the demon” that October. In January 2015, he joined Stephen Hawking in signing an open letter asking for responsible AI research; Hawking had previously warned that the creation of AI might be the last thing humans ever do.

“AI is a rare case where we need to be proactive about regulation instead of reactive. Because I think by the time we are reactive in AI regulation, it’s too late,” Musk told a gathering of US governors at the July 2017 National Governors Association summer meeting. In March 2018, he added: “AI is far more dangerous than nukes. So why do we have no regulatory oversight?”

In the wake of Facebook’s Cambridge Analytica data scandal, the Meta founder suggested his company should be regulated — if it’s the “right” regulation. It was largely not about AI, but he did go there:

Now that companies increasingly over the next five to 10 years, as AI tools get better and better, will be able to proactively determine what might be offensive content or violate some rules, what therefore is the responsibility and legal responsibility of companies to do that?

In March 2019, he added that the internet needs a “more active role” for regulators.

Earlier in 2018, Microsoft began taking fire for working with ICE on facial recognition technologies, at a time many tech CEOs had condemned Donald Trump’s policy of separating children from parents who are arrested at the US border.

While this form of AI isn’t quite the same as self-replicating intelligence, Microsoft president Brad Smith opened the door to voluntary regulation in a speech at the Brookings Institution, saying, “We believe it’s important for governments in 2019 to start adopting laws to regulate this technology. … We believe that the only way to protect against this race to the bottom is to build a floor of responsibility that supports healthy market competition. And a solid floor requires that we ensure that this technology, and the organizations that develop and use it, are governed by the rule of law.”

“[T]here is no question in my mind that artificial intelligence needs to be regulated. It is too important not to,” wrote Pichai in an editorial for the Financial Times. “The only question is how to approach it.”

“Companies such as ours cannot just build promising new technology and let market forces decide how it will be used,” he added. Like with Microsoft a year earlier, these comments emerged around fears about AI facial recognition specifically.

In 2023, as generative AI fervor began to reshape the entire tech industry, scientists began suggesting that companies should slow down their research — or else.

“If such a pause cannot be enacted quickly, governments should step in and institute a moratorium,” the letter, published by the nonprofit Future of Life Institute and signed by Musk, among others, reads.

Two weeks later, Musk announced he’d created his own new AI company. Two years later, he would use DOGE to dismantle chunks of the federal workforce instead of helping to regulate anything.

Brought before the US Senate for a series of AI hearings, Altman agreed that Congress should create a new agency to regulate artificial intelligence. “We believe that the benefits of the tools we have deployed so far vastly outweigh the risks,” he argued. “However, we think that regulatory intervention by governments will be critical to mitigate the risks of increasingly powerful models.”

In another hearing on May 25th, Anthropic’s CEO warned US senators that AI could help create bioweapons.

When OpenAI’s CEO was before Congress, we noted that the tone was friendly and they seemed to be trying to shape legislation together; here is an example of Microsoft trying to shape future regulation as well.

One regulatory example: “New laws would require operators of these systems to build safety brakes into high-risk AI systems by design. The government would then ensure that operators test high-risk systems regularly to make certain that the system safety measures are effective.”

“Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”

While the 22-word statement doesn’t explicitly ask the government to step in, it’s hard to think world leaders wouldn’t raise their eyebrows even higher than before. OpenAI CEO Sam Altman and then-Google DeepMind CEO Demis Hassabis were among the cosigners.

In a private Senate meeting, closed to the public and media, tech CEOs reportedly agreed that the AI industry needed regulation. One set of remarks was not so private: Meta CEO Mark Zuckerberg said that “Congress should engage with AI to support innovation and safeguards.”

“We think policymakers, academics, civil society, and industry should all work together to minimize the potential risks of this new technology, but also to maximize the potential benefits.”

Between July 2023 and May 2024, tech companies signed a flurry of nonbinding agreements with governments that would supposedly keep them in check:

Also in September 2023: A book from Inflection AI CEO Mustafa Suleyman — who is now Microsoft’s AI chief — argued that governments should regulate.

“This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill,” Musk posted in August. Anthropic originally came out against it in July, as did OpenAI in August, but Anthropic changed its tune in September. California Gov. Gavin Newsom vetoed SB 1047, though, stating that “I do not believe this is the best approach to protecting the public from real threats posed by the technology.”

“Governments should urgently take action on AI policy in the next eighteen months. The window for proactive risk prevention is closing fast,” Anthropic wrote in an open letter (bolding theirs).

“It is critical over the next year that policymakers, the AI industry, safety advocates, civil society, and lawmakers work together to develop an effective regulatory framework,” the company added.

The document begins by suggesting the UK stunted the development of its own automobile industry with regulation, while proposing “democratic AI” instead: “Rules and regulations for the development and use of AI should be based on the democratic values the country has always stood for.”

In a guest essay for The New York Times, Anthropic CEO Dario Amodei wrote: “Federal law does not compel us or any other A.I. company to be transparent about our models’ capabilities or to take any meaningful steps toward risk reduction. Some companies can simply choose not to.” The essay is against the Trump administration’s proposed 10-year moratorium on US states regulating AI.

He wasn’t arguing for a slowdown, though, but rather a “national transparency standard” that “would help not only the public but also Congress understand how the technology is developing.”

As California tried to regulate its AI giants again, Anthropic threw its support in early, and this one actually became law. It’s a transparency standard, like the one Anthropic was asking for, with safety reporting requirements and whistleblower protection. OpenAI lobbied against it.

This summer, shit hit the fan. It became public that rogue AI agents can — and did — attack other companies. OpenAI said it hit the brakes in August, and on September 8th, a senior Anthropic safety researcher resigned in protest, declaring there’s a 10 percent chance AI could kill all humans by the end of the decade. Another Anthropic safety researcher agreed.

Now, suddenly, the AI industry has seemingly agreed to slow down the pace of development. More:

Is this a new call for regulation? From Anthropic’s CEO, it seems to be. In his essay “We Must Pace the Frontier,” Amodei writes that “The most effective method of pacing is via regulation that targets all US frontier AI companies, as that covers even those who are unwilling to cooperate voluntarily,” and he suggests governments might embed people inside AI companies to help prevent incidents.

It’s not clear every AI company wants to go that far. “Where we will need the help of our government is for international coordination. But first we should do what we can ourselves,” OpenAI CEO Sam Altman writes. But Altman does say that he’ll commit to “independent evaluators with employee-like access” as well.

Meta’s Mark Zuckerberg also doesn’t seem to be aligned. On September 15th, he posted that each company has its own individual responsibility to move at the right pace and that Meta is already doing so. He also prominently linked back to his manifesto from August, which contains this line: “Any policy that slows American model releases — even by a month — could add significant risk to American leadership while letting foreign models race ahead.”

Regardless of what tech leaders are saying, regulation seems highly unlikely to come under the Trump administration, which has signaled it wants the industry barreling full speed ahead.

Even if these companies don’t actually want the regulation they seem to be asking for, this is one of the loudest alarm bells they’ve sounded so far — and the world is watching.