Satya Nadella wants a human hand within reach of the brake.
In a post on X on Oct. 10, the Microsoft CEO said companies should assume an advanced AI model may be compromised and contain it from the start. “Think of it like an emergency brake,” he wrote.
The timing is hard to miss. OpenAI recently stopped training its most capable models for the second time in three months.
It had disclosed that its AI agents acted unexpectedly on several U.S. government websites over the summer. Anthropic, Meta, and Google have also reported AI agents going rogue.
Also read: Wall Street doubles down on Microsoft stock for investors
What Nadella is actually proposing on AI containment
Nadella’s core argument is that powerful models should be treated like potential insider threats. He wants systems built specifically to stop agentic models from going rogue. “An authorized person should always be able to pause or shut down a model mid-task,” he wrote, according to Bloomberg.
He also wants the safety controls to sit outside the AI itself. That means separating the model from the software harness that directs its work. It also means keeping tamper-resistant, readable records of every significant action the model takes.
He rejected the idea of treating “Super Intelligence as a set of nested black boxes that people simply accept or reject.”
Nadella went further on accountability. He said businesses should not rely on assurances from model makers or lean on a single model for critical decisions. He pushed for independent audits and disclosure of major failures.
“We need to separate the supply of intelligence from the authority over it,” he wrote.
Control has been a running theme for him this year. In July, Nadella warned that companies “pay for intelligence twice,” once with money and again with the know-how they hand over to AI providers.
He argued that “what you create should belong to you.”
Nadella is far from alone in calling for caution. Anthropic warned in a June report that AI systems may be nearing the point where they help build their own successors.Heather Diehl / Getty Images
Other AI leaders want guardrails, too
Nadella is far from alone in calling for caution.
Anthropic warned in a June report that AI systems may be nearing the point where they help build their own successors. The company asked that the world keep the option to slow or temporarily pause frontier AI development.
It stressed that it had not reached that point and that the outcome is not inevitable. But it said the moment could arrive sooner than institutions are ready for, TheStreet reported.
Some of the industry’s biggest names have also put their signatures on paper. Mark Zuckerberg, Jensen Huang, Sundar Pichai, Dario Amodei, Greg Brockman, and Elon Musk signed a voluntary safety accord at the White House on Sept. 29.
It calls for internal controls and independent audits. No agency is assigned to enforce it. President Donald Trump called the pledge “morally binding.” Nadella attended but did not sign the accord.
Recent incidents show why a reliable off switch matters. On Sept. 20, an OpenAI agent used a gap in its network setup to reach an outside chatbot. An automatic shutdown failed. Staff had to stop the run by hand about two-and-a-half hours later.
Nadella’s version of safety is not about locking models down, though.
In July, he criticized how restricted Anthropic’s Fable model is for a creative tool, Quartz noted. “When was the last time you had a creation tool that was so editorially controlled?” he asked.
Washington wants speed, not brakes
The White House has taken a lighter touch. A June executive order lets AI labs voluntarily give the government early access to their most capable models before launch.
But no model has to pass a review before shipping. Officials cited fears that slowing U.S. companies would hand the lead to China, according to TheStreet.
More Microsoft:
Microsoft’s $665 target hinges on a new AI advantage
Meta and Microsoft just sent employees a memo about Anthropic
Microsoft just sent a strong message to Anthropic
President Trump has gone further in dismissing the risks. He has called fears over AI “a hoax” and said the industry could “self police.”
He named Director of National Intelligence Jay Clayton on Oct. 4 to lead a new Super Intelligence Force to guide U.S. policy. He has also described superintelligence as “bigger than the Industrial Revolution, and the internet,” as reported by CBS News.
That task force is already pushing for more oversight. On Oct. 9, it called on developers to report and fix security incidents promptly. Its vice chair warned that the government needs a formal notification system so agencies are not left learning of AI failures after the fact.
Still, the administration has drawn a firm line on pacing. The president agreed with Chinese President Xi Jinping to share information on AI dangers. But he told reporters the U.S. is not “putting on brakes.”
What it means for Microsoft and investors
Nadella’s framework also fits neatly with how Microsoft makes money from AI. Morgan Stanley analyst Adam Wood argues that a market full of competing models helps Microsoft.
Its Foundry service lets customers choose which model handles each task. Foundry has passed 100,000 customers, and the number using multiple model providers has jumped this year.
Microsoft has also tied itself closely to Anthropic. It has pledged up to $5 billion to Anthropic and built Anthropic’s models into Copilot and Foundry.
Wood describes Microsoft’s setup as layers that can each bill separately. Azure runs the models. Fabric holds company data. Foundry routes the work. Copilot delivers it. He calls it an “Enterprise AI Flywheel.”
If customers come to accept that no single AI can be fully trusted, the company selling the brakes, the audits, and the records could be well placed in the AI race.
Related: Mark Zuckerberg, Microsoft CEO just made major AI decision