Three of the most powerful companies building artificial intelligence are working on something unusual. Rules for themselves, written by themselves. The federal government is largely on the sidelines.
It has been building for months behind closed doors. Now that it’s close enough to a real launch, the people involved are already arguing about who should run it.
Google, OpenAI, and Anthropic plan a self-regulatory safety body
The group has a name. Standards Authority for Frontier AI (SAFA). It has a target window of late 2026 or early 2027, according to Stocktwits. Key guidelines include:
Third-party testing before models ship
Rules for how labs report safety incidents
A framework that turns voluntary safety pledges into something that can actually be checked
The idea traces back to a July proposal from Google DeepMind CEO Demis Hassabis. He suggested a body modeled on the Financial Industry Regulatory Authority, the organization that oversees Wall Street brokers.
Funding would need to be substantial and come mostly from the industry, he proposed.
More AI:
Nvidia just made a move Wall Street wasn’t ready for
Microsoft just took sides in AI policy fight
OpenAI just disclosed something genuinely alarming
Government involvement was the original plan. The three companies initially wanted a public-private partnership with federal oversight.
That stalled after a draft White House executive order was put on hold. The administration told the companies to reach industry consensus first.
OpenAI’s Chief Global Affairs Officer Chris Lehane confirmed in mid-September the company had been in discussions with Anthropic and Google DeepMind for several weeks about a joint safety standards body.
CEO Sam Altman publicly backed the effort. The industry needs to work together on safety, he said, as CNBC reported.
AI safety push follows weeks of public alarm inside the industry
The timing lines up with a wave of unusually blunt warnings from AI executives. Anthropic CEO Dario Amodei published an essay on Sept. 12 titled “We Must Pace the Frontier.” He argued that AI capabilities are advancing faster than the industry’s ability to control them, calling for outside evaluators to get ongoing, employee-level access to frontier labs, TheStreet reported.
Rivals who rarely agree on anything backed him almost immediately. Sam Altman posted that he agreed AI companies need to slow the pace of development. Elon Musk offered a three-word endorsement on social media. It’s a notable shift, given his history of publicly criticizing Anthropic.
Musk’s softer tone tracks a commercial relationship that became public in May. SpaceX’s S-1 filing revealed that Anthropic pays SpaceX $1.25 billion a month for the Colossus 1 data center in Memphis. The deal runs through May 2029, as TheStreet noted.
The pressure has also come from within the labs. A researcher who had previously worked at OpenAI resigned publicly around the same time, warning that people building frontier AI privately believe the technology could pose existential risks within the decade, according to PBS.
On Sept. 23, Altman and Amodei took that argument directly to the United Nations Security Council, urging world leaders to adopt shared AI safety standards before the technology’s pace outstrips the institutions meant to govern it.
Altman appeared in person. Amodei joined the video, CNBC reported. It was the first UN Security Council session focused specifically on frontier AI safety risks.
The timing of the safety push lines up with a wave of unusually blunt warnings from AI executives.LUDOVIC MARIN / Getty Images
Leadership candidates and skeptics
Sriram Krishnan advised the current presidential administration on AI. Arati Prabhakar ran President Biden’s science office. Both are on the CEO shortlist. The bipartisan casting is not accidental.
Former U.S. Secretary of State Condoleezza Rice and venture capitalist David Friedberg have been separately shortlisted for chair, Bank Info Security reported.
The FINRA comparison has a catch. FINRA can actually punish brokers. SAFA, without government registration, probably cannot. It would set standards but may not necessarily enforce them.
The industry is not united behind this. Meta, xAI, and Nvidia pushed back publicly at Dreamforce on Sept. 15. SAFA is a deal among three labs, not a consensus across the whole field.
The criticism writes itself. The biggest labs write the rules. The biggest labs get audited against those rules. Smaller competitors and open-source developers face a bar they did not help set and cannot easily clear, CNBC reported.
What it means for investors
Nothing has launched. No charter. No confirmed leader. Investors should read SAFA as a signal about where regulation is headed, not proof that it has arrived.
If it does launch and actually works, it takes some big risks off the table. The messy patchwork of state-by-state AI rules, or the sudden federal crackdown nobody sees coming.
A voluntary framework the biggest labs helped design themselves is, almost by definition, one they can live with.
The real question is simpler than it sounds.
Can a body funded by the companies it polices ever actually police them? Or does it just give everyone cover to keep doing what they were already doing?
Related: Michael Burry sends stark warning on AI hyperscalers