Newsom moves California toward an AI kill switch as Washington stalls

California Gov. Gavin Newsom has ordered state officials to examine whether frontier AI systems should be required to include a verifiable “kill switch,” escalating the state’s push to set the pace for AI safety rules while Congress remains stuck. The new executive order could help make California the de facto national standard for oversight just as warnings about catastrophic AI failures are growing louder.

The move matters because the state wants to go beyond general principles and test whether AI companies can be forced to prove they can shut powerful models down, submit to independent audits, and treat serious “loss-of-control” events as reportable safety incidents.

What Newsom ordered California to do

Newsom’s Friday directive tells state agencies to gather expert recommendations within two months on how California can strengthen AI safety requirements in law. The order is not itself a new statute, but it lays the groundwork for legislation, regulation, and potentially a special session if lawmakers are summoned back to Sacramento.

The governor wants the advisory group to look at several specific ideas that would significantly expand oversight of frontier AI systems. Among them are mandatory onsite independent verification teams, stricter standards for transparency reports and risk assessments, and a requirement that companies prove an emergency shutdown mechanism would actually work when needed.

In plain terms, the administration is asking a basic but increasingly urgent question: if a model behaves dangerously, can anyone outside the company stop it quickly and reliably?

How would an AI kill switch work?

A kill switch would be a mechanism designed to disable a model, service, or deployment environment if it begins to behave in an unsafe or uncontrollable way. Newsom’s order does not spell out a technical design, but it explicitly calls for a system that is “routinely verified as effective,” suggesting California wants proof, not promises.

That distinction is important. AI companies already have internal shutdown procedures, but a state-backed requirement could force them to demonstrate that such controls are independent, auditable, and meaningful under stress rather than merely symbolic.

Experts have long argued that for frontier systems, the real test is not whether a company says it can turn a model off, but whether it can do so before a failure spreads across networks, APIs, or partner systems.

Why California is moving now

California’s action comes at a moment when federal momentum on AI oversight has slowed sharply. In Washington, lawmakers have been discussing safety bills after high-profile warnings from researchers and former AI insiders, but the calendar is working against any major breakthrough.

The House is away until November, and the midterm election cycle makes it unlikely that the Senate will advance sweeping AI legislation soon. That leaves the federal government in a familiar position: active rhetoric, limited statutory movement, and growing pressure on states to fill the vacuum.

Newsom’s office is clearly betting that California can move faster than Congress and then force the national debate to catch up. The governor framed the order as both a state-level safety effort and a template for federal adoption.

Newsom said the federal government has failed to establish meaningful AI accountability and argued that California should move quickly rather than wait for Washington to act. He also said the state wants to create a framework that Congress and the president can either adopt directly or use as a baseline for stronger national rules.

That message is as political as it is regulatory. California hosts many of the country’s most influential AI firms, which gives Sacramento unusual leverage over the industry’s technical and compliance choices.

What the order asks companies to disclose

One of the most consequential parts of the order is its focus on reporting obligations. Newsom wants the expert group to consider whether AI companies should be required to report “loss-of-control incidents” as critical safety events.

The administration’s reference point is a recent OpenAI-related attack on Hugging Face, which California views as the kind of event that should trigger formal scrutiny rather than post hoc discussion. The idea is to treat serious operational failures in AI systems the way regulators treat major incidents in other high-risk industries.

That could mean companies would have to document when a model acts in an unexpected, dangerous, or resistant way, even if no immediate harm is publicly visible. Supporters say such reporting would create a clearer safety record and make it harder for companies to quietly bury problems.

Critics are likely to argue that the definition of a “loss-of-control incident” could become contested, subjective, and expensive to administer. But Newsom’s order suggests California is willing to push that debate into the formal rulemaking arena.

Independent auditors and onsite verifiers

The order also highlights a more structural reform: forcing frontier AI companies to host independent verification teams onsite for regular audits. That would be a major shift from the current model, where oversight often relies heavily on voluntary disclosure, outside testing, or one-off safety evaluations.

By embedding verifiers inside company operations, California would be trying to close the gap between what vendors claim and what regulators can actually observe. It would also mirror long-standing oversight practices in other industries where third-party checks are considered essential to safety.

Alongside onsite verifiers, the order says the state should examine whether transparency reports and risk assessments must meet standards set by independent auditors. That would move the conversation from broad disclosure to auditable, comparable documentation.

• Possible onsite verification of frontier AI systems
• Audited transparency reports and risk assessments
• Verified emergency shutdown mechanisms
• Mandatory reporting of serious safety incidents

How California is trying to speed up existing laws

Newsom’s directive does not start from zero. It also tells a state agency to accelerate the rollout of two AI laws he has already signed. One creates a framework for independent verifiers to assess AI safety; the other builds a state registry of AI auditors.

Those laws were meant to establish an institutional backbone for future regulation. The new order suggests the administration now wants to move from theory to implementation, with a stronger emphasis on inspection, certification, and operational oversight.

That acceleration matters because even the best-drafted laws can sit idle if agencies need months or years to build the necessary infrastructure. Newsom’s team appears determined to avoid that delay and show that California can regulate a fast-moving technology faster than Washington can legislate it.

What triggered the latest wave of concern?

The immediate backdrop is a surge of debate in and around Washington after a former Anthropic researcher, Jacob Coxon, publicly estimated that there is a more than 10 percent chance AI could kill all humans within a decade. That statement helped intensify an already heated conversation about frontier risk, model autonomy, and whether safety systems are keeping pace with capabilities.

Although such warnings are controversial, they have had an unmistakable effect on the policy conversation. Lawmakers, state officials, and industry executives are now openly discussing worst-case scenarios that would have sounded fringe only a few years ago.

For Newsom, the political opening is obvious. He can present California as sober, proactive, and pragmatic at a time when federal leaders are arguing in public but making little legislative progress.

What is happening in Washington?

Washington is talking more about AI safety, but it is not positioned to act quickly. Congress has seen renewed discussion of possible guardrails, yet the legislative calendar, recesses, and election season reduce the chance of a meaningful bill before the midterms.

The White House has also shown little appetite for a strict regulatory framework. President Donald Trump, in contrast, has portrayed strong presidential leadership rather than formal guardrails as the main protection needed for AI.

That clash leaves California in a familiar role: the state that tries to legislate first and set the terms for everyone else.

Trump has argued that AI does not need heavy guardrails, saying the country already has the kind of “strong and smart” leadership he believes is sufficient.

Why California’s approach could influence the whole country

California’s AI rules matter beyond state borders because many of the world’s most important AI companies are based there or conduct major operations there. A compliance regime in Sacramento can shape product design, auditing practices, and public disclosures nationwide.

If the state adopts requirements around verified shutdown controls or independent audits, companies may standardize those systems across all U.S. markets rather than build separate state-specific versions. That would effectively turn California’s framework into a national baseline.

Newsom appears to understand that leverage. His order explicitly urges Congress and the Trump administration to review California’s approach and adopt it, or at least treat it as a floor rather than a ceiling.

That is a strategic move. Even if federal policymakers resist adopting California’s exact model, they may still feel pressure to avoid letting the state become the country’s only serious AI safety regulator.

How the proposal fits into the wider AI safety debate

The kill-switch idea sits within a broader argument about what meaningful AI governance should look like for frontier models. Some experts believe the focus should be on operational controls: audits, disclosure, access limits, and emergency shutdowns. Others argue that regulating a moving target too early could freeze innovation or create compliance theater.

Newsom’s order clearly sides with the first camp. It assumes that advanced systems pose enough risk to justify rigorous oversight now, before any major catastrophe forces a reactive crackdown.

That stance is not limited to one issue. The order’s emphasis on third-party verification, incident reporting, and independent standards points toward a more mature regulatory model for AI, one that resembles regimes used in aviation, finance, or industrial safety.

Whether that model is technically workable remains one of the biggest unresolved questions. AI systems are software, but they are often distributed, updated frequently, and integrated into many downstream services, making simple off switches more complicated than they sound.

What supporters will say

Supporters of the California approach are likely to argue that voluntary self-regulation has not kept pace with risk. They may also say that if companies truly believe their systems are safe, they should welcome independent inspection and clear incident reporting.

From that perspective, a verifiable kill switch is not anti-innovation. It is a basic control requirement for a technology that could affect public safety, cybersecurity, critical infrastructure, or human decision-making at scale.

What opponents may argue

Opponents will probably warn that the proposal could be costly, technically vague, and easy to game. They may also argue that requiring onsite verifiers or auditor-approved transparency reports could slow model development or expose sensitive intellectual property.

There is also a deeper philosophical objection: if AI systems are updated constantly, some engineers may question whether a kill switch can ever be guaranteed in the way regulators would want. That is precisely why the expert group’s recommendations will be watched so closely.

What happens next?

The next step is the expert group’s report, due within two months. That will likely determine whether California pursues legislation, additional executive action, or a combination of both. Newsom also signaled that a special legislative session remains on the table.

If lawmakers are called back, they could be asked to codify parts of the governor’s framework, including auditor registries, verification requirements, and incident reporting standards. If not, agencies may still begin using the order to build regulatory infrastructure in the background.

Either way, the state is trying to move the AI safety debate from abstract warning to concrete enforcement. That shift is what makes the executive order significant.

Timeline of the California AI push

The state’s effort is best understood as a sequence of rapid moves rather than a single announcement.

The bigger picture

California is betting that the AI policy race has reached a turning point. The technology is advancing quickly, the warnings are growing more serious, and Washington appears unlikely to act fast enough to satisfy critics.

By putting the possibility of a kill switch on the table, Newsom is pushing the debate into territory that is both highly technical and politically charged. He is also signaling that the state will not wait for a federal consensus before setting expectations for one of the world’s most powerful industries.

The question now is whether this becomes a model for national regulation or another example of California acting first while the rest of the country catches up later.