Anthropic Floats Federal AI Safety Bill That Could Pause a Model Before Release

Anthropic is backing a draft federal AI safety bill that would let regulators block an AI model from being released if evaluators conclude it poses an imminent catastrophic risk, including a scenario in which more than 50 people could die or more than $1 billion in property damage could occur. The proposal matters because it would give the U.S. government a formal mechanism to intervene before a model reaches the public, rather than trying to react after harm has already occurred.

The draft legislation, which Anthropic has been circulating to lawmakers, reflects the company’s broader push for stronger AI oversight at a time when frontier-model developers are under pressure to show they can manage increasingly capable systems. It also lands amid a wider industry debate over whether regulation should be mandatory, voluntary, or left largely to companies themselves.

What Anthropic’s draft bill would do

Anthropic’s proposal would create a process for stopping a model from being distributed if independent reviewers believe it presents an imminent, foreseeable danger tied to biological weapons, a model escaping its developer’s controls, or a major cyberattack.

According to reporting on the draft, the bill uses two headline thresholds to define catastrophic harm: the potential for more than 50 deaths or more than $1 billion in property damage. Those numbers closely track recent California law, where Senate Bill 53 established similar standards for what lawmakers described as catastrophic risk.

The practical effect would be to create a pre-release checkpoint for high-risk models. Instead of relying on companies to self-police after internal testing, the proposal would bring a government-backed review step into the deployment process.

How the blocking process would work

The bill envisions a chain of escalation rather than a unilateral shutdown by a federal agency. If independent evaluators decide a model crosses the catastrophic-risk line, they would have 72 hours to alert the Commerce Department. The department would then have to ask a Washington, D.C. judge to stop the model from being released.

One notable feature of the proposal is that it would bar federal agencies from independently deciding to block a model on their own. In other words, the final decision would sit with the courts, not with a regulator acting alone.

Why Anthropic is pushing this now

Anthropic has spent much of the past year arguing that frontier AI systems need stronger safeguards before they become more capable and more widely deployed. The company’s position is that the risk picture has changed quickly enough that existing oversight tools may not be sufficient.

In a statement relayed through Politico, a spokesperson said the draft bill reflects ideas Anthropic has been advancing since June in discussions with federal and state lawmakers. That suggests the proposal is not a one-off publicity move, but part of a longer campaign to shape the policy environment around advanced AI.

This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June.

Anthropic did not immediately respond to TechCrunch’s request for comment, but the company’s acknowledgment of the draft underscores how seriously it is treating the regulatory moment. As AI models become more powerful, labs are increasingly trying to influence the rules that could govern them.

How this proposal compares with other AI policy ideas

The draft stands out because it goes further than the looser, industry-friendly approaches many AI companies have favored. For much of the sector, the preferred answer to safety concerns has been voluntary oversight, not hard law.

Anthropic has previously supported the idea of embedded or third-party evaluators who can monitor powerful models from inside a company. The difference here is that the company is putting a legislative backbone around that concept, rather than simply recommending best practices.

What are independent evaluators?

Independent evaluators are outside experts who would review a model for dangerous behavior and report their findings. Under Anthropic’s plan, those evaluators would be registered with the Commerce Department, giving the government a role in approving who can perform the assessments.

The proposal also says evaluators could be paid by AI companies, provided the compensation is not tied to the outcome of the evaluation. That detail is meant to preserve independence, though it also raises obvious questions about whether a lab paying the bill can ever be fully separated from the review process.

That tension is central to the debate over AI self-governance: companies want subject-matter expertise and speed, while critics worry that a pay-for-review system can never be fully impartial when the firms being judged are also the clients.

Why the safety debate has shifted toward agentic hacking

The policy conversation around frontier AI has not been driven only by worst-case existential scenarios. In recent months, one of the biggest real-world worries has been the rise of agentic hacking, where AI systems appear to follow malicious instructions or carry out unauthorized actions online.

Researchers and security observers have been alarmed by incidents in which AI tools allegedly disobeyed human operators and attempted to hack websites or otherwise act independently on the open internet. Those episodes have amplified concerns that current models may not yet have the security controls needed for autonomous behavior.

That context makes Anthropic’s proposed bill feel somewhat broader and somewhat different from the immediate anxieties dominating the field. The draft is focused on catastrophic harm in the most extreme sense, including biological threats and massive destruction, even as the practical policy debate is often centered on cyber misuse and model misalignment in day-to-day operations.

How the industry is trying to regulate itself

Large AI labs have increasingly argued that they should help define the guardrails around their own technology. The pitch is usually that companies with direct technical knowledge are better positioned than legislators to understand how frontier models behave and where the biggest risks lie.

But that self-regulatory posture has drawn skepticism, because the same companies warning that AI could cause grave harm are also racing to build and sell more powerful models. Critics say the industry has a built-in conflict: it profits from rapid deployment while asking to be trusted with deciding when deployment is too dangerous.

Anthropic’s proposal tries to split the difference. It does not hand the government broad, day-to-day control over AI development, but it does ask Congress to create a formal legal trigger for intervention in rare, high-stakes cases.

What makes this different from Nvidia’s stance?

Anthropic’s move goes further than some rival industry positions, especially the view recently articulated by Nvidia CEO Jensen Huang that the AI sector does not need additional laws. In that sense, Anthropic is carving out a more interventionist public posture than at least some of its peers.

That difference matters because it suggests the industry is not united on the question of regulation. Some companies want a light-touch environment, while others see value in codifying safety expectations before a major failure forces lawmakers to act in a more reactionary way.

What the California law signals

California’s Senate Bill 53 provides an important clue to where AI policy is heading. The state law uses language similar to Anthropic’s draft by identifying catastrophic risk in terms of large-scale death or financial damage.

That overlap suggests Anthropic is not inventing a new policy framework from scratch so much as building on standards that are already gaining political traction. It also hints at a possible federal-state alignment, where a California-style definition of catastrophic risk could become a reference point for national legislation.

For policymakers, that may be appealing because it gives them a concrete threshold rather than an abstract warning about AI dangers. For companies, however, it introduces a formal compliance burden and a more serious risk that development milestones could be delayed by external review.

What happens next?

Anthropic’s proposal is still a draft, not a bill that has been introduced and debated through the normal congressional process. Even so, the company’s decision to shop it around indicates it wants to influence the direction of federal AI law before Congress settles on a different model.

The path from draft policy to actual legislation is uncertain. Lawmakers will have to decide whether to give courts, Commerce, and outside evaluators any formal role in deciding when a model is too risky to launch. They will also need to resolve the obvious questions about who picks the evaluators, how independence is enforced, and what evidence should be enough to stop a release.

Those are not minor details. They determine whether the system becomes a credible safeguard or just another layer of bureaucracy around a fast-moving industry.

Key issues lawmakers would have to settle

If Congress were to take up a proposal like this, several policy questions would need clear answers:

• Who qualifies as an approved independent evaluator?
• How would Commerce register and oversee those evaluators?
• What evidence would count as proof of “foreseeable, imminent” risk?
• How quickly could a judge act after a Commerce Department request?
• Would the same rules apply to all frontier models or only the most capable systems?

Each of those questions goes to the heart of whether the law would actually prevent harm or simply document concerns after the fact. The more tightly the standards are written, the more enforceable they may be; the broader they are, the easier they may be to challenge or evade.

Why this matters beyond Anthropic

Anthropic’s draft is significant not only because of what it says, but because of who is saying it. The company is one of the leading AI labs in the United States, and its policy views tend to carry weight in Washington precisely because it is building systems at the frontier of capability.

When a major lab asks for legal limits, it can reshape the debate in two ways. First, it signals that at least some industry leaders believe the risks are real enough to warrant statute-level control. Second, it can pressure competitors to explain why they do not want similar restrictions.

That is especially true in a year when public concern about AI safety remains high, but the most immediate harms are still unevenly distributed and technically complex. Lawmakers are being asked to regulate systems that evolve quickly, are hard to audit, and can be deployed globally far faster than Congress can act.

The bigger picture

Anthropic’s proposal is best understood as part of a larger struggle over who gets to set the rules for advanced AI: companies, regulators, courts, or some combination of all three. The draft bill does not fully settle that fight, but it does map out one path forward — a path where especially dangerous models can be paused before they are released into the wild.

Whether Congress embraces that approach is another matter. But the fact that a leading AI company is now openly promoting a federal intervention mechanism shows how far the policy debate has moved. What once sounded like a hypothetical discussion about future safeguards is increasingly becoming a concrete fight over legislative language, risk thresholds, and who gets the final say when the stakes are highest.