Former Anthropic and METR Leaders Bet $40M on AI Agent Audits for Enterprises

AI safety-focused startup Artificial Intelligence Underwriting Company, or AIUC, has raised $40 million in new funding to build a third-party audit and certification system for AI agents, betting that enterprises will need outside verification before they trust autonomous software at scale. The company says its framework is already being used by customers including Cursor, Lovable, Harvey, and ElevenLabs, as businesses look for a way to measure whether agents can be deployed safely.

The new financing, announced Tuesday, was led by Ribbit Capital with participation from First Harmonic. It brings AIUC’s total funding to $55 million after a $15 million seed round backed by Nat Friedman’s NFDG, Emergence, Terrain, and Anthropic co-founder Ben Mann, among others.

Founded by Rune Kvist, an early Anthropic employee, and Rajiv Dattani, the former chief operating officer of AI safety research group METR, AIUC is trying to solve a problem that is becoming more urgent as AI systems grow more capable: the gap between what agents can do and what organizations can safely allow them to do.

The company’s pitch is straightforward. As AI agents become more useful, they also become harder to constrain, more difficult to evaluate, and more likely to create operational, legal, or security risk if they are deployed without outside scrutiny. AIUC wants to provide a standardized way for buyers to understand those risks before rolling a system into production.

Why AI agents are creating a new enterprise risk problem

AI agents are software systems that can take actions on behalf of users, not just generate text. That added autonomy is what makes them appealing for enterprise workflows — and what makes them unsettling for companies handling sensitive data or regulated operations.

In sectors such as finance, healthcare, government, and defense, the issue is no longer simply whether a model is intelligent enough. The bigger question is whether an organization can prove to customers, regulators, and internal stakeholders that the system will behave within defined boundaries.

According to AIUC, that is where deployment decisions are slowing down. Businesses are increasingly willing to experiment with advanced AI, but they want assurances around containment, reliability, privacy, and misuse resistance before they let those systems touch critical processes.

“AI is getting smarter at an increasingly rapid rate,” Kvist said, describing the paradox that more capable tools can also be more difficult to govern. He argued that adoption gets harder as systems improve, because control mechanisms do not advance at the same pace.

Kvist’s view reflects a wider tension in the market. The more autonomous agents become, the more they resemble junior workers with broad access privileges rather than simple software tools. That makes them powerful, but also potentially dangerous when they hallucinate, ignore instructions, leak data, or are manipulated through jailbreaks and other prompt-based attacks.

How does AIUC test AI agents?

AIUC says it built a testing and certification framework modeled on a familiar cybersecurity concept: SOC 2. In the same way that SOC 2 gives companies a structured way to evaluate how vendors handle data and operational security, AIUC-1 is meant to give buyers a repeatable method for judging AI agents.

The startup says it created the standard with input from roughly 250 security and risk leaders — the people most likely to approve or reject an AI purchase inside large organizations. Dattani said the company regularly asks those stakeholders what they would need to see before trusting an external agent vendor, and what questions should be part of the review process.

That feedback is then translated into a formal test suite. AIUC runs an agent through around 5,000 scenarios designed to probe common failure modes, including jailbreak attempts, hallucinated outputs, and data leakage risks. The results are compiled into an approximately 100-page report that maps where the system is dependable and where it is weak.

The company says it uses AI agents to help conduct the tests and analyze the outputs, but that humans still review and verify the final audit. In other words, AI is helping to inspect AI, but not replacing human sign-off.

Who is behind AIUC?

AIUC was founded by two people with deep ties to the AI safety ecosystem: Kvist, who joined Anthropic early in its history, and Dattani, who served as COO of METR from 2024 to 2025 and remains on the group’s board.

That background matters because AIUC is not positioning itself as a generic compliance vendor. The company’s founders are part of the same safety-oriented network that has been warning for years that more powerful frontier systems may require stronger oversight than the market currently provides.

Dattani’s previous employer, METR, has done testing work for frontier AI labs, although its public focus has traditionally centered on performance questions — such as whether agents can complete tasks reliably — rather than broader enterprise certification. METR was also one of the independent research organizations involved in an OpenAI investigation into an issue involving Hugging Face.

AIUC’s founders are now trying to adapt that testing mindset to the commercial market. Instead of helping model developers understand whether an agent can work, they want to help customers decide whether an agent should be trusted.

Why the investors are paying attention

The $40 million Series A suggests that major investors believe AI safety evaluation could become a category of its own, not just an add-on to existing security services. Ribbit Capital led the round, with First Harmonic also participating.

The investor list is notable because it includes names associated with both AI infrastructure and AI safety. The earlier seed round featured support from Nat Friedman’s NFDG, Emergence, Terrain, and Anthropic co-founder Ben Mann. Taken together, the backers signal that sophisticated capital sees a business opportunity in standardized AI risk review.

That interest is tied to a practical market reality: enterprises are already using AI agents, but many are doing so cautiously and often in limited settings. A third-party certification could serve as a trust bridge between enthusiastic product teams and risk-conscious executives.

How does AIUC compare with the wider AI safety debate?

AIUC’s approach sits at the intersection of enterprise software procurement and frontier AI governance. The company is not trying to regulate model development directly, nor is it arguing that AI labs should stop building powerful systems. Instead, it is proposing a market mechanism that could shape adoption by making safety claims easier to verify.

That idea comes at a moment when questions about control are gaining urgency across the AI industry. Anthropic chief executive Dario Amodei has recently argued that frontier development should slow down, pointing to an increase in harmful or concerning system behavior. In that context, he has also floated the idea of embedded third-party evaluators at frontier labs, and specifically named METR as one possible participant.

AIUC is not adopting that embedded model. It is not proposing to install evaluators inside customer environments. But the logic is similar: independent assessment can help answer what a vendor can safely promise and what a buyer should treat as out of bounds.

According to Dattani, the goal is to give buyers a clearer picture of where an agent is trustworthy and where caution is still required. In his framing, the audit should show not just whether a system works, but where the customer should hesitate before deploying it.

That distinction is important. Traditional software procurement often relies on contracts, penetration tests, and security questionnaires. AI agents, however, can behave unpredictably in ways that are difficult to capture through static documentation alone. AIUC is trying to turn that uncertainty into a structured, repeatable evaluation.

What is the AIUC-1 standard trying to prove?

AIUC-1 is designed to answer a set of practical enterprise questions: Can the agent be manipulated? Does it expose sensitive data? Can it follow instructions consistently? Does it degrade when faced with adversarial inputs? And can a buyer rely on the vendor’s claims?

The company’s view is that enterprises do not need another generic score. They need a certification process tied to use cases and risk thresholds that matter in real deployments. That is why the startup says the standard was built by incorporating feedback from security and risk decision-makers rather than solely from engineers.

The resulting report is meant to be something like a due diligence dossier. A company evaluating an AI agent could review the findings, identify red flags, and decide whether the product is acceptable for a pilot, a limited rollout, or no rollout at all.

What kinds of failures are included?

The test suite looks for failures that are common in current AI systems and especially important for agents that can take actions:

• Jailbreak vulnerability: whether the agent can be tricked into ignoring its rules or guardrails.
• Hallucination risk: whether it produces false or fabricated information in operational contexts.
• Data leakage: whether it reveals sensitive or restricted information.
• Reliability issues: whether it behaves consistently across repeated trials and edge cases.

Those categories matter because even a small percentage of failures can become serious when an agent is given access to tools, databases, or business workflows. A model that is merely inaccurate in a chat interface is one thing; a model that misfiles records, mishandles customer data, or takes a harmful action is another.

What does AIUC’s rise say about the market?

AIUC’s funding round points to a growing belief that AI safety can be productized. As more companies move from experimentation to deployment, there is likely to be demand for independent assurance — especially from regulated industries and large enterprises that cannot afford to rely on vendor assurances alone.

That is also why the company’s pitch is not limited to model creators. AIUC says it wants to serve both companies building AI systems and enterprises buying them. In practice, that could make it useful as a middle layer between innovation and governance.

If the market embraces that model, AIUC could help establish a new norm: before buying an autonomous AI system, enterprise customers might ask for a certification report the same way they ask for security attestations today.

Whether that becomes common will depend on several factors, including whether buyers trust the framework, whether vendors are willing to submit to the process, and whether the standards keep pace with rapidly changing agent capabilities.

Timeline of AIUC’s development

The company’s progress has been rapid, moving from seed backing to a large Series A as the AI agent market has accelerated.

What happens next?

AIUC now has the funding to try to make its certification model credible enough to influence enterprise procurement. The challenge will be turning a promising safety concept into a standard that vendors, buyers, and auditors actually use.

If it succeeds, AIUC could become one of the first companies to define what trustworthy AI-agent deployment looks like in a commercial setting. If it fails, the startup may join a long list of safety ideas that proved technically interesting but difficult to turn into market infrastructure.

For now, the broader significance is clear: as AI agents move deeper into businesses, the market is looking for a way to answer a simple but consequential question before deployment — can this system be trusted to do only what it is supposed to do?

AIUC is betting that the answer will increasingly depend on independent certification, not just vendor promises.