Circuit Breaker Labs is building AI crash tests to spot dangerous chatbots before they hurt people

Circuit Breaker Labs, a 2026 Startup Battlefield finalist, is building AI safety tests designed to catch psychologically dangerous chatbot behavior before it reaches users. The San Francisco startup says its system can stress-test models for harmful interactions across ages, languages, slang and cultures, a mission that matters as lawsuits and tragedies have put the risks of companion-style AI under a harsh spotlight.

The company will pitch at TechCrunch Disrupt in San Francisco from October 13 to 15, arriving at a moment when the industry is facing intense scrutiny over how chatbots respond to vulnerable people, especially teenagers and users seeking emotional support.

What sets Circuit Breaker Labs apart is its focus on the messy reality of human conversation. Instead of evaluating AI with neat, standardized prompts, the company simulates how people actually talk, including slang, typos, code words and emotionally loaded messages. Its founders say that approach is necessary because a model that seems safe in a lab can behave very differently when exposed to ambiguous, vulnerable or prolonged real-world conversations.

Why this startup exists now

Circuit Breaker Labs was shaped by one of the most painful warning signs in consumer AI: the death of 14-year-old Sewell Setzer, whose family said he formed a deep emotional attachment to a Character.AI chatbot before dying by suicide. The company’s founders, siblings Shirali Nigam and Arul Nigam, point to that case as a reminder that chatbot failures are not abstract technical bugs; they can become life-or-death failures when a system misunderstands a user’s intent or emotional state.

That concern has only grown as other families have sued OpenAI, alleging that ChatGPT contributed to suicides and delusional thinking in their loved ones. Meanwhile, Character.AI recently settled multiple wrongful death lawsuits brought by families of underage users. Together, those cases have transformed AI safety from a niche engineering concern into a mainstream legal, ethical and reputational risk for the entire industry.

The company’s mission is to make AI safer across languages and cultures, not just in English-speaking, technically literate environments. The founders say models are often trained and tested on polished language patterns, but real users speak in fragments, with slang, sarcasm, emotion and cultural references that can alter meaning dramatically.

Arul Nigam, the company’s CTO, said the team is focused on preventing safety failures that happen even when users are not trying to break the system, but are instead interacting naturally and expectingly receiving support. He said those failures can emerge when models lose context, miss nuance, or make dangerously wrong assumptions about what a person means.

How does Circuit Breaker Labs test AI safety?

Circuit Breaker Labs uses simulated users to probe whether AI systems can recognize and respond appropriately to risky behavior. The company describes these simulations as an army of crash-test dummies for AI: artificial personas that mimic people from different age groups, regions, cultural backgrounds and language communities.

These agents are built to imitate realistic speech, including informal phrasing, coded references and mistakes that are common in everyday conversation. The goal is not to trick models with adversarial jargon alone, but to see how they behave in sustained, natural exchanges where danger may unfold gradually rather than in a single obvious prompt.

Shirali Nigam, the company’s CEO, said common demographic and linguistic differences can expose hidden weaknesses. A phrase that sounds harmless to one user might read differently to another, and a model that performs well with standardized English may stumble when confronted with teenage slang, second-language phrasing or highly contextual emotional language.

She said the startup’s testing framework reflects the reality that people do not talk like carefully edited transcripts. In her view, if a model cannot handle that messiness, it may misread vulnerable users and respond in ways that could make a bad situation worse.

What is red-teaming in this context?

In AI safety, red-teaming means deliberately attacking or probing a system to uncover weaknesses before the public encounters them. Circuit Breaker Labs says it runs these tests at scale, using tens of thousands to hundreds of thousands of simulated conversations each day to measure where a model breaks down.

The startup says it works with human domain experts to construct realistic scenarios and then applies a proprietary scoring method that produces auditable, explainable results. In practical terms, that means companies can see not only that a model failed, but also why it failed and in what kind of conversation the failure occurred.

That explainability could matter for buyers operating in high-stakes settings. A mental health app, a journaling product or an AI coaching service may need more than a simple pass-fail rating; it may need evidence that a model can handle difficult conversations consistently and safely over time.

What kinds of AI products is it testing?

At the moment, Circuit Breaker Labs is focused on applications where chatbot behavior can have serious psychological consequences. That includes AI coaching tools, journaling apps and other mental health-adjacent products, though the company has not disclosed the names of its marquee customers.

Arul Nigam said the startup already has a working product, but it remains very small, with only five employees including the two founders. That early-stage footprint suggests Circuit Breaker Labs is still proving out its business model even as demand for AI safety tools rises sharply.

Still, the founders believe the same testing framework could eventually be applied far beyond therapy or wellness apps. They point to AI “co-worker” agents and similar systems that may develop persistent, quasi-social relationships with users over time. In those settings, a model’s responses can vary from one interaction to the next, making safety harder to guarantee through one-off testing alone.

Could the platform help prevent AI psychosis concerns?

Yes, that is one of the broader problems Circuit Breaker Labs says it wants to address. The company is targeting the kind of spiral sometimes described as “AI psychosis,” in which users become absorbed in a parasocial bond with a chatbot or begin to rely on it in destabilizing ways.

That issue is increasingly important as more people turn to AI systems for companionship, advice and emotional reassurance. If a chatbot validates delusions, encourages dependency or fails to interrupt self-harm ideation, the harm can be severe even if the model is operating “as designed.”

The Nigams argue that safety concerns should not automatically lead to abandoning useful AI tools. Arul Nigam said skepticism is healthy, but he warned that banning technology outright because it can be misused would be a backward step. Their view is that better testing, not fear, should be the answer.

He said the company’s goal is to strengthen public trust in AI by helping developers identify dangerous behavior before release, rather than after a crisis has already occurred.

Why language and culture matter so much in AI safety

Language is where many AI safety failures begin. A chatbot may recognize a phrase literally but miss the emotional signal behind it, especially if the speaker is young, distressed or using slang that the system has not been tuned to interpret accurately.

That problem becomes more complicated across cultures. A phrase that signals affection, irony or escalation in one community may mean something else in another. A model that lacks cultural context may overreact, underreact or respond in a way that unintentionally deepens a user’s distress.

Circuit Breaker Labs is betting that this challenge can be turned into a market. As AI companies push into consumer apps, wellness tools and workplace agents, they need more than general assurances that a model is “safe.” They need testing systems that can demonstrate how those tools behave under realistic pressure.

That need is particularly strong for products that invite people to share personal details. The more a user trusts a chatbot, the more damaging a bad response can become. In that sense, safety is not just a technical feature; it is part of the product promise.

How big is the opportunity for AI safety testing?

The opportunity appears likely to grow as regulation, litigation and consumer expectations all move in the same direction. AI developers are under pressure to show that their systems can handle vulnerable users responsibly, especially as parents, educators and policymakers question whether chatbots are being deployed too quickly.

That pressure creates space for specialized vendors. A company like Circuit Breaker Labs can serve as an outside validator, helping AI builders identify dangerous behaviors before users do. If its methods prove reliable, it could become part of the standard development stack for products that interact with people in emotionally sensitive ways.

The market may also widen as enterprises adopt more agentic AI systems. The more autonomous and conversational those systems become, the more likely they are to encounter ambiguous situations that traditional software testing cannot fully anticipate.

Below is a snapshot of the startup, its product focus and the broader timeline shaping the conversation around AI safety.

Timeline of the AI safety debate

The company’s pitch arrives after several developments that pushed chatbot safety into the spotlight.

• 2024: The family of Sewell Setzer filed a lawsuit alleging that a Character.AI chatbot played a harmful role in the teenager’s death by suicide.
• 2025: Multiple families sued OpenAI, saying ChatGPT allegedly contributed to suicides and delusional beliefs among their loved ones.
• Earlier in 2026: Character.AI settled wrongful death lawsuits brought by families of underage users.
• October 13-15, 2026: Circuit Breaker Labs is scheduled to pitch at TechCrunch Disrupt in San Francisco.

What makes the company different from other AI safety tools?

The most important difference is its emphasis on psychologically realistic simulation. Many safety tools focus on policy violations, jailbreaks or content moderation. Circuit Breaker Labs is trying to detect something more subtle: whether a model can safely handle emotionally fraught, cumulative conversations with vulnerable people.

That approach reflects a shift in the industry. As AI systems become more conversational and more personal, the highest-risk failures may not look like dramatic hacks. They may look like a chatbot that says the wrong thing at the wrong time, or keeps a user engaged when it should be encouraging outside help.

By testing for those edge cases at scale, the startup hopes to give AI companies a way to measure trustworthiness before deployment. Its founders believe that if models can be made safer in the most sensitive contexts, the broader public will become more willing to adopt them elsewhere too.

What comes next for Circuit Breaker Labs?

For now, the startup is still in the early innings. With only five employees and undisclosed customers, it is far from a household name. But its debut on the Startup Battlefield stage gives it a prominent platform at a moment when investors, regulators and users are all asking the same question: how can AI become useful without becoming dangerous?

The Nigams think the answer lies in better simulation, better scoring and a more honest recognition that real users are messy, emotional and culturally varied. If their tools can help developers catch harmful behavior before release, the company could become an important layer in the AI safety stack.

And if the industry takes their message seriously, Circuit Breaker Labs may not just be another startup competing for attention at Disrupt. It could become part of the standard infrastructure for building AI systems that people, including children and vulnerable users, can trust.

For AI companies racing to launch new products, that may be the most valuable promise of all.

FAQ

What does Circuit Breaker Labs do?

It builds AI safety tests that simulate real people and real conversations to find harmful chatbot behavior before users encounter it. The startup focuses on psychologically risky interactions, including situations involving vulnerable users, slang, cultural nuance and emotionally sensitive topics.

Why is Circuit Breaker Labs getting attention now?

It is gaining attention because AI chatbot safety is under intense scrutiny after several wrongful death lawsuits and public concern about emotional dependency on bots. The startup is also a finalist in TechCrunch’s 2026 Startup Battlefield and will pitch at Disrupt in San Francisco.

How does its testing work?

The company uses simulated AI agents as crash-test dummies for models. Those agents imitate different ages, cultures, languages and speech styles, then run large-scale red-team tests to see whether a chatbot can respond safely and consistently in realistic conditions.

Who founded Circuit Breaker Labs?

It was founded by siblings Shirali Nigam and Arul Nigam. Shirali is the CEO, while Arul serves as CTO. They say the company was motivated in part by the harmful consequences of chatbot interactions for young users.

What kinds of AI products could it test in the future?

It could eventually test any conversational AI system that risks creating unhealthy dependence or emotional harm. That includes mental health apps, journaling tools, AI coaches, workplace agents and other systems where repeated interaction can shape user behavior over time.