Trillium Labs Bets Open AI Research Can Make Frontier Models Safer

Two former industry scientists are launching Trillium Labs, a nonprofit that plans to study risky AI capabilities in public rather than behind closed doors. The new lab, announced on October 2, 2026, will examine areas including recursive self-improvement, AI agents, and reinforcement learning, with the goal of making frontier model research more transparent and easier for outside scientists to verify.

The move pushes directly against a dominant strategy in Silicon Valley, where the most capable models are often kept inside tightly controlled systems and exposed only through apps or APIs. Trillium Labs argues that openness, replication, and public scrutiny are the better path for understanding how advanced systems behave and for reducing the dangers they may create.

What Trillium Labs is trying to change

Trillium Labs wants to challenge the idea that the safest way to handle powerful AI is to keep it hidden from everyone except a small group of company employees. The nonprofit is being built around the belief that researchers outside big labs should be able to inspect methods, repeat experiments, and test claims about how frontier models are trained and tuned.

That philosophy matters because modern AI systems are no longer just chatbots or text generators. The latest models can automate software discovery, help probe weaknesses in digital systems, and act with increasing independence when deployed as agents. Those abilities have intensified arguments over whether secrecy protects society or simply slows the research community’s ability to understand emerging risks.

Founders Nathan Lambert and Tom Zick say they want to bring more of the scientific method back into frontier AI work. In their view, public experimentation and detailed reporting can surface hidden failure modes sooner than proprietary development does.

Lambert argues that the field has drifted away from the openness that made science effective in the first place, saying that closed development at the frontier is moving the industry in the wrong direction.

Why open research matters in frontier AI

Open research matters because the newest AI models are powerful enough to create real-world risks, but their behavior is still poorly understood outside a handful of companies. When only a few institutions can run the biggest systems, the wider research community has less chance to test assumptions, uncover weaknesses, or challenge claims about safety.

Trillium Labs is staking its case on a simple idea: if risky systems can be studied more like traditional science projects, then their behavior can be measured, debated, and improved in public. That does not remove danger, but it may make dangerous systems easier to evaluate before they are widely deployed.

The dispute is not just academic. Frontier models can already be used to automate vulnerability discovery, assist with hacking techniques, and produce behavior that surprises even their creators. As those capabilities improve, pressure has grown for a research model that is less secretive and more accountable.

How the industry is split

How AI should be developed is now one of the field’s central fights. Some companies argue that restricting access to the most powerful systems keeps dangerous capabilities in the hands of a trusted few while researchers work on safeguards. Others believe that limited access prevents the broader scientific community from spotting problems early enough.

Trillium Labs falls squarely into the second camp. Its founders say more eyes on the work will generate better ideas, more credible testing, and a richer understanding of how model capabilities scale.

Who founded Trillium Labs?

Trillium Labs was founded by Nathan Lambert and Tom Zick, both of whom have spent time in AI research, policy, and technical work around responsible deployment. Lambert previously worked at Ai2, which has been known for publishing model details, datasets, and training methods alongside its systems. He also spent time at Hugging Face, writes a widely read technical blog, and founded American Truly Open Models, an effort encouraging more open AI releases in the United States.

Zick’s background includes work at Harvard University and policy-related efforts at Charles Schwab focused on responsible AI. The pair met on Zoom during the pandemic while both were graduate students at UC Berkeley. Their collaboration was shaped by a shared frustration that much of the most important AI work inside large companies cannot be replicated by academics who lack access to similar resources.

That replication gap is central to their new nonprofit’s mission. Lambert and Zick say the field has reached a point where research papers, blog posts, and marketing claims often reveal too little about how major models are actually built and tuned. Trillium Labs wants to narrow that gap by publishing experiment details, not just outcomes.

Zick says the nonprofit was created in part because modern frontier research has drifted so far from universities that many professors and students cannot realistically reproduce what big labs are doing.

What problems will the lab study first?

The lab’s first research agenda will focus on post-training, recursive self-improvement, and reinforcement learning. Those are not the most visible parts of AI development, but they are some of the most consequential for how models behave after the initial training run is finished.

Post-training refers to the phase where a model is further refined after the base system exists. That can include fine-tuning, behavioral shaping, and other methods that strongly influence how useful, compliant, or unpredictable a system becomes.

Recursive self-improvement, or RSI, is even more provocative. It describes the possibility of AI systems contributing to the creation of newer, better AI systems, potentially creating a feedback loop in which models help improve themselves. Some researchers see that as a route to rapid progress; others view it as a path toward control problems that humans may not be able to manage well.

Reinforcement learning is another major focus. It rewards systems for desirable outputs and penalizes them for poor ones, helping them become more capable but also sometimes making them more likely to develop odd, overly agreeable, or unexpected behaviors. Trillium Labs wants to study not just whether reinforcement learning improves performance, but how it shapes a model’s character.

Why reinforcement learning is under scrutiny

Why is reinforcement learning drawing so much attention now? Because it has become a key ingredient in making AI agents more capable, while also making them harder to predict. The method can unlock better task completion and stronger decision-making, but it can also create unwanted side effects such as excessive sycophancy or strategic behavior that is difficult to detect during casual testing.

Zick says that understanding the scaling behavior of reinforcement learning requires serious computational resources and careful experimental design. Trillium Labs believes that if those experiments are documented publicly, outside scientists can spot patterns and raise questions that internal teams might overlook.

The stakes behind recursive self-improvement

The stakes behind recursive self-improvement are unusually high because the concept touches one of the field’s deepest fears: that AI progress could accelerate beyond human control. If models start helping design better models in a sustained loop, some researchers worry that the pace of improvement could outstrip human oversight.

That concern has moved from abstract speculation into mainstream AI debate. Earlier this month, an Anthropic researcher left the company and publicly warned that RSI could create an existential threat to humanity. Trillium Labs is not claiming that outcome is inevitable, but it is signaling that the topic deserves direct, open investigation rather than quiet speculation inside closed research teams.

In practical terms, that means the nonprofit wants to treat these questions as testable research problems. Instead of arguing over ideas in theory alone, the lab intends to run experiments, publish methodology, and invite outside review.

How does Trillium Labs differ from OpenAI and Anthropic?

How Trillium Labs differs from OpenAI and Anthropic is straightforward: the nonprofit wants to publish more of its methods, while the leading frontier labs generally keep most of theirs private. The biggest AI companies usually expose their systems through applications or APIs, which lets users interact with the models without seeing the training process, internal architecture, or tuning details.

That closed model gives companies more control over deployment, abuse prevention, and commercial strategy. But it also limits how much independent researchers can verify about model behavior, reproducibility, and safety claims.

By contrast, Trillium Labs wants its work to be inspectable. That does not mean everything will be unrestricted or low risk, but it does mean the lab is built around a more open norm than the one followed by most frontier AI firms.

What open AI looks like in practice

Open AI research is not a single thing. It can include publicly releasing model weights, publishing training data details, sharing experimental logs, or documenting post-training methods closely enough that another team can reproduce the work. Trillium Labs is expected to lean heavily on the latter forms of openness, especially in areas where direct model release might create unnecessary risk.

The broader ecosystem already contains examples of this approach. Some Chinese companies have published more transparent information about major training runs, and researchers at Stanford have been training the Marin model in the open. Ai2 has also built a reputation for releasing unusually detailed information about its model development process.

Those projects do not settle the debate. But they show that a spectrum exists between total secrecy and total openness, and that useful research can be conducted somewhere in the middle.

Why investors are backing the project

Trillium Labs has already raised an undisclosed amount from Schmidt Sciences, Halcyon Futures, and other supporters. The founders say they ultimately hope to raise between $40 million and $100 million, and they expect to spend about $30 million on training over the next 18 months.

That is a meaningful war chest for a nonprofit, particularly one focused on compute-intensive research. The budget suggests the founders believe there is room for a well-funded public-interest lab to do frontier-level experimentation rather than simply commentary or policy analysis.

For backers, the bet appears to be that more transparent work could shape the norms of a field still deciding how open it wants to be. If the lab can produce high-quality results and show its methods, it could influence both policy debates and scientific practice.

Tim Fist, who leads emerging technology policy at the Institute for Progress, said he strongly supports greater transparency in research and development.

Key facts about Trillium Labs

The following table summarizes the main facts surrounding the launch and the lab’s early plans.

Timeline of the launch

The lab’s emergence reflects several years of debate over openness, safety, and control in AI development.

What happens next?

What happens next will depend on whether Trillium Labs can deliver rigorous research while staying open enough to attract attention from academics, policymakers, and other labs. The group is trying to prove that transparency is not a weakness but a strength, especially in an era when AI systems are becoming more autonomous and more difficult to audit.

Its success will also depend on whether the broader field is willing to treat openness as more than a branding exercise. If the lab’s work produces replicable findings about post-training, reinforcement learning, or RSI, it could give critics of closed-door development more evidence to build on.

For now, Trillium Labs is entering a crowded and contentious debate with an unusually clear message: understanding frontier AI may require less secrecy, not more. Whether the industry agrees may shape how the next generation of powerful models is built, studied, and governed.