China’s AI Leaders Are Worried About the Same Threats as the US

BEIJING — At first glance, the mood inside a major artificial intelligence conference in Zhongguancun last week suggested the industry was still in its expansion phase: full auditoriums, polished demos, and sessions stretching from self-modifying models to humanoid robots. Yet the most striking message emerging from the event was not one of triumph, but of unease. In conversations with researchers and company executives in Beijing, a clear consensus surfaced: the world’s two leading AI powers may need to cooperate on safety before the technology outpaces both of them.

The conference, hosted in China’s high-tech capital, brought together academics, industry figures, and prominent names from the history of computing. Among the attendees were Whitfield Diffie, the cryptography pioneer, and Andrew Barto, whose reinforcement learning work helped shape modern AI. Their presence underscored the global significance of the moment. So did the subject matter. Sessions covered not only model scaling and robotics, but also the risks posed by agentic systems, cyber-enabled abuse, and models that may eventually improve themselves with limited human oversight.

What stood out most was that concerns about AI safety are no longer limited to Washington, Silicon Valley, or a handful of Western policy circles. Chinese researchers and executives are openly wrestling with the same questions: how to keep advanced systems from becoming tools for cyberattacks, how to stop open models from being repurposed for harm, and how to prevent a fast-moving race for capability from creating a broader systems failure.

The stakes are rising quickly. Governments in the United States and China have treated each other’s AI progress as a strategic threat, with Washington leaning heavily on export controls, chip restrictions, and security reviews to slow China’s access to frontier compute. But as models become more agentic — able to plan, use tools, and act across software environments — the dangers no longer fit neatly into a zero-sum geopolitical frame. If advanced AI systems are deployed carelessly, the costs could spill across borders in the form of automated cyberattacks, infrastructure disruption, or severe model misuse.

Inside the Beijing conference: capability was everywhere, but so was caution

The event’s agenda reflected how far the field has moved beyond simple chatbots. Panels touched on recursive self-improvement, a concept that imagines systems rewriting and improving their own code in repeated cycles. Others focused on humanoid robotics and on security challenges arising from AI-generated software. The breadth of topics gave the conference an almost double meaning: it was a celebration of technical ambition and a warning about what that ambition may unleash.

That duality appeared in a remark from MIT computer scientist Stephen Casper, who spoke remotely and later expanded on his views. He argued that AI should be understood as a global technology whose benefits and harms will also be global, and that the dissemination of advanced capabilities is likely unavoidable.

Casper said AI has global upsides and downsides, and that new capabilities tend to spread regardless of where they are first developed.

His message echoed a familiar historical comparison: the way the United States and the Soviet Union eventually had to collaborate on nuclear safety, even as they remained adversaries. Casper’s point was not that the two AI powers should trust one another on everything, but that some risks are too large to manage through competition alone.

In his view, the industry needs to avoid a catastrophic public failure that could resemble a nuclear accident — a “Chernobyl moment” for AI. The phrase captures an anxiety increasingly shared by researchers: that a high-profile failure involving frontier systems could trigger public backlash, regulatory overreaction, or worse, genuine harm on a global scale.

The national security tension is real — but it may not be the whole story

For years, American policy toward China’s AI sector has been shaped by two overlapping concerns: economic competition and national security. U.S. officials have tightened access to advanced chips, restricted semiconductor equipment, and scrutinized cross-border technology flows in an effort to slow China’s progress in training the most powerful systems.

That approach has expanded from hardware controls into model access concerns. Most recently, U.S. authorities ordered Anthropic to block foreign nationals from using its most capable systems, Mythos and Fable 5, citing national security concerns. Anthropic then broadened the restriction, cutting off access for everyone. Among the entities that raised concern, according to earlier reporting, was a South Korean telecom company alleged to have ties to China.

The move illustrates how frontier AI is now being governed not just as software, but as strategic infrastructure. The logic is straightforward: if the most powerful models can reason, code, plan, and act with increasing autonomy, then access to them may resemble access to dual-use technology. But the Beijing conference suggested that a purely adversarial framing misses an equally serious issue. If both countries continue accelerating without coordinated safety work, both may end up more vulnerable.

That idea may sound idealistic in a climate of export controls, industrial policy, and geopolitical suspicion. Yet the researchers in Beijing did not present cooperation as charity. They framed it as self-preservation.

Why AI risk is becoming a shared problem

The newest frontier systems are no longer just text generators. They can invoke external tools, browse software environments, draft code, manipulate workflows, and chain multiple steps toward a goal. Those capabilities are a major reason companies want them. They are also the source of new risk.

Advanced models may introduce new software vulnerabilities by generating flawed code at scale. They may help attackers find weaknesses faster than defenders can patch them. And with agentic tool use, a malicious actor may be able to automate social engineering, reconnaissance, or other forms of cyber abuse in ways that are far more effective than yesterday’s spam or phishing kits.

One full-day session at the conference focused squarely on these issues. Researchers discussed the security risks of code generation, new attack surfaces created by tool-using agents, and the way automated systems may lower the cost of deception. The conversation was not abstract. It reflected a broader shift in the AI field, where cybersecurity is no longer a peripheral concern but one of the central arguments for and against rapid deployment.

Lin Yun, a professor at Shanghai Jiao Tong University who works at the intersection of AI and computer security, told me the short-term balance may still favor attackers. That is a sobering assessment, but not a fatal one. He argued that defensive techniques will improve, and that AI itself may eventually become part of the solution.

Lin said countries that see the risks similarly will find it easier to agree on safety principles and technical standards, even if they continue competing in other areas.

His view reflects a practical approach that many security experts now share: the goal is not to erase geopolitical competition, but to create narrow lanes for cooperation where doing so reduces the chance of systemic harm.

The open-weight model debate is now a security debate

One of the sharpest tensions in AI policy today is the push and pull between openness and control. Open-weight models, whose parameters are publicly available, have become important engines for research, customization, and startup experimentation. They let developers inspect, modify, and deploy systems without depending entirely on a closed platform provider.

That openness has also made Chinese models influential beyond China. In the United States, developers increasingly use open models from firms such as Moonshot, Alibaba, and Z.ai, because they are competitive, flexible, and often inexpensive to run. The U.S. has responded with its own renewed push toward open-weight development, including Nvidia’s Nemotron family.

But the same qualities that make open-weight models attractive can also make them risky. If a model is released without sufficient guardrails, attackers may be able to use it to identify vulnerabilities, write malicious code, or refine harmful workflows. As the systems improve, the line between a useful open model and a dangerous one may become much thinner.

That concern is no longer theoretical. Analysts reviewing Z.ai’s latest model, GLM 5.2, say it includes frontier-level coding and agentic abilities. In parallel, 360 Security Technologies, a major Chinese cybersecurity company, said this week that it had built an AI model with hacking capabilities comparable to Mythos, the frontier system referenced in recent U.S. restrictions.

The larger implication is that even models considered less advanced today may cross into dangerous territory if they lose their constraints. That means security review, provenance checks, and update mechanisms are becoming as important as raw benchmark performance.

Why open models still matter

Despite the risks, researchers and developers are unlikely to abandon open-weight models. They support reproducibility, local deployment, customization, and academic scrutiny. For many teams, they are also the only practical way to experiment with frontier-like systems without depending on a cloud provider’s rules or pricing.

• They allow researchers to study model behavior directly.
• They help startups build specialized applications quickly.
• They reduce dependence on a few closed AI platforms.
• They can accelerate innovation in countries or sectors with limited access to top-tier proprietary models.

The problem, experts say, is that the benefits of openness also scale the misuse risk. A model that can be downloaded, modified, and redistributed can spread useful capabilities quickly — but it can also spread harmful ones just as fast.

The emerging security challenge for open-source AI

Lin argued that the industry will need new technical and governance mechanisms to make open models trustworthy. That includes ways to ensure that models are current, have not been tampered with, and are free from hidden malicious behavior or overlooked vulnerabilities.

In practical terms, that may require stronger release auditing, signed model artifacts, standardized safety evaluations, and better methods for tracking whether a model has been modified after publication. It may also mean distinguishing between open research releases and models that are too capable to be safely distributed in unrestrained form.

A source at a major Chinese AI company told me, on condition of anonymity because the person was not authorized to speak publicly, that security worries are already shaping release decisions. In some cases, advanced models are reportedly no longer being shared as open source for precisely that reason.

If that trend continues, the industry may enter a new phase in which full openness is seen less as an ideological commitment and more as a risk management decision.

What the US and China might actually do together

Calls for cooperation between geopolitical rivals often sound vague until they are tied to specific problems. In this case, the possible areas of collaboration are not hard to identify. They include cybersecurity standards, red-team evaluations, incident reporting protocols, and shared terminology for classifying model risk.

Researchers also point to a narrower and perhaps more realistic objective: creating shared safety principles without exposing sensitive operational details. That distinction matters. The goal is not necessarily to exchange proprietary model weights or reveal every security weakness in a system. It is to develop common frameworks for identifying dangerous behaviors and preventing catastrophic failure.

Casper pointed to research suggesting that the upside of cross-border collaboration on AI safety may exceed the national security downside. The logic is that some information about risk can be shared without undermining strategic interests, and that refusing all cooperation could leave both sides more exposed to the same threats.

That framing is especially relevant as AI systems become more deeply embedded in critical infrastructure, software development, finance, logistics, and everyday communication. When models begin to mediate more of the digital economy, a security failure in one country is less likely to stay local. A vulnerability discovered in one widely used model can spread across organizations and borders in days.

A timeline of the key developments

Why the geopolitical frame may be too narrow

It is tempting to view all of this through a Cold War lens: the United States and China are locked in a competition for technological supremacy, and AI is simply the latest arena. There is truth in that. Compute supply chains, chip access, talent, and model leadership are all deeply strategic.

But the conference in Beijing suggested that this framing is incomplete. AI safety is not merely a side effect of competition; it is one of the few areas where mutual vulnerability is obvious. If a malicious model can be used to automate large-scale cybercrime, both sides have a stake in limiting its spread. If an autonomous system can generate harmful software or manipulate information environments, both sides face downstream consequences.

That does not mean cooperation will be easy. Trust is low. Information is sensitive. Policy goals diverge. And there is always the fear that safety collaboration could be used as cover for strategic intelligence gathering. Still, the alternative — an uncontrolled race in which each side assumes the other will absorb the costs — is increasingly hard to justify.

Three reasons cooperation may be more likely than it looks

1. Shared exposure: Cyber risk does not respect borders, especially when models are distributed globally.
2. Common technical language: Researchers in both countries increasingly use similar methods to evaluate model behavior and safety.
3. Economic interdependence: Even rival AI ecosystems depend on global supply chains, standards, and research communities.

What comes next for frontier AI safety

The next phase of AI development may be defined less by bigger benchmarks and more by governance questions. Can developers prove that a model is safe enough to release? Can open releases be audited the way critical software is audited? Can agentic systems be constrained before they gain too much autonomy? And can rival powers agree on minimum standards before the technology becomes too embedded to regulate effectively?

Those questions are becoming harder to defer because the systems themselves are changing. When models can plan multi-step actions, write and debug code, and interact with external tools, the gap between a helpful assistant and a harmful autonomous actor narrows. The same technical leap that makes AI more useful also makes it more dangerous.

The Beijing conference did not provide a final answer. What it did provide was evidence that concern about AI’s trajectory is no longer confined to one side of the Pacific. China’s leading researchers and AI companies are watching the same danger signs as their counterparts in the United States. That creates a rare point of overlap in an otherwise tense rivalry.

If there is a lesson from the week’s discussions, it is that the next great contest in AI may not be just about who builds the best model. It may be about who helps prevent the worst one from being used in the wild.

Key facts at a glance

For now, the AI race between the US and China continues. But the message from Beijing was hard to miss: when the technology gets powerful enough, rivalry alone may not be a workable safety strategy.

superintelligencenews@gmail.com