OpenAI’s COBOL push shows AI can translate code — but not yet legacy risk

OpenAI says its newest models can do a much better job of migrating COBOL and other legacy software into modern code, but experts say that still does not amount to safely replacing critical systems. The dispute matters because COBOL still underpins banking, government and other infrastructure where a bad migration can cause real-world failures.

What OpenAI is claiming

On Monday, OpenAI said its latest models delivered substantial gains in moving older-language software into modern code. The company framed the work as a way to help organizations maintain and modernize aging systems, especially those built on COBOL, the business language created in 1959 that still sits inside many high-stakes environments.

That is a meaningful technical claim, but it is not the same as proving that AI can safely replace a production system. OpenAI said successful migration requires all tests to pass, and the company’s own benchmark numbers show why that condition matters so much.

How strong were the benchmark results?

According to The Verge, GPT-6.1 Sol reached 87% behavior-preservation accuracy at low reasoning and 98% at higher reasoning. The Verge also reported that GPT-6 Astra scored about 86% at low reasoning in the same benchmark, while GPT-6 Sol scored 21% and GPT-5.6 Sol scored 5% at low reasoning.

Those numbers suggest rapid improvement, but they also show how sensitive the results are to model choice and reasoning settings. In a setting where the goal is to preserve behavior exactly, even a small miss can be consequential.

Why it matters

This is not a niche software story. COBOL remains embedded in systems that support online banking, credit card processing, ATM networks and government operations, according to The Verge, which cited IBM figures showing more than 40% of online banking systems, 80% of in-person credit card transactions and 95% of ATM transactions are supported by COBOL. The same report said IBM has estimated COBOL inefficiencies cost the U.S. economy about $105 billion a year.

That scale explains why the market cares. If AI can reliably reduce the cost and risk of maintaining old software, banks, agencies and vendors could save time and money. If it cannot, the same tools may simply generate a new layer of code that still depends on old assumptions, hidden dependencies and fragile business logic.

Yahoo Finance described that risk as a “translation trap” rather than a true modernization path. The outlet’s framing is important: translating code is not the same thing as replacing a system, and conversion tools can preserve old technical debt inside freshly written syntax.

What is the real business problem?

The real problem is not COBOL text alone. It is the web of rules, workflows, exceptions and undocumented behavior that sits around that code. Yahoo Finance said one major barrier is the loss of institutional knowledge as veteran staff retire and documentation disappears, leaving companies with systems they cannot fully explain, even before they try to change them.

That helps explain why banks are not just asking for a code generator. Yahoo Finance said they want orchestration of systems, not simple code ownership or a rewrite for rewrite’s sake. In other words, they need help understanding how all the pieces fit together before anyone rewires them.

Background: why COBOL is still here

COBOL first appeared in 1959, and it has lasted because it was built for business data processing at a time when many organizations were laying the foundations of their core systems. Those systems did not disappear. They were patched, expanded and routed into newer layers of software over decades, which is why COBOL still survives in banking, government and other critical infrastructure.

That longevity has a cost. Legacy systems are difficult to rewrite because the original documentation is often incomplete, the people who built them may no longer be available, and the software’s behavior may be more important than its source code. In that environment, modernization usually means preserving behavior exactly, not merely translating syntax from one language to another.

Experts quoted by The Verge said the current state of AI does not solve that problem. Vadim Zaytsev, associate professor at the University of Twente, captured the core distinction in a line that gets to the heart of the issue:

“Rewriting code is not the same as replacing a system.”

That difference matters because a successful migration is only successful if the new version behaves like the old one in every important case, including the weird and rarely tested ones that keep critical operations running.

How does OpenAI’s claim compare with Anthropic’s pitch?

OpenAI is not the only company trying to position AI as a modernization tool. Earlier, Anthropic pitched Claude Code and a modernization playbook aimed at migrating COBOL to Java or Python. But according to The Verge, Anthropic did not provide performance benchmarks for those migrations.

Anthropic said the economics of COBOL modernization had shifted, a claim that suggests the business case for replacing old code may now be more attractive than before. Still, without detailed benchmark data, the company’s pitch remains a broad argument for modernization rather than a measured demonstration of safe system replacement.

The Verge also reported that IBM shares fell after Anthropic’s announcement, underscoring how closely investors watch any suggestion that legacy systems may be rewritten or displaced. That reaction illustrates a broader market sensitivity: modernization tools can be read not only as productivity software but as signals about future demand for enterprise platforms and services.

OpenAI’s new disclosure raises the competitive stakes. If one model family can preserve behavior more reliably than another, vendors will try to turn that into proof that their systems are better suited for enterprise migration work. But the benchmark gap between high and low reasoning also suggests that success may depend as much on workflow and oversight as on the raw model itself.

What the numbers actually say

The available figures point to both promise and caution. The strongest result in the fact sheet is GPT-6.1 Sol’s 98% behavior-preservation score at higher reasoning. The weakest is GPT-5.6 Sol’s 5% score at low reasoning. That spread is too wide to treat all AI-assisted migration as a single category of capability.

OpenAI also reported improved model performance on building database software in several languages, including COBOL. That broadens the story beyond language translation alone and suggests the company sees legacy modernization as a broader engineering workflow, not just a code-conversion task.

What experts are warning about

The strongest skepticism in this story is not that AI cannot write code. It is that code generation alone does not guarantee operational safety. The concern is especially acute in systems where a hidden rule, a missed edge case or a misunderstood dependency can interrupt payments, processing or government services.

Yahoo Finance argued that AI should be used more as a systems-historical aid than as a pure coding tool. That is a useful way to think about the problem: the machine may help surface old assumptions, map dependencies and explain what a legacy system does, but a human-led process still has to decide what should be changed and how far.

Claudio González, Global CTO and EVP at intive, made a similar point from the banking perspective:

“AI’s most important role is helping banks understand what they already have before deciding on what comes next.”

That is a more conservative vision than full automation. It treats AI as a guide to understanding existing systems, not as a final authority on replacing them.

Background context: why modernization is so hard

Legacy replacement is difficult because the system itself is often larger than the code people can see. A large COBOL application may be intertwined with databases, job schedulers, manual procedures and decades of business decisions. If one part changes, the effect can ripple across a network of services that were never designed to be cleanly separated.

That is why formal verification and regression testing are common safeguards in high-stakes software changes. OpenAI’s statement that successful migration requires all tests to pass reflects that reality. Testing is not a formality here; it is the only practical way to compare the old behavior with the new one.

The scale can also be staggering. The Verge reported that one expert had seen a COBOL system with 250 million lines. Systems at that size are not refreshed with a quick rewrite. They are approached cautiously, often in pieces, with a heavy emphasis on not breaking what already works.

How should readers interpret the new AI tools?

They should be seen as promising but incomplete. OpenAI’s numbers indicate that newer models may be better at preserving behavior and handling legacy-language migration than older generations. That is real progress, especially if organizations are drowning in obsolete code and shrinking pools of engineers who understand it.

But promise is not proof of safe deployment. The difference between 87% and 98% may look small in a product demo and enormous in a bank’s production environment. The real standard is not whether the AI can generate a plausible rewrite. It is whether the rewritten system passes every required test, matches operational behavior and survives edge cases that only show up under load, in production or during audits.

That is why the industry’s language matters. OpenAI is talking about migration and modernization. Critics are talking about replacement risk and hidden technical debt. Those are not the same goal, and they should not be measured by the same standard.

What happens next

The biggest unanswered question is whether OpenAI will release more detailed benchmarks or testing methods. Without that, it is difficult for outside engineers, banks or government agencies to judge how transferable the results are to real-world systems.

Another open question is whether any government or bank will complete a real migration safely using these models. The fact sheet does not point to a completed deployment, which means the story is still about capability claims rather than proven rollout results.

Watch for three things:

• Whether OpenAI publishes more detail on the benchmark design and success criteria.
• Whether Anthropic responds with comparable metrics for its own modernization tools.
• Whether any financial institution or public-sector agency offers a public case study showing system-level modernization, not just code translation.

If those results arrive, the conversation could move from theoretical promise to operational proof. Until then, the central question remains the same: can AI modernize brittle legacy systems without losing the logic that keeps them safe?

Timeline

For now, the most defensible reading is also the most restrained one. OpenAI may be getting better at translating old code. That does not mean the hard part is solved.