French startup ZML bets on cross-chip AI inference in challenge to Nvidia lock-in

A Paris-based startup backed by some of the biggest names in European and global AI is taking aim at one of the industry’s most stubborn bottlenecks: the difficulty of running large language models efficiently across different types of chips. ZML, a French company with support from Turing Award winner Yann LeCun and several high-profile founders and investors, has unveiled a new inference server designed to run open-source models on hardware from Nvidia, AMD, Google, Apple and Intel.

The launch adds fresh momentum to a growing push to reduce dependence on any single chip supplier at a time when AI costs are rising and inference is becoming a bigger part of the equation. While training the large models that power today’s AI systems still grabs the headlines, the real day-to-day expense for many companies is increasingly in inference — the process of handling user prompts and generating responses at scale.

ZML’s new product, called ZML/LLMD, is being offered for free at launch. The company says the goal is to gather usage data, understand what enterprises need, and eventually monetize in a way that does not slow adoption. Its pitch is simple but ambitious: make AI software portable across chips, squeeze more performance from existing infrastructure, and give customers more leverage when deciding how to deploy AI workloads.

What ZML is launching

ZML/LLMD is an inference server aimed at running open-source large language models across a broad range of hardware. In practice, that means the software is built to help models perform efficiently on Nvidia GPUs, AMD chips, Google TPU systems, Apple’s Metal stack and Intel Arc hardware.

That interoperability is the core of ZML’s value proposition. Rather than forcing enterprises into a single vendor’s ecosystem, the startup wants to help customers mix and match chips based on performance, energy use, availability and cost.

Founder Steeve Morin told TechCrunch that the goal is to remove the silos that have long made AI infrastructure more fragmented than it appears from the outside. In his view, the limits are not only hardware-related. Software stacks, system architecture and vendor-specific tools often create hidden barriers that make it hard to move workloads efficiently from one platform to another.

Those barriers matter more as AI becomes embedded in products and internal workflows. For companies trying to deploy models at scale, the prompt-processing layer is becoming increasingly important — and increasingly expensive. A system that can improve speed across more than one chip family may therefore deliver both technical and financial upside.

Why inference has become the new battleground

The AI industry spent much of the last few years focused on model training. That is still a capital-intensive and strategically important task, but the economics of AI are shifting. As more users interact with AI systems every day, inference is taking a larger share of total compute demand.

That is one reason analysts and investors have started describing the market as an “inference gold rush.” The phrase reflects a simple reality: once a model is built, the ongoing cost of serving it often becomes the larger business challenge.

For ZML, that shift creates an opportunity. If enterprises can run models faster, more flexibly and on cheaper or more energy-efficient chips, they may be able to lower operating costs without sacrificing performance. That is especially relevant for cloud providers and large organizations that need to manage compute budgets carefully.

“The idea is to give people back the power to create their own system and achieve real efficiency gains that allow AI to be disseminated,” Morin said.

The company is not positioning itself as anti-Nvidia. Morin said ZML maintains a good relationship with the chip giant and recognizes its existing supply and influence. But ZML is also betting that the market is mature enough to support more choice, especially as inference demand expands and customers become more sensitive to cost and efficiency.

Cross-chip support and the case for flexibility

One of the strongest arguments for ZML’s approach is that AI customers increasingly want optionality. A single infrastructure strategy can expose companies to pricing risk, supply constraints and performance bottlenecks. A cross-chip software layer gives organizations more room to optimize around their own workloads.

That flexibility can matter in several ways:

• Cost control: customers can choose hardware based on price-performance tradeoffs rather than compatibility alone.
• Energy efficiency: some workloads may run more economically on less power-hungry chips.
• Supply resilience: companies can reduce dependence on one vendor’s availability.
• Deployment freedom: enterprises may be able to place workloads where their existing infrastructure already lives.

Morin also suggested that ZML’s technology could help give a lift to emerging AI chip makers, many of them based in Europe. He named companies including Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud and VSORA as examples of firms that could benefit from software able to make use of their hardware.

That said, Morin framed the opportunity less as a European protectionist story and more as a technological one. The point, he said, is not simply where a chip is made, but whether ZML can help unlock performance outcomes that have not been achieved before.

The competitive landscape is getting crowded

ZML is entering a market that is already attracting significant investment and attention. Inference software has become a hot category, and several well-funded companies are competing to define the technical and commercial standards for the next phase of AI infrastructure.

Among ZML’s better-known rivals is Baseten, which was recently valued at $13 billion. Others include Inferact, which comes from the creators of the open-source project vLLM, and RadixArk, the commercial company behind SGLang.

Those projects overlap with ZML in different ways. Both vLLM and SGLang compete at least partially with LLMD, but Morin said ZML’s ambitions are broader than serving as another inference stack. The startup is trying to build for a wider slice of the infrastructure layer, including the emerging convergence between software and chip design.

Morin said the market has reached a stage where software teams and hardware designers are increasingly working in tandem, describing the current moment as one in which companies are effectively co-designing silicon.

That is a notable statement for a startup with just 20 employees. But in AI infrastructure, small teams can move quickly if they have the right technical depth, strong investor backing and clear product focus. ZML appears to have all three.

How ZML is funded and who is watching

Morin’s background helped open doors. He previously served as vice president of engineering at Zenly, the social mapping app acquired by Snapchat in a deal worth nine figures in 2017. That track record helped ZML raise $20 million from a group of venture investors that includes 20VC, >commit, AALVC, Drysdale Ventures, Kima Ventures, Kindred Capital, LocalGlobe and Puzzle Ventures.

The investor list is also notable for the names it connects to the broader AI ecosystem. The company’s cap table includes founders and operators such as Solomon Hykes, the founder of Dagger and Docker; Clément Delangue and Julien Chaumond of Hugging Face; and LeCun, who is now affiliated with AMI Labs.

That mix of investors matters for more than prestige. It suggests that ZML is being watched closely by people who understand both the software and infrastructure sides of the AI stack. It also reinforces the idea that Europe’s AI scene is not merely catching up, but increasingly producing companies that can attract global attention on their own terms.

Why the product is free for now

Despite the product’s strategic ambitions, ZML is launching LLMD as a free offering rather than an open-source release. That choice marks a shift from the company’s first public project, an inference-focused machine learning framework that came out in 2024 and was updated in March.

The free launch is not an act of charity. Morin said the company wants to learn from real usage before deciding how to commercialize the product. In his view, making a premature move to monetize could slow growth and limit the startup’s ability to understand where customers derive the most value.

Morin argued that he would rather measure how the product is used first and monetize later in the areas that prove most effective, instead of being too aggressive too early and hurting adoption.

That approach reflects a familiar startup playbook, especially in developer infrastructure. Free tools can help win users, reveal workflows, and create a path to enterprise adoption. The challenge is turning that engagement into revenue without damaging trust or momentum.

It is still unclear when ZML/LLMD will become a paid product or what pricing model the company will eventually adopt. For now, the launch is as much about market intelligence as it is about revenue.

France’s AI scene and the argument for building in Paris

ZML’s launch also speaks to a broader shift in Europe’s startup landscape. For years, major AI infrastructure companies were more likely to emerge from the United States or cluster near the largest U.S. cloud and chip ecosystems. But that geography is changing as talent, capital and technical ambition deepen across Europe.

Morin said he could only have built ZML in Paris, a comment that reflects both personal conviction and a growing confidence among European founders that top-tier AI work can happen close to home. The statement also suggests that Paris is increasingly seen as a credible base for ambitious technical startups, not just an outpost for local market plays.

That matters because AI infrastructure businesses tend to be globally competitive from day one. A startup needs access to engineers, customers, investors and hardware expertise — and all of that must come together quickly. ZML’s progress suggests the French capital can support that mix.

What makes this moment important for Nvidia and its challengers

Nvidia remains dominant in AI compute, and no one is suggesting that dominance is about to disappear. But the market is widening around it. Buyers want alternatives, regulators are watching concentration risks more closely, and chip suppliers are racing to prove they can deliver enough performance to justify switching.

ZML sits at the intersection of those trends. Its software is not designed to replace hardware innovation; it is designed to make more hardware usable, more efficiently. In a world where AI demand keeps growing, that kind of abstraction layer can become strategically valuable.

For Nvidia, the rise of cross-chip inference tooling is not necessarily a threat in the short term. It may even deepen the company’s relevance by making its own products easier to integrate into mixed environments. But over time, software that reduces lock-in can expand customer choice and make procurement decisions more competitive.

That is why the launch matters. It is less a declaration that one chipmaker is losing and more a sign that the infrastructure stack is becoming more modular, more contested and more aware of cost.

How the product fits into the broader AI stack

To understand ZML’s strategy, it helps to place it in the context of the AI software stack.

Where inference sits

At a high level, an AI model passes through several stages before it is useful to end users. It is trained, fine-tuned, deployed and then served repeatedly through inference. Each query, prompt or request consumes compute resources. When the user base is large, those incremental costs add up quickly.

Why infrastructure software matters

The companies winning in AI infrastructure are often not the ones with the flashiest demos. They are the ones that can make models easier to serve, cheaper to operate and more portable across environments. Infrastructure software can be the difference between a promising model and a scalable business.

What ZML is trying to solve

ZML is aiming at that middle layer between model development and final deployment. Its bet is that the future belongs to systems that can run the same workload across different hardware with minimal friction, while adapting to the needs of each customer and each deployment environment.

Key facts at a glance

Timeline of ZML’s rise

What happens next

ZML is still early in its commercial life, and several questions remain unanswered. How quickly developers and enterprises will adopt the new server is one of them. Another is whether the company can turn technical differentiation into durable market share in a sector already crowded with well-financed competitors.

Still, the startup’s positioning is strong. It has a focused product, a clear market problem, heavyweight backers and a founder with a proven track record. It is also targeting a problem that matters more every month: how to run AI economically at scale.

If ZML succeeds, it could help prove that the next phase of AI infrastructure will not be defined solely by the biggest chip seller. Instead, it may be shaped by the software layers that make compute more flexible, more portable and less expensive to use.

That is the bet ZML is making from Paris — and it is a bet with implications well beyond France.