French startup Kog bets software can make GPUs dramatically faster for AI inference

French startup Kog says it can make large-language-model inference much faster on the GPUs companies already own, a claim that could help relieve one of AI’s biggest bottlenecks if it holds up in production. The 11-person company is now racing to prove that its software-first approach can deliver major speed gains on standard datacenter chips and turn early interest into a Series A.

Kog, founded by CEO Gaël Delalleau, first drew wide attention in May with a technical preview that suggested “extremely fast single-request decoding” could be achieved on conventional datacenter GPUs rather than on specialized inference hardware. The demo used AMD MI300X and NVIDIA H200 chips, and it sparked strong interest from enterprise buyers looking for lower latency, higher throughput and better economics in AI applications.

The startup is part of a growing group of companies arguing that the next leap in AI performance will not come only from new chips, but from better software that squeezes more work out of existing hardware. That thesis matters because inference — the process of running models after they are trained — is becoming a central cost and speed challenge for businesses building AI products.

Why Kog thinks the GPU bottleneck is solvable

Kog’s core argument is that today’s GPUs, especially newer datacenter parts, have far more memory bandwidth and raw capability than many AI systems are currently extracting. Rather than assuming dedicated inference accelerators are required for every workload, the company says careful low-level optimization can unlock much higher performance from mainstream hardware.

Delalleau has framed the issue as a misconception that GPUs are poorly suited to decoding. In his view, the problem is not the fundamental architecture of the chip, but how much of that architecture software can actually tap. The company’s pitch is that inference speed can improve dramatically without forcing enterprise customers to throw out expensive infrastructure and start over.

Delalleau says the startup believes newer GPUs offer increasingly rich memory bandwidth that remains underused, and that the right engineering can expose far more of their headroom.

That message resonated with people watching AI infrastructure costs rise. As model usage expands, latency becomes more than a technical annoyance: it affects user experience, cloud bills and whether certain products are viable at all. For some customers, a 20-second delay can be acceptable. For others, it can kill the workflow.

How Kog’s approach differs from chipmakers and other software plays

Kog is not trying to compete with chip designers by making a new processor. Instead, it is building a software stack it calls the Kog Inference Engine, or KIE, to speed up model execution on existing GPUs. That puts it in a different lane from purpose-built hardware startups, even if the end goal is similar: faster inference and lower cost per token.

The distinction matters because the AI infrastructure market is crowded with different answers to the same problem. Some companies are betting on specialized silicon. Others are building software layers that work across multiple vendors’ chips. Kog sits closer to the second camp, but with a much deeper focus on the hardware internals than many software-only rivals.

Delalleau compared the company’s orientation less to broad compatibility efforts and more to the kind of low-level work done in research labs. The emphasis is on digging into the behavior of each GPU architecture in a way that allows the software to exploit fine-grained details rather than merely abstracting them away.

What the demo showed

Kog’s demonstration in May claimed around 3,000 per-request tokens per second on a purpose-built model with roughly 2 billion parameters. The model, Laneformer 2B, was later open-sourced. The figure was eye-catching, but it came with an important caveat: the result was shown on a relatively small model, not on the much larger systems that dominate today’s enterprise AI use cases.

That gap is now the company’s biggest challenge. Kog’s broader promise is “30x faster LLM inference,” but translating a fast demo into a reliable product for major models is a much harder engineering problem. The startup has shifted its attention to larger models precisely because that is what potential customers say they want.

Who is Kog targeting first?

Kog’s first likely customers are software teams and professionals who use AI for high-value work and feel the pain of long waits. Delalleau said software engineering appears to be the clearest early use case, in part because experienced users of coding tools already know that output can take a long time to arrive.

That is a meaningful commercial point. If a developer is waiting an hour for a coding response, productivity suffers. If a company is paying a premium for faster-mode access to a model, as some users do with Anthropic’s Claude offering, there is already proof that speed can command a higher price.

Kog is also working with design partners in more consumer-facing or creator-oriented contexts, including prompt-based game and app generation. In those cases, faster inference would not merely improve the experience; it could directly increase revenue by reducing the time between a request and a monetizable output.

Why delay is expensive in AI workflows

In enterprise and professional settings, speed affects more than convenience. It can shape how many tasks a worker completes in a day, how often a model gets queried, and whether a user stays engaged long enough to finish a workflow.

• Developers may abandon tools that feel sluggish.
• Creators may generate fewer iterations if each one takes too long.
• Businesses may face higher cloud costs as slow inference ties up resources.
• Applications that rely on real-time interaction can become unusable if response times stretch too far.

Kog’s bet is that customers will pay for performance if the gains are large enough. That is especially true where AI is embedded in revenue-generating systems, not just experimental internal tools.

How did a solo founder build early momentum?

Delalleau said the company’s early traction came fast after the May preview, with roughly 200 tangible business leads following the attention from Hacker News. For a startup with a tiny team and no widely known product yet, that kind of inbound response suggests the market is actively searching for ways to reduce inference cost and latency.

The founder’s background helps explain why the company’s technical ambitions are unusually deep. Delalleau studied solid-state physics at École Polytechnique in France before moving into offensive cybersecurity, where he spent years working at a low level of systems understanding. He said that blend of scientific training and reverse-engineering experience shaped the mindset behind Kog.

Delalleau says his physics education taught him to understand the rules governing a system, while his cybersecurity work taught him to reverse-engineer software and hardware down to assembly and binary level in pursuit of a goal the original designers may not have intended.

That background is unusual for a startup CEO in the AI infrastructure space, where many founders come from machine learning research or large-scale cloud engineering. Kog’s founder appears to be leaning into a hacker-style philosophy: understand the machine as closely as possible, then make it do more than it was meant to do.

What makes Kog’s engineering strategy difficult?

Kog’s approach is labor-intensive by design. The company says that for each new GPU architecture, the team may spend weeks or even months studying its specifics and conducting engineering research before it can adapt its optimization methods.

That kind of work can yield impressive performance wins, but it does not scale easily. Kog has only 11 people, which means it must choose carefully which chips to support and how broadly it can expand in the near term. The more deeply the company tunes for a specific GPU, the more time and expertise each new port requires.

This creates a classic startup tension. Deep specialization may create a real performance edge, but broad market adoption usually requires portability and speed of rollout. Kog is trying to solve both problems at once.

Where Kog stands in the French AI ecosystem

Kog is one of several French companies trying to prove that software optimization can stretch the life and usefulness of existing AI infrastructure. ZML, another France-based startup, recently released hardware-agnostic software designed to bypass Nvidia’s CUDA stack and enable fast inference across competing chips.

Kog is taking a more intimate route, according to Delalleau, closer to academic work focused on GPU acceleration at a very low level. That places it somewhere between product company and performance research lab, which can be powerful if it succeeds, but risky if the gains prove hard to replicate outside the demo environment.

The broader European significance is also clear. If companies on the continent can build essential AI tooling without relying exclusively on US-based chip ecosystems, that could strengthen Europe’s position in the AI value chain. Kog already has support from French institutions and infrastructure partners that align with that goal.

Why Europe’s sovereignty push matters here

Kog’s story overlaps with a larger policy and industry debate in Europe: how to build strategic independence in AI while still competing with the biggest American and Chinese players. The startup is backed by Bpifrance and the French Tech 2030 program, and it is also supported by Scaleway, the cloud provider associated with Europe’s push for more domestic compute capacity.

That backing gives the company more than money. It signals that Kog fits a broader industrial strategy centered on local capability in compute, software and AI deployment. If the company can prove that it helps European enterprises get more out of hardware already installed in regional datacenters, it could become more than a niche optimization tool.

Still, sovereignty narratives only go so far. Investors and customers ultimately care about whether the product works, how much it saves, and whether performance gains are repeatable on mainstream models.

What has to happen next?

Kog’s immediate test is whether its optimization methods can deliver real speed gains on large language models, not just on a custom-built smaller model. Delalleau said the company expects to implement its first major model at roughly 10x speed in September, a milestone that would let it start showing customer traction.

If that happens, the startup plans to use the evidence to support a Series A fundraise. That next financing step will likely depend on whether Kog can show not only engineering breakthroughs, but also commercial demand strong enough to justify scaling the team and broadening hardware support.

That means the coming months are likely to be decisive. Kog needs to show:

1. That its software works on real, larger models.
2. That the speed-up is meaningful in customer workflows.
3. That the gains are reliable across supported GPU hardware.
4. That enterprise buyers are willing to pay for the improvement.

In the AI infrastructure market, demos can open doors, but production deployments close deals. Kog has earned attention with an ambitious engineering claim. The next stage is to prove that the claim survives contact with real customers and real models.

What the market reaction tells us about AI inference right now

The attention around Kog reflects a broader shift in the AI industry. Training breakthrough models still gets headlines, but the economics of running those models at scale are becoming just as important. Inference has moved from a back-end technical detail to a business-critical constraint.

That shift is why companies that can shave milliseconds or even seconds off model response times are attracting serious interest. It is also why software that improves throughput on widely available GPUs can be more attractive than it might have been a few years ago, when buyers were still focused mainly on raw model quality.

The strong reception to Kog’s preview suggests investors and customers are willing to examine unconventional approaches, even if they are not yet ready to fully believe them. The startup’s challenge is no longer getting noticed. It is converting curiosity into validated performance and, eventually, revenue.

Bottom line

Kog is trying to prove that the next big breakthrough in AI inference may come from software that extracts more performance from ordinary GPUs, not from replacing them. If the French startup can demonstrate its approach on large language models in the coming months, it could become a significant name in the race to make AI faster, cheaper and easier to deploy.

For now, the company has something many startups want but few have: a technical claim big enough to attract attention, and a near-term deadline that will determine whether that attention turns into a real business.