cd ~/series/demystifying-ai
Mercury 3: The Diffusion LLM at 1,000 Tokens Per Second
EP 46 NEW RELEASE Oct 19, 2026 5 min

Mercury 3: The Diffusion LLM at 1,000 Tokens Per Second

Inception Labs just proved that diffusion language models can compete with autoregressive giants. Mercury 3 generates 1,068 tokens per second, matches GPT-6 Luna quality, and costs a tenth of the price.

Share:
// TL;DR

Inception Labs launched Mercury 3, a diffusion language model that generates 1,068 tokens per second - 4-8x faster than GPT-6 Luna - while matching its quality. It has a 1M token context window, costs $0.10/M input tokens, and ships with specialized voice and structured-decision models. This is the first proof that diffusion LLMs scale to compete with autoregressive models on both speed and quality.

1,068
Tokens/Second
1M
Context Window
$0.10
Per 1M Input Tokens
4-8x
Faster Than GPT-6 Luna

# Why Diffusion Changes Everything

Every LLM you use today is autoregressive. GPT-6, Claude, Gemini, Llama - they all generate tokens one at a time, left to right. Each token waits for the previous one. That sequential bottleneck is why inference is slow and expensive, no matter how much hardware you throw at it.

Mercury 3 is a diffusion language model. Instead of predicting one token at a time, it starts with noise across the entire output and iteratively refines all tokens in parallel. Think of it like a sculptor revealing a statue from a block of marble rather than building it brick by brick. The result: massive parallelism that scales with GPU cores rather than sequence length.

AUTOREGRESSIVE (GPT-6, Claude)
T1 → T2 → T3 → ...

One token at a time, sequential. Speed limited by depth, not width.

DIFFUSION (Mercury 3)
T1 T2 T3 T4

All tokens refined in parallel. Speed scales with GPU width.

Mercury 3 is 40x larger than Mercury 1 (their first diffusion model) yet achieves comparable speed. That scaling trajectory is the real story. If diffusion models maintain speed as they grow, the autoregressive bottleneck becomes a competitive disadvantage.

# How It Stacks Up

Inception claims quality on par with GPT-6 Luna and Claude Haiku 5.5. Here is what the numbers look like side by side.

SPEED (TOKENS/SECOND)
GPT-6 Luna~150-250
Claude Haiku 5.5~180-280
Mercury 31,068
INPUT PRICE (PER 1M TOKENS)
GPT-6 Luna$1.25
Claude Haiku 5.5$0.80
Mercury 3$0.10
CONTEXT WINDOW
GPT-6 Luna256K
Claude Haiku 5.5200K
Mercury 31M

The context window jumped from 260K (Mercury 2.5) to 1M tokens. The pricing includes a launch 50% discount, but even at full price ($0.20 input / $0.70 output) it undercuts every comparable model by 4-10x. These are the numbers that make platform teams rewrite their cost projections.

# Beyond the Base Model

Inception did not just ship a faster model. They shipped a product suite targeting the two use cases where speed matters most: voice and structured decisions. CEO Stefano Ermon (Stanford professor, diffusion model pioneer) is betting that speed-sensitive applications are where diffusion wins first.

🎙️ Mercury Voice

Purpose-built for real-time voice agents. Time-to-first-token under 170ms - fast enough that voice conversations feel natural with no perceptible delay. At 1,068 tok/s, the model finishes generating before the TTS engine needs the next chunk.

TTFT < 170ms Real-time
🎯 Mercury Decide

A structured decision model that outputs calibrated probabilities instead of free-form text. Feed it a decision with options, get back ranked choices with confidence scores. Designed for routing, classification, and agentic tool selection where you need reliable structured output at extreme speed.

Structured output Calibrated
AVAILABILITY
Inception API
OpenRouter
Baseten
Vercel

OpenAI API compatible. Drop-in replacement - swap the base URL and API key, keep your existing code. No SDK changes, no prompt rewrites.

DROP-IN REPLACEMENT
# Before (OpenAI)
client = OpenAI(
    api_key="sk-...",
    # base_url defaults to api.openai.com
)

# After (Mercury 3) - change two lines
client = OpenAI(
    api_key="mercury-...",
    base_url="https://api.inceptionlabs.ai/v1",
)

# Everything else stays the same
response = client.chat.completions.create(
    model="mercury-3",
    messages=[{"role": "user", "content": "..."}],
)

# Why This Matters Right Now

Mercury 3 is not just a faster model. It is the first credible proof that the autoregressive paradigm has a real competitor. Here is what engineers should pay attention to.

⚡ Speed unlocks new use cases

At 1,068 tok/s, you can run inference in real-time loops: voice agents, live coding assistants, game NPCs, and interactive tutors that respond faster than human perception. These applications were cost-prohibitive with autoregressive models.

💰 Economics flip the math

At $0.10/M input and $0.35/M output, you can process 10x more tokens for the same budget. Batch jobs that cost $500/day on GPT-6 Luna could drop to $40. That is the difference between "prototype only" and "ship to production."

📈 Scaling trajectory matters

Mercury 3 is 40x larger than Mercury 1 with comparable speed. Autoregressive models get slower as they scale. Diffusion models get smarter at the same speed. If this trajectory holds, the next generation could match GPT-6 Pro quality at Mercury 3 prices.

🔧 Zero migration cost

OpenAI API compatibility means you can test Mercury 3 in your existing stack without changing a single line of application code. Swap the base URL, run your eval suite, and compare. The switching cost is effectively zero.

// Bottom Line

Mercury 3 is the moment diffusion LLMs stopped being a research curiosity. 1,068 tokens per second, 1M context, GPT-6 Luna quality, and pricing that makes autoregressive models look like luxury goods. Inception Labs, led by Stanford's Stefano Ermon, just gave every team building speed-sensitive AI applications a reason to run benchmarks this week. The question is no longer "will diffusion LLMs work?" - it is "how fast will they improve?"

Enjoyed this?

New episodes Mon, Wed, Sat.