Researchers from Cambridge, EPFL, Anthropic, and Mistral introduce ORL (Open Reasoning via Latent diffusion), a model that generates text the way Stable Diffusion generates images: all tokens at once, then refined through iterative denoising. On a single RTX 5090, ORL hits 2,788 tokens per second - compared to roughly 60 tok/s for GPT-6 Comet's autoregressive decoding. It matches GPT-6 Luna quality on GPQA Diamond, MATH-500, and LiveCodeBench. The 3B and 8B checkpoints are fully open-sourced.
# What Is ORL?
Every LLM you use today - GPT-6, Claude, Gemini, Llama - generates text one token at a time, left to right. Each token depends on the previous one. This is called autoregressive (AR) decoding, and it is inherently sequential. You cannot parallelize it.
ORL flips this entirely. Instead of predicting one token at a time, it generates all tokens simultaneously as a noisy initial draft in a continuous latent space, then refines the entire sequence through iterative denoising steps - exactly how Stable Diffusion generates images from noise.
Step 1: The _ _ _ _ _ _ _ Step 2: The cat _ _ _ _ _ _ Step 3: The cat sat _ _ _ _ _ Step 4: The cat sat on _ _ _ _ ... Step 8: The cat sat on the warm mat . Speed: ~60 tok/s (sequential)
Step 1: ██ ███ ███ ██ ███ ████ ███ █ Step 2: Tha cet sot en tha werm met . Step 3: The cat sat on the warm mat . Step 4: The cat sat on the warm mat . Speed: 2,788 tok/s (parallel)
The key insight: reasoning happens in a continuous latent space, not in discrete token space. The model thinks in vectors, not words. Only at the final step does it decode the latent representation back into text tokens. This means the entire "thinking" process is trivially parallelizable across GPUs.
# The Benchmarks
Speed is meaningless if quality drops. ORL does not drop quality. The 8B checkpoint matches GPT-6 Luna (the smaller, faster GPT-6 variant) across three of the hardest reasoning benchmarks.
Read those throughput numbers again. 2,788 vs ~60 tokens per second. That is not a marginal improvement. That is a paradigm shift. ORL-8B is slightly below GPT-6 Luna on LiveCodeBench (53.9% vs 54.3%) but matches or exceeds on the other two benchmarks - while being 25x faster than Luna and 46x faster than Comet.
# How It Works
ORL has three core innovations that make diffusion-based text generation competitive with autoregressive models for the first time.
Instead of operating on discrete tokens (vocabulary IDs), ORL maps the input into a continuous latent space using a learned encoder. All "reasoning" - the actual computation that transforms a question into an answer - happens in this continuous space. Discrete tokens are only produced at the final decoding step. This avoids the sequential bottleneck because continuous vectors can be processed in parallel.
Not all problems need the same number of refinement steps. ORL uses a learned difficulty estimator that predicts how many denoising iterations a given input requires. Simple questions (factual recall, short answers) may need only 4-8 steps. Complex multi-step reasoning (MATH-500 level) may use 32-64 steps. This dynamic scheduling is why ORL stays fast on easy queries without sacrificing accuracy on hard ones.
Because every token position is processed simultaneously, ORL scales linearly with GPU count. Two RTX 5090s give you roughly 5,500 tok/s. Four give you roughly 11,000. Autoregressive models hit diminishing returns with multi-GPU setups because of the sequential token dependency. ORL has no such constraint - each denoising step is a single forward pass over all positions.
Input: "What is the derivative of x^3 * sin(x)?" Step 1 - Encode: tokens --> encoder --> latent vectors (continuous) Step 2 - Initialize: z_T = gaussian_noise(shape=[max_output_len, latent_dim]) Step 3 - Denoise (N steps, all positions in parallel): for t in range(T, 0, -1): z_t-1 = denoise_step(z_t, input_latents, t) # N = 4-64 steps (adaptive to difficulty) Step 4 - Decode: z_0 --> decoder --> output tokens Output: "3x^2 * sin(x) + x^3 * cos(x)"
# Why This Changes Everything
ORL is not just a faster model. It is a fundamentally different architecture that breaks assumptions the entire LLM industry is built on.
2,788 tok/s on a single RTX 5090 means high-quality reasoning at speeds that feel instant. No data center required. The 3B checkpoint runs on a laptop GPU. This democratizes access to reasoning-capable models in a way that trillion-parameter autoregressive models never will.
If you can serve the same quality at 46x the throughput, your cost-per-token drops by roughly 46x. For companies spending millions on API calls, ORL-class architectures could reduce inference costs from dollars to cents. The economics of LLM deployment fundamentally change.
Autoregressive models hit a wall with multi-GPU parallelism because each token depends on the previous one. ORL processes all token positions simultaneously, so adding GPUs gives near-linear throughput gains. Two GPUs = 2x speed. Four GPUs = 4x speed. This makes scaling trivial.
Both the 3B and 8B checkpoints are released under permissive licenses. The training code, evaluation harness, and pre-trained weights are all available. Cambridge + EPFL + Anthropic + Mistral collaborating on open research is notable - especially given Anthropic's and Mistral's commercial interests.
GPUs AR (GPT-6 Comet) ORL-8B ------ ---------------- ------ 1x 5090 ~60 tok/s 2,788 tok/s 2x 5090 ~90 tok/s ~5,500 tok/s 4x 5090 ~140 tok/s ~11,000 tok/s 8x 5090 ~200 tok/s ~22,000 tok/s ^diminishing ^linear
ORL proves that autoregressive decoding is not the only path to high-quality text generation. 2,788 tokens per second on a single consumer GPU, matching GPT-6 Luna quality on hard reasoning benchmarks, with linear multi-GPU scaling. The 3B and 8B checkpoints are open-sourced by a collaboration between Cambridge, EPFL, Anthropic, and Mistral. If this architecture scales to larger models, the autoregressive paradigm that every major LLM is built on becomes the slow, expensive option. Paper: arXiv 2610.17894.
Enjoyed this?
New episodes Mon, Wed, Sat.