cd ~/series/demystifying-ai
Nemotron 3.5: NVIDIA's Mamba-Transformer Hybrid Eats SWE-bench
EP 50 NEW RELEASE Oct 28, 2026 5 min

Nemotron 3.5: NVIDIA's Mamba-Transformer Hybrid Eats SWE-bench

120B parameters, 12B active. A hybrid Mamba-Transformer MoE that hits 51.78% on SWE-bench Pro while running at 4,856 output tokens per second per GPU on GB200.

Share:
// TL;DR

NVIDIA Research dropped Nemotron 3.5 on October 14 - a family of hybrid Mamba-Transformer models that surpasses Nemotron 3 Ultra on SWE-bench Pro with 4.6x fewer active parameters. The Super variant packs 120B total params into 12B active via MoE, hits 51.78% on SWE-bench Pro, and runs at 4,856 output tokens/s/GPU on GB200. NVFP4 quantization recovers 99.62% of accuracy while delivering 2.43x throughput over BF16.

120B
Total Params (Super)
12B
Active Params
51.78%
SWE-bench Pro
4,856
Tok/s/GPU (GB200)

# Two Models, One Hybrid Architecture

Nemotron 3.5 is not one model. It is a family of two, both built on the same hybrid Mamba-Transformer backbone - combining structured state space models (Mamba) for efficient long-sequence processing with attention layers for precise in-context reasoning.

SUPER FLAGSHIP
Total params 120B
Active params 12B
Architecture MoE Hybrid Mamba-Transformer
SWE-bench Pro 51.78%
Modality Text, image, video, tools
LIGHTNING EFFICIENT
Total params 30B
Active params 3B
Architecture Hybrid Mamba-Transformer
Target Edge/on-device
Modality Text, image, video, tools

Both models support text, images, video, and tool-use environments. The Super variant is the headline act, but Lightning at 3B active params is designed to run on edge hardware - a 40x parameter efficiency ratio that makes on-device agentic AI practical.

# SWE-bench Domination

The headline number is 51.78% on SWE-bench Pro - surpassing Nemotron 3 Ultra while using 4.6x fewer active parameters. But the story behind that number is more interesting than the number itself.

AGENTIC TRAINING PIPELINE
1
Agentic Mid-Training

NVIDIA introduced agentic tasks directly into mid-training. DeepSWE scores jumped from 6.4% to 10.4% - a 62% relative improvement just from exposing the model to tool-use trajectories during pre-training, not post-training RLHF.

2
Parallel Agent + Self-Verification

At inference time, multiple agent instances run in parallel and cross-verify results. This pushed Terminal-Bench from 62.7% to 71.9% - a 14.7% relative gain from the inference strategy alone, no model changes needed.

3
MTP Boosting

Multi-Token Prediction heads are trained alongside the base model and used at inference to improve speculative decoding acceptance rates. More tokens accepted per step means fewer forward passes needed per response.

KEY INSIGHT

The agentic mid-training result is the real breakthrough. Most labs bolt agentic capability onto models after pre-training through RLHF or tool-use fine-tuning. NVIDIA baked it into the pre-training data mix itself. The model learns to use tools as naturally as it learns to complete sentences.

# Inference Speed That Changes the Math

Raw intelligence means nothing if you cannot serve it. Nemotron 3.5 Super outputs 4,856 tokens per second per GPU on the GB200, a number that makes real-time agentic workloads economically viable at scale.

Metric BF16 Baseline NVFP4
Throughput 1x 2.43x
Accuracy recovery 100% 99.62%
Memory footprint ~240 GB ~100 GB
Bits per weight 16 4
⚡ NVFP4 quantization

NVIDIA's custom 4-bit floating point format loses only 0.38% accuracy while more than doubling throughput. This is not generic GPTQ or AWQ - it is hardware-native quantization designed for the GB200's tensor cores, with per-channel scaling that preserves outlier weights.

🚀 MTP speculative decoding

Multi-Token Prediction heads are not just a training trick. At inference, they act as draft models for speculative decoding - but because they share the backbone, the overhead is near zero. Higher acceptance rates mean fewer wasted compute cycles per generated token.

# Why This Matters for Builders

Nemotron 3.5 is not just a benchmark model. It is a multimodal, tool-using, agentic system that processes text, images, and video while interacting with tool environments. The hybrid architecture is the key unlock.

WHY HYBRID MAMBA-TRANSFORMER WINS
01

Mamba layers handle long sequences in linear time. Transformers scale quadratically with sequence length. Mamba's state-space approach processes 128K+ context windows without the memory explosion. This is why the hybrid costs 4.6x fewer active params than a pure transformer of equivalent quality.

02

Transformer layers preserve in-context learning. Pure Mamba models struggle with tasks requiring precise retrieval from context. The attention layers in the hybrid handle exactly these cases - copy tasks, few-shot learning, and exact recall.

03

MoE keeps compute constant. Only 12B of 120B params activate per token. The routing network selects the right experts for each input, so the model has the knowledge of 120B params at the inference cost of 12B. Combined with Mamba's linear scaling, this is a double efficiency win.

The practical result: you can run a model that beats much larger pure transformers on coding benchmarks, at a fraction of the compute cost, while also processing images, video, and tool-use trajectories. That is the pitch for the post-transformer era - and NVIDIA just made its strongest case yet.

// Bottom Line

Nemotron 3.5 Super is the strongest evidence yet that hybrid architectures are the future of foundation models. 120B total, 12B active, 51.78% SWE-bench Pro, 4,856 tok/s/GPU on GB200, and 99.62% accuracy recovery under NVFP4 quantization. The Mamba-Transformer hybrid is not a research curiosity anymore. It is a production architecture that beats pure transformers on quality, speed, and cost simultaneously.

Enjoyed this?

New episodes Mon, Wed, Sat.