NVIDIA Research dropped Nemotron 3.5 on October 14 - a family of hybrid Mamba-Transformer models that surpasses Nemotron 3 Ultra on SWE-bench Pro with 4.6x fewer active parameters. The Super variant packs 120B total params into 12B active via MoE, hits 51.78% on SWE-bench Pro, and runs at 4,856 output tokens/s/GPU on GB200. NVFP4 quantization recovers 99.62% of accuracy while delivering 2.43x throughput over BF16.
# Two Models, One Hybrid Architecture
Nemotron 3.5 is not one model. It is a family of two, both built on the same hybrid Mamba-Transformer backbone - combining structured state space models (Mamba) for efficient long-sequence processing with attention layers for precise in-context reasoning.
Both models support text, images, video, and tool-use environments. The Super variant is the headline act, but Lightning at 3B active params is designed to run on edge hardware - a 40x parameter efficiency ratio that makes on-device agentic AI practical.
# SWE-bench Domination
The headline number is 51.78% on SWE-bench Pro - surpassing Nemotron 3 Ultra while using 4.6x fewer active parameters. But the story behind that number is more interesting than the number itself.
NVIDIA introduced agentic tasks directly into mid-training. DeepSWE scores jumped from 6.4% to 10.4% - a 62% relative improvement just from exposing the model to tool-use trajectories during pre-training, not post-training RLHF.
At inference time, multiple agent instances run in parallel and cross-verify results. This pushed Terminal-Bench from 62.7% to 71.9% - a 14.7% relative gain from the inference strategy alone, no model changes needed.
Multi-Token Prediction heads are trained alongside the base model and used at inference to improve speculative decoding acceptance rates. More tokens accepted per step means fewer forward passes needed per response.
The agentic mid-training result is the real breakthrough. Most labs bolt agentic capability onto models after pre-training through RLHF or tool-use fine-tuning. NVIDIA baked it into the pre-training data mix itself. The model learns to use tools as naturally as it learns to complete sentences.
# Inference Speed That Changes the Math
Raw intelligence means nothing if you cannot serve it. Nemotron 3.5 Super outputs 4,856 tokens per second per GPU on the GB200, a number that makes real-time agentic workloads economically viable at scale.
| Metric | BF16 Baseline | NVFP4 |
|---|---|---|
| Throughput | 1x | 2.43x |
| Accuracy recovery | 100% | 99.62% |
| Memory footprint | ~240 GB | ~100 GB |
| Bits per weight | 16 | 4 |
NVIDIA's custom 4-bit floating point format loses only 0.38% accuracy while more than doubling throughput. This is not generic GPTQ or AWQ - it is hardware-native quantization designed for the GB200's tensor cores, with per-channel scaling that preserves outlier weights.
Multi-Token Prediction heads are not just a training trick. At inference, they act as draft models for speculative decoding - but because they share the backbone, the overhead is near zero. Higher acceptance rates mean fewer wasted compute cycles per generated token.
# Why This Matters for Builders
Nemotron 3.5 is not just a benchmark model. It is a multimodal, tool-using, agentic system that processes text, images, and video while interacting with tool environments. The hybrid architecture is the key unlock.
Mamba layers handle long sequences in linear time. Transformers scale quadratically with sequence length. Mamba's state-space approach processes 128K+ context windows without the memory explosion. This is why the hybrid costs 4.6x fewer active params than a pure transformer of equivalent quality.
Transformer layers preserve in-context learning. Pure Mamba models struggle with tasks requiring precise retrieval from context. The attention layers in the hybrid handle exactly these cases - copy tasks, few-shot learning, and exact recall.
MoE keeps compute constant. Only 12B of 120B params activate per token. The routing network selects the right experts for each input, so the model has the knowledge of 120B params at the inference cost of 12B. Combined with Mamba's linear scaling, this is a double efficiency win.
The practical result: you can run a model that beats much larger pure transformers on coding benchmarks, at a fraction of the compute cost, while also processing images, video, and tool-use trajectories. That is the pitch for the post-transformer era - and NVIDIA just made its strongest case yet.
Nemotron 3.5 Super is the strongest evidence yet that hybrid architectures are the future of foundation models. 120B total, 12B active, 51.78% SWE-bench Pro, 4,856 tok/s/GPU on GB200, and 99.62% accuracy recovery under NVFP4 quantization. The Mamba-Transformer hybrid is not a research curiosity anymore. It is a production architecture that beats pure transformers on quality, speed, and cost simultaneously.
Enjoyed this?
New episodes Mon, Wed, Sat.