Researchers from Stanford, Together AI, UIUC, and Meta built HybridAgent, a system that reasons in latent space (no text tokens) for internal thinking and only switches to text when it needs to call tools or produce output. A novel HybridLink module maps hidden states back to input-embedding space, enabling seamless transitions. Results: 10.4% performance improvement, 35.5-54.7% token reduction, 1.8-2.6x faster inference, and 2.7-3.7x cost savings. An 8B checkpoint (Qwen3-8B base) is available on HuggingFace.
# The Token Tax on Thinking
Current AI agents think in text. Every internal reasoning step - planning what tool to use, evaluating intermediate results, deciding what to do next - gets serialized into tokens. Those tokens cost money, take time to generate, and consume context window space.
The insight behind HybridAgent: most of that internal reasoning does not need to be text. The model already has rich representations in its hidden states. Forcing those representations through the text bottleneck wastes compute. What if the model could think in its native latent space and only produce text when it actually needs to communicate?
Text-only agent (current approach): User query -> [think in text] -> [think in text] -> tool call -> [think in text] -> [think in text] -> tool call -> [think in text] -> final answer Tokens used: ~2,400 | Latency: 8.2s | Cost: $0.019 HybridAgent (this paper): User query -> [latent think] -> tool call -> [latent think] -> tool call -> [latent think] -> final answer Tokens used: ~1,090 | Latency: 3.2s | Cost: $0.005
# How HybridAgent Works
The architecture has three key innovations that make latent reasoning practical for tool-using agents.
The agent learns when to think in latent space and when to switch to text. Internal planning and evaluation happen as latent "thoughts" - continuous hidden-state vectors that never get decoded into tokens. Text mode activates only for tool calls and final outputs.
The core technical contribution. HybridLink maps hidden states from the model's output space back to its input-embedding space. This creates a continuous loop: the model produces a latent thought, HybridLink projects it back, and the model consumes it as the next input - all without generating a single token.
Stage 1 (HybridSFT) teaches the model to reason in latent space using supervised examples where text chain-of-thought is converted to latent representations. Stage 2 (HybridRL) optimizes the latent reasoning policy with reinforcement learning, improving task performance beyond what imitation learning achieves.
# Standard text reasoning: hidden_state = model.forward(input_tokens) text_tokens = model.decode(hidden_state) # expensive! next_input = model.embed(text_tokens) # re-encode! # HybridLink latent reasoning: hidden_state = model.forward(input_tokens) next_input = hybrid_link(hidden_state) # direct map! # No decode step. No token generation. No re-encoding. # The model loops through its own latent space. # When a tool call is needed, it switches to text mode.
# The Numbers
The team evaluated HybridAgent against text-only baselines across standard agentic benchmarks. Every metric improved.
Better accuracy on agentic benchmarks
~1,090 tokens vs ~2,400 per task
Latent reasoning skips token generation
Fewer tokens = lower API and compute costs
# The Surprise: Readable Latent Thoughts
Here is the unexpected part. When the researchers decoded the latent thoughts back into token space (using the model's own vocabulary), they found readable pseudo-text. The latent representations are not random noise. They look like compressed, slightly garbled versions of the chain-of-thought the model would have produced in text mode.
"I need to search for the current weather in Tokyo. Let me call the weather API with the location parameter set to Tokyo, Japan."
"search weather Tokyo... call weather_api location=Tokyo"
This has major implications for interpretability. Latent reasoning does not mean opaque reasoning. The model compresses its thoughts but does not encrypt them. You can still audit what the agent was "thinking" at each step.
No information bottleneck. Text forces continuous hidden states through a discrete token vocabulary. Latent reasoning preserves the full representational richness of the model's internal state.
Context window savings. With 54.7% fewer tokens, the same context window fits roughly twice the reasoning. For long-horizon tasks, this means fewer truncation failures.
RL-compatible. HybridRL optimizes the latent policy directly. Reinforcement learning on continuous representations is more stable than on discrete token sequences - fewer reward hacking modes.
Still inspectable. Decoded latent thoughts are readable pseudo-text. Safety auditing does not require a new interpretability framework - just a decode step that is already built into the model.
# Try It Yourself
The team released an 8B checkpoint built on Qwen3-8B as the base model. It is available on HuggingFace right now.
# Install dependencies pip install transformers torch # Load the HybridAgent checkpoint from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "hybrid-reasoning/hybrid-agent-8b", torch_dtype="auto", device_map="auto" ) tokenizer = AutoTokenizer.from_pretrained( "hybrid-reasoning/hybrid-agent-8b" ) # The model automatically switches between latent # and text modes based on the task context. # Base model: Qwen3-8B | Training: HybridSFT + HybridRL
HybridAgent proves that agents do not need to think in text. By reasoning in latent space and switching to text only for tool calls, the system achieves 10.4% better performance while using 54.7% fewer tokens, running 2.6x faster, and costing 3.7x less. The HybridLink module that bridges output and input embedding spaces is the key innovation. And the latent thoughts are still readable when decoded - so interpretability is not sacrificed. If this scales, every agent framework that generates long reasoning chains in text is leaving performance and money on the table. The 8B checkpoint is on HuggingFace. Go benchmark it.
Enjoyed this?
New episodes Mon, Wed, Sat.