Inline grep beats vector retrieval in every agent harness tested. But switching harnesses caused a bigger accuracy swing (16 points) than switching retrieval methods. The architecture wrapping your search matters more than the search itself.
# The Experiment
116 questions from LongMemEval. Four agent harnesses. Two retrieval methods. The setup nobody had tested before.
# The Numbers
The punchline: Switching from vector to grep improved accuracy. But switching harnesses with the same retrieval method caused an even bigger swing. The architecture wrapping your search is the real variable.
# Why Grep Wins in Agent Loops
Speed
Agent loops iterate fast. Grep returns in milliseconds. No embedding computation, no index lookup, no network round-trip.
Precision
When it hits, it hits exactly. No "close but not quite" results polluting the context window with noise.
Token Efficiency
Grep returns only matching lines. Vector search returns full chunks with surrounding context you don't need.
Determinism
Same query, same results, every time. No embedding drift, no index staleness, no similarity threshold tuning.
# The Real Insight: Harness > Retrieval
The paper tested four harnesses with the same model and retrieval. The harness made the difference, not grep vs vector.
All using the same model (Claude Opus 4.6) + inline grep. The only variable is the agent harness. Dynamic prompts, tool-calling style, and how results are presented to the model created a 16-point accuracy gap.
# The Plot Twist: File-Based Results
When results were written to files instead of inlined into the context window, vector search came back. It beat grep in 5 out of 10 configurations.
Results appended directly into context window.
Grep wins. Precision matters when context is crowded.
Results written to files, agent reads separately.
Vector competitive. Semantic range becomes an asset when agent reasons deliberately.
# When to Use What
| Scenario | Use | Why |
|---|---|---|
| Tight agent loops, inline context | grep | Precision + speed + token efficiency |
| File-based tool results | vector | Semantic range is an asset here |
| Known patterns (error codes, IDs) | grep | Exact match = zero noise |
| Conceptual questions, paraphrases | vector | Needs semantic understanding |
| Production agent system | harness | 16pt gap from harness > retrieval |
grep Precision + speed + token efficiency
vector Semantic range is an asset here
grep Exact match = zero noise
vector Needs semantic understanding
harness 16pt gap from harness > retrieval
Don't default to vector search. Inside tight agent loops, grep is faster, cheaper, and more predictable. But the real lesson: stop obsessing over retrieval and start investing in your agent harness. The prompts, tool-calling patterns, and result delivery format matter more than whether you use grep or embeddings.
Enjoyed this?
New episodes Mon, Wed, Sat.