Your RAG System Shouldn't Use the Same Retrieval Strategy for Every Query

Your RAG system probably shouldn't use the same retrieval strategy for every query. 🔍

A recent arXiv paper — Experience-RAG Skill — explores an interesting idea: what if retrieval strategy itself was a pluggable agent skill?

A fact lookup, a multi-hop question, and a scientific verification task don't necessarily need the same retrieval pipeline. Instead of hardcoding that logic into the agent, Experience-RAG puts a skill layer between the agent and a pool of retrievers:

• Analyze the task — understand what kind of retrieval the query needs
• Consult experience — use previous retrieval outcomes to inform the strategy
• Choose a retriever — select the appropriate strategy from the available pool
• Return structured evidence — keep retrieval complexity away from the main agent

The Architectural Shift

The interesting part is the separation of concerns. Your agent doesn't need to know whether the answer came from BM25, dense vector search, or a hybrid multi-hop strategy. It just asks:

"Can you retrieve the evidence I need?"

The paper reports an overall nDCG@10 of 0.8924 across NQ, HotpotQA, and SciFact benchmarks, outperforming fixed single-retriever baselines.

The Engineering Takeaway

Don't make retrieval strategy part of your agent's core business logic. Treat it as a replaceable capability that can evolve independently.

Of course, if your application has simple, uniform queries, this extra layer may be unnecessary. But for complex RAG systems with heterogeneous query types, retrieval orchestration deserves to be a first-class architectural concern.