Agentic RAG: from one-shot retrieval to research agents

Two years ago “RAG” mostly meant a one-shot pipeline: query → embedding → top-N chunks → answer. The 2025-2026 research changed that picture significantly. Retrieval is no longer a preprocessing step — it is a multi-step decision process the system runs. The literature calls this agentic RAG, and by 2026 it even has its own systematization: SoK: Agentic RAG (arXiv:2603.07379).

What changed?

• Retrieval became a policy. The model decides when to query, which source to use, and when to stop. Among the notable 2025-2026 techniques are retrieval policies trained with reinforcement learning and “deep research”-style agents; the survey (arXiv:2501.09136) maps the agentic RAG landscape well.
• Self-correction became standard. Approaches like CRAG (arXiv:2401.15884) evaluate the retrieved result and, if it is insufficient, re-search or fall back to the web; Self-RAG questions its own sources while generating.
• Interfaces got layered. A-RAG (arXiv:2602.03442) opens hierarchical retrieval interfaces so the model decides “when to search, what to search, at which granularity” — a toolbox instead of a single search() function.

When is it worth it?

Agentic RAG is not the right tool for every problem. Its cost is explicit: every extra retrieval round means extra latency, extra tokens and extra attack surface. A practical ladder:

1. Start with a simple pipeline. Query expansion + hybrid search + reranking is enough for most internal use cases.
2. Add a corrective layer. A step that retries when retrieval came back empty-handed has the best price/performance ratio.
3. Go full agentic only when the task demands it. Multi-hop questions (“find the contradictions across these three documents”), volatile sources and research-style tasks are exactly that.

When does it fail?

• Single-hop questions: an agentic flow produces the same answer at 3-5x the cost.
• Without evaluation: errors compound in multi-step systems; without measuring each step, “it feels better” is misleading. That is why agent evaluation became its own 2026 research topic — I cover it in Agent evaluation in 2026.
• With dirty sources: an agent may be better at finding wrong information; if it cannot verify what it finds, it reaches wrong conclusions faster.

Practical takeaway

Build the hierarchy in stages: a solid one-shot pipeline first, then a correction loop, then the full agent. Every stage transition should have a measurable justification. For the fundamentals see What RAG is (and when it fails); for an end-to-end implementation see the RAG-powered document assistant case study.