Agentic RAG

When retrieval is an action: Self-RAG, CRAG, and Search-R1

Retrieval inside the loop

Source

A reader’s guide to part of CS 329Z Lecture 3 (Diyi Yang, Michael Ryan and John Yang; Engineering AI Agents, Stanford, autumn 2026). The outline and the pointers come from that lecture; the wording on this page is ours. The course is also listed under Resources.

Everything on the previous pages answers one question well: given a query, find passages and write. None of it decides whether to retrieve, what to ask for, or when to stop. Agentic RAG treats retrieval as an action the model can take, in a loop, rather than as a fixed first step.

That is the same turn already described on Compound AI systems and LLMs for agents: a workflow becomes more agent-like when the model writes the next step. The lecture’s three decisions are a useful test for any such loop.

Everyday example

A careful researcher does not always go to the library. Sometimes they already know the answer. Sometimes they need a different book from the one they first pulled, or a second search with a better keyword. And they need a rule for when to stop reading and start writing. Self-RAG is the habit of asking “do I need a source?” and of grading the page you found. CRAG is a critic who can send you back for a better search. Search-R1 is practice: you learn the habit by being rewarded for questions you eventually get right.

flowchart TD
  Q["Question"] --> D{{"Retrieve, or answer directly?"}}
  D -->|answer| A["Write the answer"]
  D -->|retrieve| S["Issue a search query"]
  S --> R["Read the result"]
  R --> G{{"Is this good enough?"}}
  G -->|yes| A
  G -->|no| S

Self-RAG

Self-RAG (Asai et al., ICLR 2024; the lecture writes 2024) trains a language model to retrieve on demand, to generate, and to critique its own uses of evidence. Special reflection tokens mark whether a retrieval was needed, whether a passage was relevant, and whether a claim is supported. Segments can be generated in parallel from different passages and then compared, so the model can prefer a supported span over an irrelevant one and, if need be, retrieve again.

The point is not a new index. It is a model that has learned the three decisions — retrieve or not, which evidence to trust, whether to try again — instead of following a fixed retrieve-then-generate graph.

CRAG: a critic on the retrieval

Corrective Retrieval Augmented Generation (Yan, Jiang, Ren and colleagues, 2024) adds an explicit evaluator. A critic looks at the retrieved documents, grades them as correct, ambiguous, or incorrect, and triggers different actions: keep and refine the internal knowledge, or rewrite the query and search the web when the retrieval is wrong. The generator then sees a cleaned or replaced context, not whatever the first retriever happened to return.

If Self-RAG folds the critique into the generator, CRAG factors it out as a retrieval evaluator. Both are answers to the same failure: a pipeline that always trusts its first top-\(k\).

Search-R1: learning the search policy

Search-R1 (Jin et al., 2025) goes a step further: it learns a multi-turn search policy with reinforcement learning. The model thinks in a <think> block, issues a <search> query when it lacks knowledge, reads <information>, and may search again before it commits to an <answer>. The training setup the lecture shows is the one we already met for verifiable rewards — PPO, and GRPO — with the search engine in the loop and a reward for a correct final answer.

The example in the lecture is a comparison of two birth dates: the model searches, finds one person, searches again, and only then answers. That is retrieval as a skill, trained against a checkable outcome, not as a preprocessor.

In one table

System	Who decides to retrieve?	What happens if the evidence is weak?
Naive RAG	The pipeline, always, once	The generator sees it anyway
Self-RAG	The model, via reflection tokens	Critique, and another retrieve if needed
CRAG	A critic on the retrieved set	Refine, or rewrite and search again
Search-R1	A learned, multi-turn policy	Another `<search>`, until it can answer

FAQ

Is agentic RAG just “RAG plus an agent framework”? Not necessarily. The papers above change who decides and how that decision is trained. You can wrap naive RAG in a tool loop; Self-RAG, CRAG and Search-R1 are recipes for making the loop itself smarter.

Do I always want the model to retrieve? No. Some questions are already in the weights, and an extra passage can distract. A model that can skip retrieval is part of the point of Self-RAG.

How is this different from ordinary tool use? Search is a tool. What is new is the policy: when to call it, how to rewrite the query, how to grade the result, and when to stop. For the shape of a single tool call, see Compound AI systems. For the risk of a loop that never stops, or that trusts a poisoned passage, see failure modes.