ReAct: what the reasoning + acting paper actually says

Reading Yao et al. (Princeton/Google Brain, ICLR 2023) after watching an agent answer a multi-hop research question by hallucinating citations it never actually retrieved.

The agent had been running fine for two weeks. It would receive a question, call a search tool, read the result, and produce an answer. Then someone asked it whether a specific researcher had published work on a topic after 2022. The agent called search, got back results about a different researcher with a similar name, and then answered confidently using the wrong person's publication list. Not because the tool failed — the observation was right there in the context. The model just didn't update its reasoning based on what it actually retrieved.

This is the failure mode that ReAct is specifically designed to prevent: an agent that acts without reasoning, or reasons without grounding its conclusions in what actions actually returned.

"ReAct: Synergizing Reasoning and Acting in Language Models" — Yao, Zhao, Yu, Du, Shafran, Narasimhan, and Cao from Princeton and Google Brain, ICLR 2023 — introduces a specific prompting format that interleaves explicit reasoning traces and tool actions in an alternating chain. The approach is simple enough to explain in a paragraph and has enough production failure modes to fill the rest of this post.

The problem the paper is actually solving

The prior work splits into two camps that each fail in predictable ways.

Chain-of-thought reasoning (Wei et al., 2022) improved LLM performance on multi-step problems by prompting models to generate intermediate reasoning steps before the final answer. The model thinks through the problem, showing its work. The problem is that the model is reasoning against its internal knowledge, which is frozen at training time, can be factually wrong, and has no mechanism for correction when the model's assumptions are faulty. The paper calls this "hallucination and error propagation": the model generates a confident-looking reasoning trace that compounds incorrect premises into an incorrect answer, and each generation step treats the prior step's output as ground truth.

Action-only approaches — retrieving documents and passing them to the model — give the model access to external information but remove the explicit reasoning scaffold. The model sees a query, gets back search results, and must synthesize an answer. The problem is that for multi-hop questions, the model needs to reason about which intermediate question to ask before it can know what to retrieve. "Who was the director of the film that won Best Picture in the year the Berlin Wall fell?" requires knowing what year the Wall fell before you can ask who directed the Best Picture winner of that year. If your action step is a flat search over the original question, you'll retrieve irrelevant results and the model will either hallucinate or fail to answer.

ReAct's thesis: interleaving reasoning and actions creates a loop where reasoning guides actions and observations update reasoning. The model can adjust its plan mid-trajectory based on what it actually finds.

The thought/action/observation format

The paper's core contribution is a prompting format. A ReAct trajectory looks like this:

Question: What is the elevation range for the area that the eastern sector
of the Colorado orogeny extends into?

Thought 1: I need to search the Colorado orogeny, find the area that its
eastern sector extends into, then find the elevation range of the area.
Action 1: Search[Colorado orogeny]
Observation 1: The Colorado orogeny was an episode of mountain building (an
orogeny) in Colorado and surrounding areas...the eastern sector extends into
the High Plains...

Thought 2: The eastern sector of Colorado orogeny extends into the High
Plains. So I need to search High Plains and find its elevation range.
Action 2: Search[High Plains]
Observation 2: High Plains refers to one of two distinct land regions...The
High Plains are a subregion of the Great Plains...elevation 1,800 to 7,000 ft.

Thought 3: High Plains elevation range is 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft.
Action 3: Finish[1,800 to 7,000 ft]

Every step is a text token sequence. Thoughts are free-form reasoning; actions are structured calls that the system parses and routes to tools; observations are tool outputs injected into the context. The model generates thoughts and actions; the environment generates observations.

There's no new model architecture here. No fine-tuning required for the core paper results. It's all in-context: few-shot examples in the prompt show the model what the thought/action/observation format looks like, and the model continues the pattern for new questions. This is what makes it immediately practical and also what makes it brittle in specific ways.

What the benchmarks actually show

The paper evaluates on four tasks across two categories.

Knowledge-intensive QA and fact verification:

HotpotQA: Multi-hop Wikipedia QA requiring two-step reasoning. ReAct with Wikipedia API access achieves higher accuracy than CoT alone and direct retrieval-then-answer baselines, and generates less hallucinated content in intermediate steps. The paper reports that CoT trajectories often go wrong in the reasoning chain before any retrieval happens, inventing intermediate facts. ReAct forces each reasoning step to be grounded by a retrieval before proceeding.
FEVER: Fact verification. Similar pattern — ReAct outperforms CoT by catching cases where the model's internal knowledge contradicts the actual evidence retrieved. The model's initial "thought" might be wrong, but when the observation comes back contradicting it, the next thought can incorporate the correction.

Interactive decision-making:

ALFWorld: A text-based simulation where an agent must navigate a house, find objects, and complete tasks ("put a cool apple in the kitchen cabinet"). ReAct achieves 34% absolute improvement in success rate over imitation learning and reinforcement learning baselines, using only 1-2 in-context examples versus the baselines' thousands of training demonstrations. The gap is striking: the RL baseline had access to extensive training data. ReAct's prompting-only approach with minimal examples beat it by a large margin.
WebShop: Web shopping navigation — search for a product matching specifications, navigate listings, select the correct item. ReAct achieves a 10% absolute improvement over the best baseline with 1-2 in-context examples.

The interactive tasks are the more interesting result. Language model pretraining gives the model implicit knowledge of how sequential task completion works; ReAct's thought steps help it maintain state across steps ("I found a red apple but I need a cool one, so I should put it in the fridge first"). Without explicit thoughts, the model loses track of sub-goals across long observation sequences.

The paper also shows that fine-tuning a smaller model on ReAct-format trajectories outperforms prompting larger models. A smaller model fine-tuned on ReAct traces from GPT-3 generalizes better than prompting a larger model. The format's value isn't just about model capability — it's a structural inductive bias that benefits from being baked in.

What the error analysis says

The paper's error analysis on HotpotQA categorizes failure modes into three buckets:

Reasoning errors: The model generates a plausible-sounding thought that's factually wrong, then acts on it. The thought says "I'll search for the capital of X" but the model misstates what X is. The action retrieves information about the wrong entity.

Search errors: The Wikipedia API returns a disambiguation page or an unhelpful result. The model's subsequent thought doesn't catch this, treating the bad observation as informative.

Recovery failures: The model hits a dead end — a search returns nothing useful — but doesn't backtrack. It continues generating thoughts and actions that diverge further from the correct answer rather than trying a different search strategy.

Recovery failures are the most production-relevant failure mode. ReAct allows for backtracking in principle: the model can write "Thought: The previous search didn't help, let me try a different query." But whether it does depends entirely on the examples in the prompt. If your few-shot examples never demonstrate backtracking, the model won't backtrack. The paper's examples include recovery behaviors; most production implementations don't carefully construct prompts with failure recovery demonstrations.

Production tradeoffs no one mentions in the framework docs

Every thought step costs tokens and latency. Thoughts are generated tokens. For a 5-step trajectory with 3-sentence thoughts, that's ~200 tokens of reasoning overhead before you get an answer. At typical inference speeds, that's noticeable latency. At $15/M output tokens for frontier models, it's noticeable cost. Multi-agent systems where multiple ReAct agents call each other amplify this.

Thoughts can be decorative. The most insidious failure mode in production: the model generates thoughts that look like reasoning but don't actually constrain subsequent actions. The thought says "I need to find X" and then the action searches for Y. This happens because thoughts and actions are in the same output sequence — the model is optimizing for text that looks correct given the context, not for thoughts that causally determine actions. I've seen trajectories where changing the thought to say the opposite didn't change the action. The thought was being written to match the action that was already sampled, not the other way around.

Observations pollute the context. Each observation is injected into the context window. Tool output is often verbose — search results, API responses, code execution output. A 5-step trajectory can easily accumulate 3K–5K tokens of observation context. For long tasks, you hit the context limit before completing them. Truncating observations degrades quality in ways that are hard to predict because the truncation happens in the middle of content the model might need for the final synthesis step.

Parsing actions from generated text is fragile. ReAct's format requires extracting Action: SearchQuery[query term] from free text. This works in controlled demos. In production, the model occasionally generates malformed actions — missing brackets, wrong action name spelling, wrapped in quotation marks, capitalized differently. You need a parser that handles these variants, and every variant you don't handle causes the agent to stall or error. Teams that use more rigid JSON output formats instead of the paper's natural-language action format get fewer parse errors but lose some of the "thought influences action" expressiveness.

Infinite loops. Without a hard step limit, agents loop. The model reaches a dead end, generates a thought that sounds like it's about to do something different, repeats the same action, gets the same observation, generates a similar thought, repeats. I've seen this run for 30+ cycles before hitting a timeout. The paper's benchmarks use bounded trajectory lengths; production agents need explicit loop detection — watching for repeated action patterns over a sliding window — which adds complexity that the framework usually doesn't provide.

Observation grounding is asymmetric. ReAct's correction mechanism requires the model to actually incorporate observation content into subsequent thoughts. This works reliably when observations are concise and clearly relevant. When observations are long, contain the relevant fact buried in the middle, or are structured in a way the model doesn't recognize as informative, the model's next thought often ignores the observation and continues its prior reasoning trajectory. External grounding only helps if the model reads the observation.

When not to use ReAct

Single-step tasks. If the task requires one tool call and one synthesis step, standard function calling or retrieval-augmented generation is simpler, faster, and equally accurate. ReAct's overhead is the thought chain; if there's one link in the chain, you've added latency with no benefit.

When your tools return structured, reliable output that you control. ReAct's reasoning scaffold helps when tool output is unpredictable and the model needs to interpret it. If your tool always returns a JSON object with a known schema and you can parse it deterministically, the model doesn't need to reason about what the output means. A structured pipeline with explicit control flow is more debuggable.

When you need deterministic action sequences. ReAct generates thoughts and actions stochastically. Temperature=0 helps but doesn't fully eliminate variability. If your task requires that the agent always takes specific actions in a specific order — compliance workflows, auditable financial processes — the non-determinism is a problem that no amount of prompt engineering fully solves.

When your context budget is constrained. Each thought-action-observation cycle burns context. For tasks where trajectories get long — research synthesis, complex debugging, multi-document analysis — you'll hit context limits before completing the task. Approaches that compress or summarize intermediate observations can help, but at that point you're building significant scaffolding around a prompting technique.

When you can't construct good few-shot examples. ReAct's quality depends heavily on the quality of in-context examples. Good examples need to demonstrate: correct tool choice, recovery from bad observations, appropriate stopping conditions, and concise thoughts that actually reflect the reasoning path. This takes significant engineering effort. If your domain is novel enough that you can't write good examples from first principles — or if the task distribution is wide enough that a few examples don't cover it — the quality will be variable.

What the paper gives you

ReAct established the vocabulary for a design pattern that's now used in essentially every agent framework: the thought/action/observation loop. LangChain's AgentExecutor, Claude's tool use with thinking, OpenAI's function calling with internal reasoning — these are all variations on this structure.

The paper's insight is underrated: pure chain-of-thought fails because reasoning without grounding compounds errors; pure retrieval fails because acting without reasoning can't direct multi-step information gathering. The interleave fixes both. It's the right idea, demonstrated on real benchmarks with meaningful margins.

What the paper doesn't tell you is how to handle loop detection, observation summarization, thought quality monitoring, or parse error recovery — all of which become necessary at production scale. The framework docs for ReAct-based agents often don't tell you either. The result is that teams build the standard agent loop, see it work in demos, and then discover these failure modes individually in production.

The error analysis categories in the paper are worth reading carefully before you build: reasoning errors, search errors, recovery failures. These are exactly the categories you'll see in your agent's error logs. Knowing what to look for makes the debugging faster.

For your implementation: enforce a step limit from day one, log the full thought/action/observation trajectory for every run, watch for repeated action-observation pairs as a loop signal, and spend real time writing few-shot examples that include recovery from bad observations. ReAct works. It just works less cleanly than the demos suggest.

ReAct: Synergizing Reasoning and Acting in Language Models — Yao, Zhao, Yu, Du, Shafran, Narasimhan, Cao. ICLR 2023.