The ReAct Loop: How Every AI Agent Actually Works
Thought, action, observation: the loop that turned one-shot text generators into agents that can check their work.
Every AI agent that checks its work, uses external tools, or recovers from errors runs the same core loop: ReAct. The model interleaves reasoning steps (Thought) with tool calls (Action) and integrates their results (Observation) until it can produce a final answer grounded in real data. This loop is what separates a text generator from a reasoning system that can act on the world.
Without ReAct, asking an assistant “does my dentist appointment clash with anything next week?” returns a confident hallucination drawn from the training set. With ReAct, the same underlying model searches your actual inbox, reads the calendar, and compares real data. The model did not change. The loop around it made all the difference.
Why does a plain language model hallucinate facts it could easily look up?
A plain language model is a one-shot function: tokens in, tokens out. It has no memory of your inbox or your calendar, so the only way it can answer a question is by predicting what someone on the internet might have said. When you ask about your dentist, it cannot pause, retrieve a real email, and reconsider. It must guess.
ReAct borrows a metaphor from how a detective works a case. A detective does not produce a conclusion the moment she hears the crime. She thinks about what she knows, interviews a witness (action), reads the statement (observation), rethinks her hypothesis, consults records, and repeats. The loop ends only when the evidence supports a conclusion worth stating publicly.
Mailmind, our fictional inbox assistant, faces exactly that detective problem. A user asks, “When is my dentist appointment, and does it clash with anything?” Mailmind thinks: I need the appointment confirmation. It calls search_inbox("dentist"), reads the returned email, extracts the date and time, then thinks: Now I need the calendar for that week. It calls get_calendar(week_of_appointment), observes the results, compares entries, and finally answers with the real appointment time and any conflicts. At every step the model is the same. The loop provides the missing real-world signal.
What does each step in the loop actually do?
The ReAct loop cycles through four distinct phases. They are not architectural jargon. They are concrete positions that every agent fills with text and tool output.
Thought. Internal reasoning text that the model generates. It answers “what do I know so far?” and “what do I need next?”. This is often invisible to the end user, but it is the only place the agent can reflect and replan. For Mailmind, a typical thought might be: The user asked about a dentist. I found an appointment confirmation from SmileCare on July 8 at 10 AM. Now I need to see if anything else is on the calendar for that morning.
Action. A structured tool call that the model emits. It names a function and provides parameters. The agent is not actually searching the web. It is asking a host program to run a function. Mailmind’s actions are things like search_inbox(query="dentist"), get_calendar(date="2026-07-08"), or draft_reply(to="dentist@smilecare.com", body="…"). The model does not execute these. The agent runtime does.
Observation. The raw output from the tool, fed back into the conversation as part of the next prompt. If search_inbox returns a JSON (JavaScript Object Notation) blob with email snippets, that blob becomes the observation. The model sees it exactly as the runtime provides it. A good observation contains enough signal for the next thought. A bad observation, like an empty string on a failed search, forces the model to guess the reason for failure.
Final answer. Once the model judges it has enough observations, it produces a user-facing answer and stops looping. It does not need to call more tools. It does not need to add “let me double check” unless explicitly instructed to. The final answer is the only output the user ever sees.
How many iterations does a real agent need?
Simple tasks finish in 2 or 3 iterations. Complex tasks, like “summarize all my receipts from last year for tax season,” can take 10 to 15 or more. An uncapped agent will sometimes wander: it will re-search the inbox with slightly different keywords, over and over, convinced it might find a better result. Without a stop signal, it can loop forever.
Every production ReAct agent sets a hard max_iterations limit. Mailmind’s runtime might cap at 20. If the cap is hit, the agent must produce whatever partial answer it has, or it must escalate to a human. This is not optional. An agent that loops indefinitely blocks queue items behind it and burns compute for no value. You set the cap based on observed task complexity plus a safety margin.
What happens when a tool call fails?
Tool failure becomes just another Observation. If search_inbox returns an HTTP 500 or a connection timeout, the agent runtime packages that error into a structured message and places it in the conversation. The model sees: Error: search_inbox timed out after 5 seconds.
Now it has a reasoning problem. It can retry the same call with a longer timeout. It can fall back to a different tool, like search_inbox_by_date. It can infer the data it needs from previous observations. Or it can conclude the task is not completable and return a graceful “I couldn’t find your appointment” message. What matters is that the error contains enough context for the model to choose. A generic “ERROR” string is useless. A message like “Tool unavailable: DNS resolution failed for mailhost.internal” gives the model a fighting chance.
Mailmind’s runtime wraps every tool in a thin error handler. If the tool raises an exception, the handler catches it and formats it as an observation: Tool search_inbox failed with error: [message from the exception]. This keeps the agent loop alive and gives the model the same chance a developer would have to debug.
How does ReAct stay within the model’s context window?
Every iteration adds a thought, an action definition, and an observation to the conversation. Over 15 iterations, the prompt can easily exceed 8,000 or even 32,000 tokens, and the model has a finite limit. If the context window fills, the agent loses the ability to see its earliest steps. It forgets what it already searched or what it concluded.
The most common defense is a sliding window that keeps the most recent N tokens plus a summary of older turns. Another is to compress observations: instead of passing back a full 10-page PDF, the runtime truncates or summarizes it before injecting it. A third defense is to design the task so each iteration fetches only what it needs, not the entire world. For Mailmind’s receipt summary, processing 12 months of data in one giant loop will blow the context window quickly. The smarter approach is to chunk by month, process each chunk in its own sub-loop, store intermediate results outside the context, and assemble the final answer from those stored summaries.
- Each iteration adds thought, action, and observation, growing the prompt unbounded
- Risk of exceeding the model's context window
- Earlier observations may be lost from the context
- Keeps only the most recent N tokens in the prompt
- Older turns are summarized to preserve context
- Stays within the model's token limit
Quick Reference
| Property | Typical value |
|---|---|
| Default reasoning format | Thought: ... Action: ... Observation: ... |
| Simple task iterations | 2-3 |
| Complex task iterations | 10-15 |
Required guard max_iterations | 20-30 depending on task |
| Tool failure representation | Structured error observation |
| Context window defence | Sliding window, truncation, or chunked sub loops |
Frequently Asked Questions
Q: Can I run ReAct on any LLM (large language model), or does it require fine-tuning?
You can prompt most capable models to follow ReAct without fine-tuning. A system message that defines the Thought/Action/Observation format and a few examples usually suffice. Some models, especially smaller ones, benefit from a few-shot prompt or lightweight instruction tuning, but the loop itself is an orchestration pattern, not a model modification.
Q: What if the agent gets stuck in an infinite loop of repeated actions?
The max_iterations guard stops it. Additionally, you can detect loops by hashing recent actions and breaking if an action repeats N times. Many production agents add a “stuck detector” that inserts a special observation telling the model “You have repeated the same action twice. Try a different approach or output what you have.”
Q: Does ReAct guarantee a correct answer?
No. The tools can return incorrect data, the model can misinterpret an observation, or the reasoning chain can lead to a wrong conclusion. ReAct reduces hallucination by grounding answers in retrieved data; it does not eliminate it. The answer is only as good as the tools and the logic.
Q: How is ReAct different from earlier chain-of-thought prompting?
Chain-of-thought generates reasoning steps and then a final answer in one forward pass. ReAct interleaves reasoning with external tool calls that bring new information into the generation. Chain-of-thought can only reason about what the model already knows. ReAct can reason about the world outside the model.
Q: What is the typical latency overhead of adding ReAct compared to a single LLM call?
Each iteration is a full LLM invocation plus tool execution time. A 3-iteration loop multiplies the base inference latency by roughly three, plus the tool latencies. For latency-sensitive applications, you keep iterations low, use fast tools (millisecond in-memory fetches, not network calls), and consider streaming partial responses while the loop runs.
Test yourself
Mailmind is summarizing a year of receipts for tax season. The agent is at iteration 25 of a 30-iteration cap and has not yet produced a final answer. It keeps fetching pages of emails and adding them to its observation buffer. The previous 24 iterations are all in context. What is the root cause, and what would you change?
Answer: The root cause is too much data per iteration and a naive loop that keeps growing the context window. The agent is loading all 12 months of receipts into one long running conversation. Solve it by restructuring the task. First, pre-filter inbox search to return only receipt emails, not every message. Then split the year into 12 monthly chunks. For each chunk, run a short sub-loop (2-3 iterations) that extracts merchant names, dates, and totals, and stores a structured summary in external memory. Process chunks in parallel where possible. Only after all chunks are summarized, run a final aggregation loop to combine the monthly summaries into one tax report. Set a hard cap: if any monthly sub-loop exhausts its iterations, save a partial summary flagged “incomplete: 8 of 12 months processed.” Track which chunks succeeded. That way a user gets something useful, not a silent timeout.
If understanding how these systems actually work under the hood saves you from deploying an agent that loops forever on your users, subscribe to Internals Decoded at internalsdecoded.com. Next in this series: the tool sandbox and how an agent runtime decides what the model is actually allowed to call.