Agent Planning & ReAct

The Agent Loop

The core loop never changes: (1) receive task, (2) reason about next action, (3) call a tool/act, (4) observe result, (5) update state, repeat until done. This is the heartbeat of every agent.

Why the Loop Matters

Without planning: one-shot LLM call to generate everything → brittle, no recovery from errors, fails on unseen edge cases. With planning: multi-step, each step informed by previous results → adaptive, self-correcting, can recover from mistakes by replanning.

Basic Agent Trace

Each step depends on the previous observation. The agent reasons explicitly before acting. The observation updates context. This loop enables agents to recover from failures and adapt.

ReAct: Reason + Act Interleaved

ReAct (Yao et al. 2022): interleave chain-of-thought reasoning (Thought) with concrete actions (Action/Observation) in the same context. Why it works: explicit Thought forces the model to commit to a plan before acting; Observation updates the context; looping gives recovery from errors.

Comparison: Naive vs CoT vs ReAct

ReAct Prompt Structure & Example

Plan-and-Execute

Two-phase: (1) generate a full plan upfront (list of steps), (2) execute each step with a subagent or tool. Planner = LLM call that produces a list of steps; Executor = ReAct agent for each step, passing results forward.

Advantages and Disadvantages

Advantages: Plan is auditable, explainable, interruptible; enables parallelism (independent steps can run in parallel); stakeholders can review before execution. Disadvantages: upfront plan can be wrong; hard to re-plan mid-execution without restarting; rigid step decomposition fails on emerging needs.

Plan-Then-Execute Example

Reflection and Self-Critique

Reflexion (Shinn et al. 2023): after a failed attempt, generate a textual critique, store it in episodic memory, try again. Self-RAG: LLM decides whether to retrieve, generates, critiques its own answer for factuality. The reflection loop: generate → critique → refine → generate again.

Reflexion Loop Example

ReAct vs Plan-and-Execute vs Reflexion

Tree Search and Monte Carlo Planning

Tree of Thoughts (ToT): explore multiple reasoning paths simultaneously, backtrack from dead ends. MCTS for agents: score each candidate action with a value function (another LLM call), expand promising branches. o1/o3 test-time compute: a form of learned tree search — model runs internal reasoning traces and scores them.

Cost and Applicability

Tree search is expensive: N calls per step where N = branching factor. Explore 3 branches × 4 levels deep = 81 LLM calls. Reserve for high-stakes tasks: competition, formal verification, critical decisions. For most production agent tasks (tool-use, RAG), ReAct is 10× cheaper and works just as well.

Tree of Thoughts Structure

Tool Selection and Error Recovery

Tool selection failure modes: wrong tool chosen, correct tool with wrong args, tool returns error, infinite loop. Mitigations are critical for production reliability.

Mitigation Strategies

Tool descriptions: must be unambiguous. Bad: "search". Good: "web_search(query: str) → list of {title, url, snippet}". Validation: validate tool args before calling. Max iterations: enforce a ceiling on loop depth. Error handling: handle tool errors gracefully, don't let exceptions bubble up.

Robust Tool-Call with Retry

Memory Types and State

Agents need multiple memory systems working together. Each serves a different purpose and has different constraints.

Four Memory Types

Short-term (in-context): conversation history, scratchpad, tool results — limited by context window, cleared at session end. Long-term (external): vector store of past experiences, episodic memory, fact cache — persistent across sessions, queryable. Procedural (prompt): the agent's "skills" — encoded in system prompt, updated via prompt engineering, static within session. Checkpoint (persisted state): full graph state saved to DB (LangGraph pattern), enables pause/resume across sessions.

Memory Comparison Table

Memory Tools and Platforms

• PAPER ReAct: Synergizing Reasoning and Acting in Language Models — arxiv 2210.03629
• PAPER Reflexion: Language Agents with Verbal Reinforcement Learning — arxiv 2303.11366
• PAPER Tree of Thoughts: Deliberate Problem Solving with LLMs — arxiv 2305.10601
• DOCS LangGraph Agent Documentation — agent execution patterns
• BLOG LangChain Blog: Plan-and-Execute Agents — design patterns