Agents & Tool Use
What you’ll learn. How LLMs call tools (function calling), the ReAct pattern and the basic agent loop, how to design tools and context for agents, the Model Context Protocol (MCP), memory and planning approaches, when multi-agent designs help, sandboxing and permissions, and how to make agents reliable and affordable. We implement a minimal agent loop from scratch so the mechanism is clear.
What is an agent?
Section titled “What is an agent?”A useful working definition: an agent is an LLM running in a loop, choosing actions (tool calls) based on observations from its environment until it decides the task is done. Contrast:
- Workflow: a fixed, developer-written control flow with LLM calls at some steps (prompt chaining, routing, parallel calls, evaluator-optimizer loops). Predictable, cheap, easy to test.
- Agent: the model decides the control flow dynamically. Flexible for open-ended tasks; less predictable, costlier, harder to evaluate.
Guidance widely echoed by practitioners: start with the simplest thing (a single call with good context), move to workflows when you need structure, and use agents only when the problem’s path can’t be predetermined and the value justifies the cost and variance.
Function calling (tool use)
Section titled “Function calling (tool use)”LLMs emit text. Tool use makes that text actionable through a protocol:
- You declare tools: a name, a natural-language description, and a JSON Schema for parameters.
- The model, instead of (or before) a final answer, outputs a structured tool call:
{name, arguments}. (Models are post-trained for this: Post-Training; providers may use constrained decoding to guarantee valid arguments, see Inference & Serving.) - Your code validates and executes the call. The model never runs anything itself.
- You append the result to the conversation as a tool-result message and call the model again.
- Repeat until the model returns a final answer.
user: "What's the weather in Oslo and should I bring an umbrella?"assistant: tool_call get_weather({"city":"Oslo"})tool: {"temp_c": 4, "precip_mm": 3.2}assistant: "It's 4 C with light rain expected; bring an umbrella."Models can issue parallel tool calls (several independent calls in one turn), which cuts latency. Tool choice can be automatic, forced, or disabled.
Designing good tools
Section titled “Designing good tools”Tools are the agent’s interface; poor tools are the most common failure cause.
- Clear names and descriptions that say what the tool does, when to use it and when not to; describe each parameter with examples. Treat descriptions as prompts.
- Few, non-overlapping tools. Ambiguity between near-duplicate tools confuses selection. Consolidate where reasonable; don’t mirror a raw REST API one-to-one.
- Return concise, high-signal output: pagination, truncation with a hint (“showing 10 of 240; refine query”), human-readable identifiers rather than opaque UUIDs where possible.
- Informative errors: tell the model what was wrong and how to fix it (“date must be YYYY-MM-DD”) so it can self-correct.
- Idempotency and safety: make mutating tools idempotent or require confirmation; separate read and write tools.
- Typed parameters with enums and validation (Pydantic/JSON Schema); never trust arguments.
The ReAct pattern and agent loop
Section titled “The ReAct pattern and agent loop”ReAct (Yao et al., 2022) interleaves Reasoning (a “thought”) with Acting (a tool call) and Observation:
Thought: I need the population of Oslo, then divide by area.Action: search("Oslo population")Observation: ~700,000Thought: Now I need the area......Final answer: ...Reasoning grounds decisions in observations and lets the model revise its plan. Modern APIs implement the same loop natively with tool calls and (for reasoning models) interleaved thinking.
A minimal agent loop
Section titled “A minimal agent loop”import json
TOOLS = [{ "name": "calculator", "description": "Evaluate an arithmetic expression with + - * / and parentheses.", "input_schema": {"type": "object", "properties": {"expression": {"type": "string"}}, "required": ["expression"]},}]
def run_tool(name, args): if name == "calculator": expr = args["expression"] if not set(expr) <= set("0123456789+-*/(). "): return "error: only digits and + - * / ( ) . allowed" return str(eval(expr, {"__builtins__": {}})) # demo only; use a real parser in production return f"error: unknown tool {name}"
def agent(llm, user_msg, max_steps=8): """`llm(messages, tools)` returns {"text": str|None, "tool_calls": [{id, name, args}]}.""" messages = [{"role": "user", "content": user_msg}] for step in range(max_steps): # hard step cap out = llm(messages, TOOLS) messages.append({"role": "assistant", "content": out["text"], "tool_calls": out["tool_calls"]}) if not out["tool_calls"]: return out["text"] # model decided it is done for call in out["tool_calls"]: try: result = run_tool(call["name"], call["args"]) except Exception as e: result = f"error: {e}" # feed errors back, don't crash messages.append({"role": "tool", "tool_call_id": call["id"], "content": result}) return "stopped: step limit reached"The real SDK shapes differ in field names (see Building LLM Applications for concrete API usage), but the structure is identical: loop, dispatch, append results, cap iterations, surface errors.
Model Context Protocol (MCP)
Section titled “Model Context Protocol (MCP)”Every app integrating every tool pairwise is an problem. MCP, an open protocol introduced by Anthropic in late 2024 and since adopted widely across providers and clients, standardizes it: MCP servers expose capabilities; MCP clients (inside an agent host/app) connect to them. A server can offer:
- Tools: callable functions with JSON Schema.
- Resources: readable data (files, records, documents) addressed by URI.
- Prompts: reusable templates.
Transport is JSON-RPC over stdio (local servers launched as subprocesses) or HTTP-based transports for remote servers, with OAuth-style authorization for remote ones. Benefits: write an integration once, reuse across hosts; tool discovery is dynamic. Concerns:
- Trust: an MCP server’s tool descriptions and outputs flow into the model’s context: a malicious or compromised server can inject instructions (tool poisoning), and combining a trusted server with an untrusted one creates cross-server exfiltration paths (Prompting).
- Context bloat: connecting dozens of servers puts hundreds of tool definitions in every request. Mitigate with tool search/lazy loading, or by having the agent write code that calls tools instead of loading all definitions.
- Permissions: scope credentials narrowly; prefer read-only; require approvals for writes.
Skim the specification and the official SDKs for current details; the protocol is evolving.
Memory
Section titled “Memory”An agent’s “memory” is whatever gets back into the context window:
- Short-term / working memory: the current conversation and tool results in context. Finite and costly: truncate, summarize, or clear old tool results.
- Long-term memory: facts persisted outside the window (files, a database, a vector store) and retrieved when relevant (RAG). Common forms: user preferences, project notes, past decisions, learned procedures (“skills” or playbooks written in files).
- Scratchpad / notes file: the agent writes progress notes and todo lists to disk and reads them after context resets; very effective for long-running tasks.
- Episodic vs semantic vs procedural is a handy taxonomy (events, facts, how-to), but the implementation is the same: write, retrieve, and decide what to forget.
Hard parts: deciding what to store (not everything), keeping memory correct and non-stale, avoiding memory poisoning (an injected instruction stored and replayed later), and privacy (deletion on request). See Prompting on context engineering.
Planning
Section titled “Planning”- Implicit planning: the model reasons step by step each turn. Fine for short tasks.
- Explicit plan-and-execute: produce a plan (a todo list), execute steps, update the plan as observations arrive. Keeping a visible, editable todo list in context improves coherence on long tasks.
- Search-based: explore alternatives (tree search, best-of- with a verifier) for problems with checkable outcomes (Reasoning Models).
- Reflection with external feedback: run tests, linters, or validators and let the model fix what they report. Reflection with a real signal works; self-critique without one is unreliable.
- Human in the loop: plan approval before side effects.
Long-horizon reliability decays: if each step succeeds with probability , an -step task succeeds with about (for : 36%). Shorten horizons, add verification at checkpoints, and make steps recoverable.
Multi-agent systems
Section titled “Multi-agent systems”Patterns:
- Orchestrator-worker: a lead agent decomposes the task and delegates to sub-agents, each with its own fresh context and tools, then synthesizes their condensed results. Good for breadth-first research and parallelizable work; sub-agents also protect the lead’s context from bloat.
- Specialists / routing: separate agents for different domains with a router.
- Debate / critic: one agent proposes, another critiques; helps when verification is easier than generation.
- Handoffs: one agent transfers control to another (customer-service triage).
Caveats: multi-agent systems multiply token cost (often several times a single agent’s), add coordination failures (duplicated work, conflicting edits, lost context between agents) and are harder to debug. They pay off mainly for parallelizable, high-value tasks. For tightly coupled tasks (such as most coding), a single agent with good tools often wins. Give sub-agents explicit task descriptions, output formats and boundaries.
Sandboxing and permissions
Section titled “Sandboxing and permissions”An agent with tools can do real damage: through its own mistakes or via prompt injection from untrusted content.
- Least privilege: minimal tools, scoped and short-lived credentials, read-only by default.
- Sandbox code execution and shell: containers or micro-VMs (gVisor, Firecracker) with no network by default or an egress allowlist, resource and time limits, an ephemeral filesystem with only the project mounted. Never run model-generated code on a host with secrets.
- Approval gates for irreversible or external effects (payments, emails, deletions, deployments); present exactly what will happen.
- Allow/deny lists for commands and domains; filesystem path restrictions.
- Secrets: keep credentials out of the model’s context; inject at the tool layer.
- Audit logs: record every tool call, arguments, result and the model’s stated reason.
- Assume prompt injection from any tool result, web page, email or file: bound the blast radius rather than relying on the model to refuse (Prompting).
Reliability
Section titled “Reliability”Typical failure modes: wrong tool or malformed arguments, infinite loops or repeated failing calls, premature “done” claims, ignoring constraints late in long contexts, compounding errors, flaky external services, and drifting from the original goal.
Practices:
- Constrain: schema-validated arguments, step and token budgets, timeouts, max retries with backoff, loop detection (same call repeated).
- Verify outcomes, not claims: run the tests, check the file exists, re-query the record. Make “done” depend on a checkable condition.
- Make state explicit: todo list, structured progress files, checkpoints you can resume from.
- Idempotent, reversible actions and transactional patterns for side effects.
- Evaluate trajectories, not just final answers: tool-selection accuracy, number of steps, recovery from errors, cost per success (Evaluating LLMs). Use fixed test environments with scripted or mocked tools, and sample multiple runs, since agents are stochastic (report pass rate over attempts; “pass^k”-style consistency metrics are informative for production).
- Observability: trace every step (Building LLM Applications); replay failures.
- Graceful escalation: when confidence is low or the budget is exhausted, hand off to a human with context.
Cost and latency
Section titled “Cost and latency”Agents are expensive because each loop iteration re-sends the growing context and the model generates tokens serially.
- Token cost grows roughly quadratically with steps if the whole history is resent each time (linear context growth times linear steps). Prompt caching (stable prefix first) cuts repeated-prefix cost sharply (Inference & Serving).
- Reduce: concise tool outputs, truncate/clear old observations, use smaller models for simple steps (routing, classification, extraction) and large ones for planning, parallelize independent tool calls, cap steps, cache deterministic tool results.
- Measure cost per successful task, not per call; a cheaper model that needs 3x the retries may cost more.
- Set budgets: max steps, max tokens, wall-clock timeouts, and per-user spend limits.
Key takeaways
Section titled “Key takeaways”- An agent is an LLM in a loop with tools; prefer simple workflows unless the path truly can’t be predetermined.
- Tool calling: declare schemas, the model emits structured calls, your code executes and returns results. Tool design (clear, few, concise outputs, helpful errors) is critical.
- MCP standardizes tool/resource integration across hosts; treat servers and their outputs as untrusted input and watch context bloat.
- Memory is what you put back in the context: working notes, retrieved long-term stores, and summaries; guard against poisoning and staleness.
- Reliability needs step caps, verification of outcomes, explicit state, trajectory evals and observability; long horizons compound errors ().
- Sandbox execution, minimize privileges, gate irreversible actions, and track cost per successful task.
Exercises
Section titled “Exercises”- Hands-on. Implement the agent loop against a real LLM SDK with two tools (search over a local docs folder and a calculator). Add loop detection and a token budget.
- Conceptual. An agent succeeds on each step with probability 0.97. At what horizon does end-to-end success drop below 50%? What interventions change this math?
- Hands-on. Write a small MCP server exposing one tool and one resource using the official Python SDK, and connect it to a client; observe the tool descriptions the model sees.
- Conceptual. Design the permission model for a coding agent that can run shell commands and open pull requests.
- Hands-on. Take 20 tasks, run the agent 5 times each, and compare pass@1, pass@5 and the all-5-succeed rate. What does the gap tell you?
Solutions (sketch)
- , so around 23 steps. Interventions: per-step verification and retry (raises effective ), shorter plans, checkpointing so errors don’t compound, decomposing into independent subtasks.
- Run in a sandbox container with no secrets and an egress allowlist; filesystem limited to the repo; allowlist of safe commands, approval for others; git credentials scoped to a fork or a branch-only token; PRs opened as drafts requiring human review; full audit log; no direct push to main.
Further reading
Section titled “Further reading”- Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models” (2022).
- Schick et al., “Toolformer” (2023); Patil et al., “Gorilla” (2023); the Berkeley Function-Calling Leaderboard for tool-use evaluation.
- Anthropic, “Building Effective Agents” (2024); Anthropic engineering posts on writing tools for agents and on context engineering.
- The Model Context Protocol specification and SDK documentation (modelcontextprotocol.io).
- Park et al., “Generative Agents” (2023); Wang et al., “Voyager” (2023); Shinn et al., “Reflexion” (2023).
- Yao et al., “tau-bench” (2024) and Jimenez et al., “SWE-bench” (2023) for agent evaluation.
- Willison, writing on prompt injection and the “lethal trifecta” (2025).