Skip to content

Project: Build an Agent from Scratch in Python

What you’ll learn. By writing a small but complete agent you will connect everything so far: the loop, tool registry with schemas, error handling, a permission gate, context trimming, budgets, tracing, and tests with a scripted fake model. The result is a file-and-shell “workspace assistant” you can point at any provider by writing a 15-line adapter.

We deliberately avoid frameworks. Components:

LLMClient (protocol) <-- adapter for any provider
|
Agent.run(task) -----> loop: call model -> execute tool calls -> append results
| |-- Policy: allow / ask / deny
| |-- budgets: steps, tokens (approx), wall-clock
| |-- trim: elide old tool output
v
Tracer (JSONL) + final answer

Provider-neutral message format. We use our own small dicts:

  • {"role": "user" | "assistant" | "tool", "content": str, ...}
  • Assistant messages may carry "tool_calls": [{"id", "name", "arguments": dict}].
  • Tool messages carry "tool_call_id" and "name".

An adapter translates this to a provider’s wire format and back. The model interface is one method:

class LLMClient(Protocol):
def chat(self, messages: list[dict], tools: list[dict]) -> dict:
"""Return an assistant message: {'role':'assistant','content':str,'tool_calls':[...]}"""

Save as agent.py. It uses only the standard library.

from __future__ import annotations
import fnmatch, json, pathlib, subprocess, time, uuid
from dataclasses import dataclass, field
from typing import Callable, Protocol
class LLMClient(Protocol):
def chat(self, messages: list[dict], tools: list[dict]) -> dict: ...
# ---------- tools ----------
@dataclass
class Tool:
name: str
description: str
parameters: dict # JSON Schema
fn: Callable[..., str]
risk: str = "read" # read | write | exec
def spec(self) -> dict:
return {"name": self.name, "description": self.description,
"parameters": self.parameters}
def make_tools(root: pathlib.Path) -> dict[str, Tool]:
root = root.resolve()
def safe(path: str) -> pathlib.Path:
p = (root / path).resolve()
if root != p and root not in p.parents:
raise PermissionError(f"path escapes workspace: {path}")
return p
def read_file(path: str, start: int = 1, limit: int = 200) -> str:
p = safe(path)
if not p.is_file():
names = sorted(x.name for x in p.parent.glob("*"))[:30] if p.parent.exists() else []
return f"ERROR: no such file {path}. Nearby: {names}"
lines = p.read_text().splitlines()
chunk = lines[start - 1:start - 1 + limit]
out = "\n".join(f"{i}: {l}" for i, l in enumerate(chunk, start))
if start - 1 + limit < len(lines):
out += f"\n[truncated: {len(lines)} lines total; use start/limit]"
return out or "[empty]"
def write_file(path: str, content: str) -> str:
p = safe(path)
p.parent.mkdir(parents=True, exist_ok=True)
p.write_text(content)
return f"wrote {len(content)} chars to {path}"
def list_dir(path: str = ".") -> str:
return "\n".join(sorted(x.name + ("/" if x.is_dir() else "")
for x in safe(path).iterdir())) or "[empty]"
def run_shell(command: str, timeout: int = 30) -> str:
r = subprocess.run(command, shell=True, cwd=root, capture_output=True,
text=True, timeout=timeout)
out = (r.stdout + r.stderr)[-4000:]
return f"exit={r.returncode}\n{out}"
obj = lambda props, req: {"type": "object", "properties": props, "required": req}
s, i = {"type": "string"}, {"type": "integer"}
tools = [
Tool("read_file", "Read a text file with line numbers.",
obj({"path": s, "start": i, "limit": i}, ["path"]), read_file),
Tool("write_file", "Create or overwrite a file in the workspace.",
obj({"path": s, "content": s}, ["path", "content"]), write_file, "write"),
Tool("list_dir", "List a directory in the workspace.",
obj({"path": s}, []), list_dir),
Tool("run_shell", "Run a shell command in the workspace; returns exit code and output.",
obj({"command": s, "timeout": i}, ["command"]), run_shell, "exec"),
]
return {t.name: t for t in tools}
# ---------- policy ----------
@dataclass
class Policy:
allow: list[str] = field(default_factory=lambda: ["read_file:*", "list_dir:*",
"run_shell:pytest*"])
deny: list[str] = field(default_factory=lambda: ["run_shell:*rm -rf*", "run_shell:*sudo*"])
confirm: Callable[[str], bool] = lambda msg: False # default: refuse to ask
def check(self, tool: Tool, args: dict) -> str | None:
key = f"{tool.name}:{args.get('command') or args.get('path') or ''}"
if any(fnmatch.fnmatch(key, p) for p in self.deny):
return "DENIED by policy"
if any(fnmatch.fnmatch(key, p) for p in self.allow):
return None
return None if self.confirm(f"Allow {tool.name} {args}?") else "DENIED: not approved"
# ---------- agent ----------
SYSTEM = ("You are a careful workspace assistant. Use tools to inspect files and run "
"commands. Verify your work (e.g. run tests) before claiming success. "
"If a tool returns an error, read it and adapt. Be concise.")
@dataclass
class Agent:
llm: LLMClient
tools: dict[str, Tool]
policy: Policy = field(default_factory=Policy)
max_steps: int = 20
max_seconds: float = 300
keep_full: int = 6 # most recent tool results kept verbatim
trace_path: str | None = None
def _trace(self, **ev):
if self.trace_path:
with open(self.trace_path, "a") as f:
f.write(json.dumps({"t": time.time(), **ev}, default=str) + "\n")
def _trim(self, messages: list[dict]) -> list[dict]:
idx = [i for i, m in enumerate(messages) if m["role"] == "tool"]
old = set(idx[:-self.keep_full])
return [({**m, "content": "[old output elided; re-run the tool if needed]"}
if i in old and len(m["content"]) > 300 else m)
for i, m in enumerate(messages)]
def _execute(self, call: dict) -> str:
tool = self.tools.get(call["name"])
if tool is None:
return f"ERROR: unknown tool {call['name']}. Available: {list(self.tools)}"
args = call["arguments"]
if isinstance(args, str):
try:
args = json.loads(args)
except json.JSONDecodeError as e:
return f"ERROR: arguments are not valid JSON: {e}"
verdict = self.policy.check(tool, args)
if verdict:
return verdict
try:
return str(tool.fn(**args))[:8000]
except Exception as e: # errors become observations
return f"ERROR: {type(e).__name__}: {e}"
def run(self, task: str) -> str:
messages = [{"role": "system", "content": SYSTEM},
{"role": "user", "content": task}]
specs = [t.spec() for t in self.tools.values()]
start = time.time()
for step in range(self.max_steps):
if time.time() - start > self.max_seconds:
return "STOPPED: time budget exceeded."
reply = self.llm.chat(self._trim(messages), specs)
messages.append(reply)
self._trace(step=step, assistant=reply)
calls = reply.get("tool_calls") or []
if not calls:
return reply.get("content", "")
for c in calls:
c.setdefault("id", uuid.uuid4().hex[:8])
result = self._execute(c)
self._trace(step=step, tool=c["name"], args=c["arguments"], result=result)
messages.append({"role": "tool", "tool_call_id": c["id"],
"name": c["name"], "content": result})
return "STOPPED: step budget exceeded."

That is roughly 150 lines including the tool definitions. Every concern from the previous lectures appears: schemas and instructive errors (Tool use), output truncation and elision (Memory), path confinement and policy (Safety), budgets and tracing (Production).

Agents are non-deterministic, but the harness is deterministic code and should be unit tested with a fake LLMClient that replays a script. This catches bugs in tool execution, error handling and budgets without any API calls.

# test_agent.py (run with: pytest -q)
import pathlib
from agent import Agent, Policy, make_tools
class Scripted:
def __init__(self, replies): self.replies, self.seen = list(replies), []
def chat(self, messages, tools):
self.seen.append(messages)
return self.replies.pop(0)
def call(name, **args):
return {"role": "assistant", "content": "",
"tool_calls": [{"id": "c1", "name": name, "arguments": args}]}
def final(text): return {"role": "assistant", "content": text}
def test_write_then_read(tmp_path):
llm = Scripted([call("write_file", path="a.txt", content="hi"),
call("read_file", path="a.txt"), final("done")])
agent = Agent(llm, make_tools(tmp_path), Policy(confirm=lambda m: True))
assert agent.run("make a file") == "done"
assert (tmp_path / "a.txt").read_text() == "hi"
def test_path_escape_is_an_observation_not_a_crash(tmp_path):
llm = Scripted([call("read_file", path="../../etc/passwd"), final("ok")])
agent = Agent(llm, make_tools(tmp_path))
agent.run("x")
tool_msg = [m for m in llm.seen[-1] if m["role"] == "tool"][0]
assert "escapes workspace" in tool_msg["content"]
def test_destructive_command_denied(tmp_path):
llm = Scripted([call("run_shell", command="rm -rf /"), final("ok")])
agent = Agent(llm, make_tools(tmp_path), Policy(confirm=lambda m: True))
agent.run("x")
assert "DENIED" in [m for m in llm.seen[-1] if m["role"] == "tool"][0]["content"]
def test_step_budget(tmp_path):
llm = Scripted([call("list_dir")] * 5)
agent = Agent(llm, make_tools(tmp_path), max_steps=3)
assert agent.run("loop").startswith("STOPPED")

An adapter converts our dicts to a provider SDK’s schema. The shape below is typical for chat-completion style APIs; field names differ by provider, so treat it as a template and consult your SDK:

import json
class ProviderAdapter:
def __init__(self, client, model):
self.client, self.model = client, model
def chat(self, messages, tools):
wire_tools = [{"type": "function", "function": t} for t in tools]
wire_msgs = []
for m in messages:
if m["role"] == "assistant" and m.get("tool_calls"):
wire_msgs.append({"role": "assistant", "content": m.get("content") or None,
"tool_calls": [{"id": c["id"], "type": "function",
"function": {"name": c["name"],
"arguments": json.dumps(c["arguments"])}}
for c in m["tool_calls"]]})
elif m["role"] == "tool":
wire_msgs.append({"role": "tool", "tool_call_id": m["tool_call_id"],
"content": m["content"]})
else:
wire_msgs.append({"role": m["role"], "content": m["content"]})
resp = self.client.chat.completions.create(
model=self.model, messages=wire_msgs, tools=wire_tools)
msg = resp.choices[0].message
calls = [{"id": c.id, "name": c.function.name,
"arguments": json.loads(c.function.arguments)}
for c in (msg.tool_calls or [])]
return {"role": "assistant", "content": msg.content or "", "tool_calls": calls}

Anthropic-style APIs use content blocks (tool_use / tool_result) and a separate system parameter; the adapter’s job is the same translation. Keeping provider details in one class makes the agent portable and testable.

import pathlib
from agent import Agent, Policy, make_tools
def ask(msg: str) -> bool:
return input(f"{msg} [y/N] ").strip().lower() == "y"
agent = Agent(llm=ProviderAdapter(client, "your-model-name"),
tools=make_tools(pathlib.Path("./sandbox")),
policy=Policy(confirm=ask),
trace_path="trace.jsonl")
print(agent.run("Create fizzbuzz.py with a function fizzbuzz(n), add tests, and run them."))

Then inspect trace.jsonl (one JSON event per line) to see the trajectory.

  1. A todo tool that writes to a file and is re-injected into context; improves long tasks (Patterns).
  2. Compaction: when estimated tokens exceed a threshold, summarize with the model (Memory).
  3. Parallel tool calls via a thread pool.
  4. A subagent tool that instantiates a new Agent with a restricted toolset and returns its final answer (Multi-agent).
  5. Retry and backoff around llm.chat for transient API errors; count token usage from responses.
  6. Container execution: make run_shell call docker exec into a throwaway container.
  7. An evaluation harness: reuse the evaluate function from Evaluation with tasks that create temp workspaces and check file state.
  8. Prompt caching: keep the system prompt and tool specs byte-stable; do not mutate earlier messages except through deterministic elision (note _trim changes old tool messages, which invalidates caches from that point; a cache-aware version elides in batches).
  • fnmatch policy on shell strings is easily bypassed; production systems parse commands or avoid raw shell.
  • shell=True with the model’s string is dangerous without a sandbox; run inside a container.
  • Token accounting is absent; budgets are steps and seconds only.
  • No streaming, no persistence of conversation state across processes.
  • Path confinement is checked on resolved paths but shell commands can escape it; that is what the container is for.
  • A useful agent is a short loop plus well-designed tools, policy, budgets and tracing; no framework required.
  • Isolate provider-specific wire formats behind a one-method LLMClient interface; the same agent then runs against real and fake models.
  • Test the deterministic harness with scripted models; evaluate the whole system separately on real tasks.
  • Errors, denials and truncation notices are returned as observations so the model can adapt.
  • Treat confinement in code as a convenience, and the container as the security boundary.
  1. Hands-on. Add a todo tool (set_todos(items), get_todos()) and make Agent.run append the current todo list as a system reminder every 5 steps.
  2. Hands-on. Add token accounting: have chat return a usage dict and stop when cumulative tokens exceed a budget.
  3. Hands-on. Write a scripted test where the model first calls a nonexistent tool, observes the error listing available tools, and then succeeds.
  4. Design. Implement a subagent tool and decide which tools it should not receive. Explain why.
Solutions (4)

Exclude write_file and run_shell for a research/read-only subagent (least privilege) and exclude subagent itself unless you cap recursion depth, to prevent unbounded spawning.

  • Anthropic, “Building effective agents” (2024), whose advice to start with direct API calls this lecture follows.
  • Yao et al., “ReAct” (2023); Yang et al., “SWE-agent” (2024) for interface design.
  • Your provider’s function-calling documentation for the exact wire format.
  • Simon Willison, “Designing agentic loops” (2025), practical notes on harness design.