Project: Build an Agent from Scratch in Python
What you’ll learn. By writing a small but complete agent you will connect everything so far: the loop, tool registry with schemas, error handling, a permission gate, context trimming, budgets, tracing, and tests with a scripted fake model. The result is a file-and-shell “workspace assistant” you can point at any provider by writing a 15-line adapter.
Design
Section titled “Design”We deliberately avoid frameworks. Components:
LLMClient (protocol) <-- adapter for any provider | Agent.run(task) -----> loop: call model -> execute tool calls -> append results | |-- Policy: allow / ask / deny | |-- budgets: steps, tokens (approx), wall-clock | |-- trim: elide old tool output v Tracer (JSONL) + final answerProvider-neutral message format. We use our own small dicts:
{"role": "user" | "assistant" | "tool", "content": str, ...}- Assistant messages may carry
"tool_calls": [{"id", "name", "arguments": dict}]. - Tool messages carry
"tool_call_id"and"name".
An adapter translates this to a provider’s wire format and back. The model interface is one method:
class LLMClient(Protocol): def chat(self, messages: list[dict], tools: list[dict]) -> dict: """Return an assistant message: {'role':'assistant','content':str,'tool_calls':[...]}"""The complete code
Section titled “The complete code”Save as agent.py. It uses only the standard library.
from __future__ import annotationsimport fnmatch, json, pathlib, subprocess, time, uuidfrom dataclasses import dataclass, fieldfrom typing import Callable, Protocol
class LLMClient(Protocol): def chat(self, messages: list[dict], tools: list[dict]) -> dict: ...
# ---------- tools ----------@dataclassclass Tool: name: str description: str parameters: dict # JSON Schema fn: Callable[..., str] risk: str = "read" # read | write | exec
def spec(self) -> dict: return {"name": self.name, "description": self.description, "parameters": self.parameters}
def make_tools(root: pathlib.Path) -> dict[str, Tool]: root = root.resolve()
def safe(path: str) -> pathlib.Path: p = (root / path).resolve() if root != p and root not in p.parents: raise PermissionError(f"path escapes workspace: {path}") return p
def read_file(path: str, start: int = 1, limit: int = 200) -> str: p = safe(path) if not p.is_file(): names = sorted(x.name for x in p.parent.glob("*"))[:30] if p.parent.exists() else [] return f"ERROR: no such file {path}. Nearby: {names}" lines = p.read_text().splitlines() chunk = lines[start - 1:start - 1 + limit] out = "\n".join(f"{i}: {l}" for i, l in enumerate(chunk, start)) if start - 1 + limit < len(lines): out += f"\n[truncated: {len(lines)} lines total; use start/limit]" return out or "[empty]"
def write_file(path: str, content: str) -> str: p = safe(path) p.parent.mkdir(parents=True, exist_ok=True) p.write_text(content) return f"wrote {len(content)} chars to {path}"
def list_dir(path: str = ".") -> str: return "\n".join(sorted(x.name + ("/" if x.is_dir() else "") for x in safe(path).iterdir())) or "[empty]"
def run_shell(command: str, timeout: int = 30) -> str: r = subprocess.run(command, shell=True, cwd=root, capture_output=True, text=True, timeout=timeout) out = (r.stdout + r.stderr)[-4000:] return f"exit={r.returncode}\n{out}"
obj = lambda props, req: {"type": "object", "properties": props, "required": req} s, i = {"type": "string"}, {"type": "integer"} tools = [ Tool("read_file", "Read a text file with line numbers.", obj({"path": s, "start": i, "limit": i}, ["path"]), read_file), Tool("write_file", "Create or overwrite a file in the workspace.", obj({"path": s, "content": s}, ["path", "content"]), write_file, "write"), Tool("list_dir", "List a directory in the workspace.", obj({"path": s}, []), list_dir), Tool("run_shell", "Run a shell command in the workspace; returns exit code and output.", obj({"command": s, "timeout": i}, ["command"]), run_shell, "exec"), ] return {t.name: t for t in tools}
# ---------- policy ----------@dataclassclass Policy: allow: list[str] = field(default_factory=lambda: ["read_file:*", "list_dir:*", "run_shell:pytest*"]) deny: list[str] = field(default_factory=lambda: ["run_shell:*rm -rf*", "run_shell:*sudo*"]) confirm: Callable[[str], bool] = lambda msg: False # default: refuse to ask
def check(self, tool: Tool, args: dict) -> str | None: key = f"{tool.name}:{args.get('command') or args.get('path') or ''}" if any(fnmatch.fnmatch(key, p) for p in self.deny): return "DENIED by policy" if any(fnmatch.fnmatch(key, p) for p in self.allow): return None return None if self.confirm(f"Allow {tool.name} {args}?") else "DENIED: not approved"
# ---------- agent ----------SYSTEM = ("You are a careful workspace assistant. Use tools to inspect files and run " "commands. Verify your work (e.g. run tests) before claiming success. " "If a tool returns an error, read it and adapt. Be concise.")
@dataclassclass Agent: llm: LLMClient tools: dict[str, Tool] policy: Policy = field(default_factory=Policy) max_steps: int = 20 max_seconds: float = 300 keep_full: int = 6 # most recent tool results kept verbatim trace_path: str | None = None
def _trace(self, **ev): if self.trace_path: with open(self.trace_path, "a") as f: f.write(json.dumps({"t": time.time(), **ev}, default=str) + "\n")
def _trim(self, messages: list[dict]) -> list[dict]: idx = [i for i, m in enumerate(messages) if m["role"] == "tool"] old = set(idx[:-self.keep_full]) return [({**m, "content": "[old output elided; re-run the tool if needed]"} if i in old and len(m["content"]) > 300 else m) for i, m in enumerate(messages)]
def _execute(self, call: dict) -> str: tool = self.tools.get(call["name"]) if tool is None: return f"ERROR: unknown tool {call['name']}. Available: {list(self.tools)}" args = call["arguments"] if isinstance(args, str): try: args = json.loads(args) except json.JSONDecodeError as e: return f"ERROR: arguments are not valid JSON: {e}" verdict = self.policy.check(tool, args) if verdict: return verdict try: return str(tool.fn(**args))[:8000] except Exception as e: # errors become observations return f"ERROR: {type(e).__name__}: {e}"
def run(self, task: str) -> str: messages = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": task}] specs = [t.spec() for t in self.tools.values()] start = time.time() for step in range(self.max_steps): if time.time() - start > self.max_seconds: return "STOPPED: time budget exceeded." reply = self.llm.chat(self._trim(messages), specs) messages.append(reply) self._trace(step=step, assistant=reply) calls = reply.get("tool_calls") or [] if not calls: return reply.get("content", "") for c in calls: c.setdefault("id", uuid.uuid4().hex[:8]) result = self._execute(c) self._trace(step=step, tool=c["name"], args=c["arguments"], result=result) messages.append({"role": "tool", "tool_call_id": c["id"], "name": c["name"], "content": result}) return "STOPPED: step budget exceeded."That is roughly 150 lines including the tool definitions. Every concern from the previous lectures appears: schemas and instructive errors (Tool use), output truncation and elision (Memory), path confinement and policy (Safety), budgets and tracing (Production).
Testing with a scripted fake model
Section titled “Testing with a scripted fake model”Agents are non-deterministic, but the harness is deterministic code and should be unit tested with a fake LLMClient that replays a script. This catches bugs in tool execution, error handling and budgets without any API calls.
# test_agent.py (run with: pytest -q)import pathlibfrom agent import Agent, Policy, make_tools
class Scripted: def __init__(self, replies): self.replies, self.seen = list(replies), [] def chat(self, messages, tools): self.seen.append(messages) return self.replies.pop(0)
def call(name, **args): return {"role": "assistant", "content": "", "tool_calls": [{"id": "c1", "name": name, "arguments": args}]}
def final(text): return {"role": "assistant", "content": text}
def test_write_then_read(tmp_path): llm = Scripted([call("write_file", path="a.txt", content="hi"), call("read_file", path="a.txt"), final("done")]) agent = Agent(llm, make_tools(tmp_path), Policy(confirm=lambda m: True)) assert agent.run("make a file") == "done" assert (tmp_path / "a.txt").read_text() == "hi"
def test_path_escape_is_an_observation_not_a_crash(tmp_path): llm = Scripted([call("read_file", path="../../etc/passwd"), final("ok")]) agent = Agent(llm, make_tools(tmp_path)) agent.run("x") tool_msg = [m for m in llm.seen[-1] if m["role"] == "tool"][0] assert "escapes workspace" in tool_msg["content"]
def test_destructive_command_denied(tmp_path): llm = Scripted([call("run_shell", command="rm -rf /"), final("ok")]) agent = Agent(llm, make_tools(tmp_path), Policy(confirm=lambda m: True)) agent.run("x") assert "DENIED" in [m for m in llm.seen[-1] if m["role"] == "tool"][0]["content"]
def test_step_budget(tmp_path): llm = Scripted([call("list_dir")] * 5) agent = Agent(llm, make_tools(tmp_path), max_steps=3) assert agent.run("loop").startswith("STOPPED")Writing a provider adapter
Section titled “Writing a provider adapter”An adapter converts our dicts to a provider SDK’s schema. The shape below is typical for chat-completion style APIs; field names differ by provider, so treat it as a template and consult your SDK:
import json
class ProviderAdapter: def __init__(self, client, model): self.client, self.model = client, model
def chat(self, messages, tools): wire_tools = [{"type": "function", "function": t} for t in tools] wire_msgs = [] for m in messages: if m["role"] == "assistant" and m.get("tool_calls"): wire_msgs.append({"role": "assistant", "content": m.get("content") or None, "tool_calls": [{"id": c["id"], "type": "function", "function": {"name": c["name"], "arguments": json.dumps(c["arguments"])}} for c in m["tool_calls"]]}) elif m["role"] == "tool": wire_msgs.append({"role": "tool", "tool_call_id": m["tool_call_id"], "content": m["content"]}) else: wire_msgs.append({"role": m["role"], "content": m["content"]}) resp = self.client.chat.completions.create( model=self.model, messages=wire_msgs, tools=wire_tools) msg = resp.choices[0].message calls = [{"id": c.id, "name": c.function.name, "arguments": json.loads(c.function.arguments)} for c in (msg.tool_calls or [])] return {"role": "assistant", "content": msg.content or "", "tool_calls": calls}Anthropic-style APIs use content blocks (tool_use / tool_result) and a separate system parameter; the adapter’s job is the same translation. Keeping provider details in one class makes the agent portable and testable.
Running it
Section titled “Running it”import pathlibfrom agent import Agent, Policy, make_tools
def ask(msg: str) -> bool: return input(f"{msg} [y/N] ").strip().lower() == "y"
agent = Agent(llm=ProviderAdapter(client, "your-model-name"), tools=make_tools(pathlib.Path("./sandbox")), policy=Policy(confirm=ask), trace_path="trace.jsonl")print(agent.run("Create fizzbuzz.py with a function fizzbuzz(n), add tests, and run them."))Then inspect trace.jsonl (one JSON event per line) to see the trajectory.
Extensions (in order of value)
Section titled “Extensions (in order of value)”- A todo tool that writes to a file and is re-injected into context; improves long tasks (Patterns).
- Compaction: when estimated tokens exceed a threshold, summarize with the model (Memory).
- Parallel tool calls via a thread pool.
- A
subagenttool that instantiates a newAgentwith a restricted toolset and returns its final answer (Multi-agent). - Retry and backoff around
llm.chatfor transient API errors; count token usage from responses. - Container execution: make
run_shellcalldocker execinto a throwaway container. - An evaluation harness: reuse the
evaluatefunction from Evaluation with tasks that create temp workspaces and check file state. - Prompt caching: keep the system prompt and tool specs byte-stable; do not mutate earlier messages except through deterministic elision (note
_trimchanges old tool messages, which invalidates caches from that point; a cache-aware version elides in batches).
Limitations of this implementation
Section titled “Limitations of this implementation”fnmatchpolicy on shell strings is easily bypassed; production systems parse commands or avoid raw shell.shell=Truewith the model’s string is dangerous without a sandbox; run inside a container.- Token accounting is absent; budgets are steps and seconds only.
- No streaming, no persistence of conversation state across processes.
- Path confinement is checked on resolved paths but shell commands can escape it; that is what the container is for.
Key takeaways
Section titled “Key takeaways”- A useful agent is a short loop plus well-designed tools, policy, budgets and tracing; no framework required.
- Isolate provider-specific wire formats behind a one-method
LLMClientinterface; the same agent then runs against real and fake models. - Test the deterministic harness with scripted models; evaluate the whole system separately on real tasks.
- Errors, denials and truncation notices are returned as observations so the model can adapt.
- Treat confinement in code as a convenience, and the container as the security boundary.
Exercises
Section titled “Exercises”- Hands-on. Add a
todotool (set_todos(items),get_todos()) and makeAgent.runappend the current todo list as a system reminder every 5 steps. - Hands-on. Add token accounting: have
chatreturn ausagedict and stop when cumulative tokens exceed a budget. - Hands-on. Write a scripted test where the model first calls a nonexistent tool, observes the error listing available tools, and then succeeds.
- Design. Implement a
subagenttool and decide which tools it should not receive. Explain why.
Solutions (4)
Exclude write_file and run_shell for a research/read-only subagent (least privilege) and exclude subagent itself unless you cap recursion depth, to prevent unbounded spawning.
Further reading
Section titled “Further reading”- Anthropic, “Building effective agents” (2024), whose advice to start with direct API calls this lecture follows.
- Yao et al., “ReAct” (2023); Yang et al., “SWE-agent” (2024) for interface design.
- Your provider’s function-calling documentation for the exact wire format.
- Simon Willison, “Designing agentic loops” (2025), practical notes on harness design.