Skip to content

Tool Use, Function Calling and MCP

What you’ll learn. The mechanics of function calling (schemas, call/result message flow, parallel calls), principles for designing tools that models use correctly, error handling, and the Model Context Protocol (MCP) architecture and its trade-offs. For model-side training of tool use see Agents and tool use.

A language model only emits tokens. “Tool use” is a protocol layered on top:

  1. The developer sends the model a list of tool definitions: name, natural-language description, and a JSON Schema for arguments.
  2. The model, instead of (or before) answering, emits a structured tool call: tool name plus JSON arguments. Providers train models to do this reliably; some also support constrained decoding so the JSON is guaranteed schema-valid.
  3. Your harness (not the model) parses the call, runs the real function, and returns the output as a tool result message tied to the call id.
  4. The model continues, seeing the result.
user: What's the weather in Oslo?
assistant: tool_call(id=c1, name="get_weather", args={"city":"Oslo"})
tool: (id=c1) {"temp_c": 3, "sky": "overcast"}
assistant: It's 3 C and overcast in Oslo.

The model never executes anything. This is the root of both flexibility and security: the harness is the trust boundary (Safety and security).

get_weather = {
"name": "get_weather",
"description": "Current weather for a city. Use for questions about "
"present conditions only, not forecasts.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "City name, e.g. 'Oslo'"},
"units": {"type": "string", "enum": ["c", "f"], "default": "c"},
},
"required": ["city"],
},
}

Provider formats differ slightly in key names, but all reduce to this shape. The description and parameter docs are prompt text: the model chooses and fills tools based on them alone.

Models may emit several independent calls in one turn (look up three cities). The harness should execute them concurrently and return all results before the next model turn. Dependent calls (search, then open the top result) are inherently sequential, one model turn each, which is where latency accumulates.

Tools are an interface for a non-human user with limited context and no ability to ask questions. Principles that consistently matter:

  • Few, purposeful tools. Overlapping tools cause misrouting. Merge list_users, get_user, search_user if one find_users(query) serves. Selection accuracy degrades as the menu grows past a few dozen tools; above that, use retrieval over tool descriptions or sub-agents.
  • Describe when to use it and when not to. Say what distinguishes it from siblings.
  • Return what the model needs, not what the API returns. Strip internal ids, base64 blobs and pagination noise. Return human-meaningful names. Truncate large outputs with an explicit note (“showing 20 of 340; refine your query”).
  • Make errors instructive. "No file 'src/a.py'. Did you mean 'src/app.py'? Existing files in src/: ..." lets the model self-correct; a stack trace does not.
  • Prefer absolute, unambiguous parameters. Absolute paths, ISO dates, enums over free text. Weaker models in particular fail on relative paths after a cd.
  • Idempotency and dry-run. Make write tools safe to retry; offer dry_run for destructive operations.
  • Token efficiency. Output size counts against the context (Memory and context engineering); add response_format: concise|detailed parameters when useful.
  • Poka-yoke. Change the tool so mistakes are hard. A well-known example: requiring absolute file paths removed a class of editing errors.
import json, concurrent.futures as cf
def execute(tool_calls, registry, timeout=30):
def run(call):
spec = registry.get(call["name"])
if spec is None:
return call["id"], f"ERROR: unknown tool {call['name']}"
try:
args = call["arguments"] if isinstance(call["arguments"], dict) \
else json.loads(call["arguments"])
return call["id"], str(spec["fn"](**args))[:20_000]
except TypeError as e:
return call["id"], f"ERROR: bad arguments: {e}"
except Exception as e:
return call["id"], f"ERROR: {type(e).__name__}: {e}"
with cf.ThreadPoolExecutor() as ex:
futs = [ex.submit(run, c) for c in tool_calls]
return [f.result(timeout=timeout) for f in futs]

Always: validate arguments, cap output size, enforce timeouts, return errors as observations, and log every call and result.

With hundreds of tools you cannot put every schema in context. Options: (1) embed tool descriptions and retrieve the top-kk per task; (2) hierarchical routing to a sub-agent owning a tool group; (3) let the agent search for tools via a meta-tool and load definitions on demand. Each costs an extra step but keeps the prompt small and selection accurate.

Instead of many narrow tools, give the model a sandboxed code interpreter. Code composes operations, loops, and filters data outside the context window, so a million-row table need never enter the prompt. This idea (used in “CodeAct”-style agents) trades interface simplicity for sandboxing burden; see Code agents.

Before MCP, each application hand-integrated each tool: NN apps times MM services means N×MN \times M integrations. MCP (introduced by Anthropic in late 2024, later adopted broadly across vendors) defines an open protocol so that N+MN + M suffice, analogous to the Language Server Protocol for editors.

Architecture.

Host app (IDE, chat client, agent)
|-- MCP client 1 <--JSON-RPC--> MCP server A (e.g. GitHub)
|-- MCP client 2 <--JSON-RPC--> MCP server B (e.g. local filesystem)
  • Host embeds the model and manages user consent; it spawns one client per server.
  • Transport: local servers over stdio; remote servers over HTTP-based streaming transports.
  • Messages are JSON-RPC 2.0.

Server primitives:

  • Tools: model-invoked actions with JSON Schema inputs.
  • Resources: read-only data addressed by URI that the host can attach as context.
  • Prompts: reusable parameterized prompt templates the user can invoke.

Client-offered features include sampling (the server asks the host to run an LLM completion) and elicitation (the server asks the user for input). A session begins with capability negotiation, then tools/list discovers tools dynamically, and tools/call invokes them. Remote servers typically use OAuth-based authorization.

A minimal MCP server using the official Python SDK’s high-level interface looks like this (APIs evolve; consult the current SDK docs):

from mcp.server.fastmcp import FastMCP
mcp = FastMCP("notes")
NOTES: dict[str, str] = {}
@mcp.tool()
def save_note(title: str, body: str) -> str:
"""Save a note under a title, overwriting any existing note."""
NOTES[title] = body
return f"saved '{title}'"
@mcp.tool()
def search_notes(query: str) -> list[str]:
"""Return titles of notes whose text contains the query."""
return [t for t, b in NOTES.items() if query.lower() in (t + b).lower()]
if __name__ == "__main__":
mcp.run() # stdio transport by default

Any MCP-capable host can now list and call these tools without custom glue.

  • Pros: ecosystem reuse, dynamic discovery, separation of tool implementer from agent builder.
  • Context cost: loading many servers’ tool schemas can consume tens of thousands of tokens before the task starts. Mitigate with tool search, filtering, or code-execution wrappers that expose servers as importable APIs.
  • Security: a server is third-party code plus third-party prompt text (descriptions and results). Malicious or compromised servers can inject instructions, and servers combined in one session can exfiltrate across each other (“confused deputy”). Review servers, scope credentials, show approval prompts. See Safety and security.
  • Churn: the spec has been revised repeatedly (transports, auth). Pin versions.

MCP standardizes how tools are exposed, not how good they are. A thin wrapper over a REST API with 80 endpoints is usually a poor agent tool; design for tasks, not endpoints.

  • Tool calling is a protocol: schema in, structured call out, harness executes, result returned. The model never runs code itself.
  • Tool descriptions and outputs are prompts; design them for a model with limited context and no clarifying questions.
  • Instructive errors, bounded outputs, timeouts and logging belong in the execution layer.
  • MCP turns N×MN\times M integrations into N+MN+M via tools, resources and prompts over JSON-RPC, at the cost of context bloat and a larger attack surface.
  • Fewer, task-oriented tools beat API-mirroring tool sprawl.
  1. Conceptual. Why is it a mistake to expose a run_sql(query) tool whose description says only “Run SQL”? List three things to add.
  2. Hands-on. Write a read_file(path, start=1, limit=200) tool that returns line-numbered text, truncates with a helpful message, and returns suggestions on a missing path (use difflib.get_close_matches).
  3. Hands-on. Wrap the execute function above so that tools marked "needs_approval": True pause for a y/n prompt.
  4. Design. An MCP server exposes 60 tools and costs 18k tokens of schema. Propose two ways to reduce this and the risks of each.
Solutions (1, 4)
  1. Schema description (tables and columns), dialect and constraints (read-only, row limit, timeouts), and guidance on when to use it versus other tools; also return column names and truncate rows with a note.
  2. (a) Tool search: load only relevant tool schemas on demand; risk is a missed retrieval. (b) Code-mode wrapper: expose tools as functions in a sandbox and let the model write code; risk is sandbox complexity and less transparent per-call approval.
  • Schick et al., “Toolformer” (2023); Patil et al., “Gorilla” (2023).
  • Qin et al., “ToolLLM” (2023).
  • Anthropic, “Writing effective tools for agents” (2025) and the MCP specification (modelcontextprotocol.io).
  • Wang et al., “Executable Code Actions Elicit Better LLM Agents” (CodeAct, 2024).
  • Agents and tool use for training-side coverage.