Debugging an Agent That 'Hallucinates' a Tool Call: A Field Guide

“The model just hallucinated the tool call” is usually where debugging stops, and it’s almost always wrong as a root cause. A model requesting a tool that doesn’t exist, or passing arguments that don’t match reality, nearly always traces to something concrete in the surrounding system — not a random model failure. Here’s the order to check things in, cheapest and most likely first.

1. The tool’s description is ambiguous or the schema is loose

The model chooses arguments based entirely on the schema and description you gave it. A vague description invites a bad guess. (If the difference between the schema, the model’s decision, and your execution loop is fuzzy, start with MCP vs. tool calling vs. function calling.)

# invites hallucinated arguments — "identifier" could mean anything
{"name": "get_user", "description": "Get a user", "input_schema": {
    "properties": {"identifier": {"type": "string"}}
}}

# forces a correct guess — the model can't invent a format it wasn't told about
{"name": "get_user", "description": "Get a user by their numeric account ID (not username or email)",
 "input_schema": {"properties": {"account_id": {"type": "integer"}}}}

Check first: read your tool schema as if you’d never seen the codebase. If the argument’s meaning or format isn’t unambiguous from the schema alone, that’s the bug, not the model.

2. The tool result format doesn’t match what earlier turns implied

If a tool returned data in one shape early in the conversation and a different shape later — say, a paginated result the first time and a flat list the second — the model will confidently apply patterns from the first result to the second, producing calls that look hallucinated but are really just consistent with stale information it has no way to know is stale.

Check second: diff the actual tool outputs across the conversation. Inconsistent result shapes from the same tool are a common, easy-to-miss cause.

3. The model is working from a stale or truncated context

If context got compacted or truncated (see why your agent forgets things mid-task), the model may be calling a tool based on a fact from an early turn that’s no longer accurate, or inventing an argument to fill a gap where real information used to be.

Check third: was there a compaction/summarization event between when the correct information was available and when the bad call happened? If yes, the summarizer — not the model’s tool-calling — dropped something load-bearing.

4. Too many similar tools are available at once

Models genuinely do call the wrong tool, or blend arguments from two tools, when several available tools have overlapping names or purposes — search_users and find_user and lookup_account all present in the same tool list is a real recipe for cross-contamination, independent of prompt quality.

Check fourth: count how many tools are in scope for this call. If it’s more than roughly 15–20, or if any two have similar names/purposes, that’s a real, documented failure mode — not a hallucination, a disambiguation problem you created.

5. It’s actually a sampling issue, not a reasoning issue

After the first four are ruled out, what’s left is genuinely the model’s decision process — and even here, “hallucination” is usually imprecise. Check temperature: a high-temperature agent making tool-selection decisions will occasionally pick a less-likely option even with a perfect schema and perfect context. Tool selection is exactly the kind of decision that benefits from lower temperature than creative text generation does.

response = client.messages.create(
    model="claude-sonnet-5",
    tools=tools,
    temperature=0,  # deterministic tool selection, not creative variance
    messages=messages,
)

The actual field guide, condensed

Before calling it a hallucination, check in this order: (1) is the schema/description actually unambiguous, (2) did the tool’s result shape change without warning, (3) was there a compaction event that dropped the real information, (4) are there too many similar tools in scope, (5) is temperature doing this to you. In practice, almost every “the model just made something up” report resolves at step 1 or 4 — the system gave the model an ambiguous or overcrowded decision to make, and it made a reasonable one anyway.