Bad Tool Schemas Turn Smart AI Agents Into Production Bug Factories
Design tool names, argument schemas, error responses, and authorization checks so model tool use becomes predictable and testable.
Models cannot compensate for a confusing contract
Tool calling is often presented as a model capability: the assistant chooses a function, fills arguments, and continues. In production, the tool contract determines whether that choice is safe. Ambiguous names, optional fields with hidden meanings, permissive schemas, and vague error messages create room for accidental actions. A stronger model may make fewer mistakes, but it cannot infer an authorization rule the API never states.
The recent shift toward agent platforms makes tool design an increasingly central engineering task. A single agent can call search, databases, code runners, calendars, and business APIs. Each tool is an opportunity to expose data or mutate state. Treat the schema as a public interface between probabilistic planning and deterministic software.
Start with one clear action per tool. get_invoice should read an invoice. create_refund_request should prepare a request. Avoid a generic manage_customer operation that can search, edit, delete, and message based on a free-form instruction. Narrow tools make it easier to restrict permissions, test arguments, and explain actions to users.
Design schemas for unambiguous intent
Use explicit field names and enums for bounded choices. Distinguish an account identifier from a display name. Make currency and units explicit. Reject unknown fields when your schema system permits it. Avoid silently interpreting a missing value as “all,” “latest,” or “default account.” If the operation is dangerous without a field, make it required and validate it again in server code.
Prefer structured values over natural-language arguments. A date range should have start and end fields in a documented format. A recipient should be a verified internal identifier, not a model-generated email string. If users can choose from a constrained set, represent those choices directly.
Separate the proposal from execution. A tool can return a preview containing the affected records and expected changes. Another operation commits the preview using a one-time token or idempotency key. This lets the interface display a meaningful approval step and ensures the approved payload is the payload executed.
Make errors useful without leaking secrets
Tool errors should tell the agent what can be corrected. “Invalid date format; expected YYYY-MM-DD” is better than “error.” “The caller lacks permission for this account” is safer than revealing whether an unrelated private account exists. Use stable error codes, classify transient failures, and include a retryable flag where appropriate.
Do not return stack traces, secrets, hidden records, or internal prompts to the model. Keep diagnostic details in protected logs and return a safe summary. A model may include tool output in a user response, so every tool result should be reviewed as potentially user-visible.
Bound result size. Paginate large responses, return only fields needed for the next decision, and include a stable cursor. Tool output can consume the context budget or bury the one exception that matters. Include provenance and timestamps so the agent can tell whether data is current.
Enforce authorization outside the model
Every call must resolve the current user identity and check access to the specific resource. Do not accept a user ID supplied by the model as proof of identity. Enforce tenant isolation in the data layer and validate that a record belongs to the authorized organization. Re-check permission at execution time, especially after a human approval delay.
Use read-only credentials for read-only tools and short-lived tokens for elevated operations. Record the tool name, actor, resource, result, and approval reference. Make mutations idempotent where possible. Define whether a retry is safe and implement deduplication server-side.
Test the boundary, not only the schema
Write contract tests for missing fields, unexpected fields, invalid enums, long strings, Unicode edge cases, injection-like content, unauthorized identifiers, stale approval tokens, duplicate requests, timeouts, and partial success. Test that an error does not accidentally cause the agent to choose a more powerful alternate tool. Include prompts that ask the model to bypass the intended workflow.
Measure tool-call accuracy and task completion separately. A model can choose the correct tool but provide a wrong customer identifier. Capture whether the tool succeeded, whether the result was verified, and whether a human had to intervene. Keep representative failures in a regression set.
Finally, document each tool's effect in plain language for the product team and reviewers. The safest schema is not the most elaborate one. It is the smallest clear contract that gives the agent enough capability to complete a task while leaving authorization, validation, and irreversible decisions under deterministic control.