AI tool calling lets a model request an operation through a defined interface: search a catalog, retrieve an account, create a ticket, draft a refund, or invoke code. It turns language output into potential system activity, so the engineering problem is larger than selecting a function name. A production design must establish who asked, what the user is allowed to do, which arguments are valid, what the operation costs, whether it is reversible, and how the final effect is verified.
This AI tool calling guide focuses on cost, security, and scaling. Read it alongside Edilec's tool calling implementation guide, LLM observability checklist, and model evaluation release guide. The core principle is simple: the model may propose an action, but trusted software defines and enforces its authority.
Key takeaways
- Expose narrow, task-oriented tools instead of raw databases, shells, browsers, or generic HTTP clients.
- Authorize every call using trusted identity and current resource context outside model-generated text.
- Validate typed arguments, enforce idempotency, and separate preview, approval, execution, and verification.
- Budget model turns, tool calls, records, bytes, latency, and monetary exposure per request and tenant.
- Scale from read-only retrieval to reversible writes only after representative evaluation and incident rehearsal.
Model the full cost of a tool-enabled request
Token price is only one term. A tool-enabled request can include routing, multiple model turns, retrieval queries, vector or database reads, third-party API fees, browser or code execution, retries, human review, logging, and support. Measure cost at the workflow level, not per model call. The denominator should be a useful completed outcome, such as a correctly created service case, not a tool invocation. A cheaper model that loops or creates more review can be more expensive overall.
| Cost component | Measure | Control |
|---|---|---|
| Model inference | Input, cached, output, and reasoning units by step | Route by task; cap output and turns |
| Tool execution | Calls, records, bytes, compute time, and vendor fees | Batch safe reads; paginate and limit scope |
| Retries and loops | Repeated model and tool steps per request | Hard iteration, elapsed-time, and duplicate-call limits |
| Human review | Minutes and queue age by risk band | Improve evidence; do not hide review labor |
| Failure recovery | Reconciliation, reversal, support, and incident effort | Prefer idempotent and reversible operations |
| Platform overhead | Evaluation, logging, security, storage, and on-call cost | Allocate by workflow and tenant where useful |
Create a request budget before execution. It can specify maximum model turns, total tool calls, calls per tool, records read, bytes returned, wall-clock time, concurrent tasks, and financial amount affected. When a limit is reached, return a useful partial result or route to a person instead of silently extending the loop. Cache stable, permission-safe reads, but include tenant, user, scope, source version, and freshness in the cache design. Never let cross-user caching bypass authorization.
Design tools as constrained contracts
A useful tool name expresses a business operation such as getopeninvoices or preparerefund, not runquery. Define a strict schema, descriptions with domain meaning, bounds, enums, required fields, and explicit units. Reject unknown properties. Resolve identity, tenant, and permissions from trusted session state rather than accepting them as ordinary model arguments. Return structured results with stable error codes and minimal fields; verbose internal errors can confuse the model and expose sensitive implementation detail.
| Tool design | Read example | Write example | Required safeguard |
|---|---|---|---|
| Purpose | Retrieve active orders for the current customer | Prepare cancellation for one eligible order | One bounded business task |
| Authority | Current user may view this account | Current user may request, but not self-approve, a refund | Server-side policy check |
| Arguments | Order status and date window | Order ID, reason code, expected version | Typed validation and resource limits |
| Side effect | None beyond logged access | Creates pending command, then executes after approval | Preview and confirmation |
| Repeat behavior | Same read may return newer version | Same idempotency key returns original result | Explicit idempotency semantics |
| Result | Limited records with source timestamps | Command ID, state, effect, and receipt | Verification against system of record |
Separate planning from acting. The model can gather evidence and produce a proposed command. Trusted application code checks policy and presents the material consequence to an authorized user. A deterministic executor performs the command with an idempotency key and expected resource version. A verifier reads the system of record and records the actual effect. This pattern adds steps, but it prevents a fluent explanation from being mistaken for successful execution.
Defend the complete tool path
Threats can arrive through the user request, retrieved documents, tool descriptions, tool output, connected content, or compromised integrations. Treat all of them as untrusted data. The current OWASP agentic application guide addresses concrete controls for systems that can act, and the OWASP LLM application project covers risks including prompt injection, insecure output handling, excessive agency, and sensitive information disclosure. Controls must be enforced by the application boundary, not stated only in a prompt.
- Use least-privilege service credentials and narrow resource scopes per tool; avoid one agent credential with broad access.
- Apply input validation, output encoding, destination allowlists, network egress restrictions, and sandboxing where code or files are involved.
- Keep secrets out of prompts and tool results; use a broker that injects credentials only at execution.
- Require stronger confirmation for irreversible, high-value, external-communication, access-control, or regulated actions.
- Bind approvals to an immutable action preview so changed arguments require a new approval.
- Log decision evidence, policy result, tool version, arguments with protected fields redacted, execution receipt, and final state.
The NCSC secure AI development guidelines organize security across design, development, deployment, and operation. Apply that lifecycle to every tool addition. A seemingly harmless read tool can increase data exposure; a schema change can widen scope; a vendor integration can change retention. Review tools as privileged product features with owners, threat models, tests, and retirement plans.
Evaluate decisions, failures, and stopping behavior
Build a test set from real workflow cases. Score whether the system selected the right tool, supplied valid arguments, respected authorization, used the right sequence, stopped when evidence was missing, handled tool errors, avoided duplicate effects, and produced an accurate user explanation. Include malicious instructions inside retrieved content, stale resource versions, ambiguous accounts, timeouts, partial responses, rate limits, and approval rejection. End-to-end outcome is more important than exact match to one preferred reasoning trace.
| Evaluation slice | Pass condition | Release response |
|---|---|---|
| Routine completion | Correct outcome within budget and without unnecessary calls | Optimize only after correctness |
| Ambiguous intent | Ask a focused question or create a non-executing preview | Do not guess consequential arguments |
| Unauthorized request | No restricted data or action; useful explanation | Investigate any policy bypass |
| Injected tool content | Treat instructions as data and retain system policy | Block release on successful manipulation |
| Timeout after write | Reconcile by idempotency key before retry | Prove no duplicate effect |
| Budget exhaustion | Stop predictably with resumable context | Tune route or narrow task |
Use the NIST AI RMF to organize ownership and risk decisions, and its Generative AI Profile to consider generative-AI-specific risks. Translate framework outcomes into concrete release evidence: named owners, intended use, affected people, evaluation populations, access controls, monitoring, incident procedures, and criteria to restrict or retire a tool.
Scale with quotas and operational evidence
Begin with read-only tools over a narrow data set. Add a preview tool for a reversible action, then approval, then execution for low-risk cases. Introduce per-user and per-tenant quotas, concurrency limits, backpressure, circuit breakers, and provider rate-limit handling. Protect downstream systems from model-driven bursts. Version tool schemas and keep compatibility during rollout; a model or prompt may continue emitting an older shape. Canary changes by workflow and compare completion, cost, latency, overrides, failures, and customer impact.
Maintain a tool registry as an operational control, not just developer documentation. Each entry should show the business owner, technical owner, permitted callers, data classification, credential, downstream dependency, side effect, maximum scope, idempotency behavior, expected latency, cost, evaluation suite, monitoring, kill switch, and retirement date. Review unused tools and broad credentials quarterly. Removing a tool from the prompt is not retirement if a stale model route, background worker, or direct endpoint can still invoke it.
Capacity tests should include bursts, slow dependencies, large but valid results, and several tenants reaching limits together. Confirm that fairness controls prevent one expensive workflow from starving ordinary requests and that queueing does not outlive the user's authorization or action context.
- The tool inventory names owner, purpose, data class, credentials, side effects, reversibility, budget, dependencies, and retirement route.
- Every call is authorized against trusted identity and current resource context.
- Writes use preview, approval where required, idempotency, optimistic concurrency, and post-action verification.
- Evaluation covers normal, adverse, out-of-scope, duplicate, timeout, stale-state, and injected-content cases.
- Dashboards show useful outcomes, cost per outcome, call fan-out, error classes, policy denials, review load, and recovery.
- On-call staff can disable one tool, tenant, action class, or model route without taking down unrelated work.
Frequently asked questions
How many tools should an agent receive?
Expose the smallest coherent set for the current task and user. A large catalog increases selection ambiguity, permission complexity, testing combinations, and prompt cost. Route to a task-specific tool set rather than giving every request every integration.
Does every tool call need human confirmation?
No. Low-risk reads may proceed under ordinary authorization. Confirmation should match consequence, reversibility, surprise, policy, and user expectation. High-value transfers, external messages, deletions, access changes, and regulated decisions usually need stronger evidence and authority.
What is the most useful tool-calling cost metric?
Use total workflow cost per accepted outcome, segmented by task and risk. Include inference, tools, retries, review, and recovery. Pair it with quality and safety measures; a low cost for an incorrect or unauthorized action is not efficiency.
Conclusion
Tool calling scales when each action behaves like a controlled transaction, not an imaginative extension of the prompt. Narrow contracts, external authorization, explicit budgets, idempotent execution, verification, and targeted evaluation keep both cost and risk legible. Expand the tool set only when operators can explain the outcome, stop the path, reconcile uncertainty, and recover without trusting the model's narrative.