Testing function calling reliability (2026)

TL;DR

Testing function calling reliability requires deterministic contract tests around a probabilistic planner. Verify correct tool selection, schema-valid and semantically valid arguments, authorization, idempotent execution, structured observations, bounded recovery, multi-call ordering, and grounded final answers.

Key Takeaways

• Measure function calling as selection, argument, execution, observation, and completion reliability rather than one success rate.
• Design narrow JSON Schemas with required fields, enums, bounds, explicit additionalProperties, and semantic validation.
• Keep authorization, confirmation, rate limits, and dispatch in deterministic code outside the model.
• Use idempotency keys and operation state to protect side effects from retries and ambiguous timeouts.
• Test no-tool, one-tool, multi-tool, parallel, dependent, duplicate, malformed, and impossible-request cases.
• Return compact structured errors that support bounded recovery without exposing secrets or internal traces.
• Trace stable call IDs, schema and policy versions, validation outcomes, side-effect state, and terminal results.

Testing function calling reliability means proving that an AI system chooses the right tool, supplies valid and authorized arguments, executes it at most as intended, interprets the result correctly, and reaches a safe terminal outcome. A syntactically valid function call is only one checkpoint. The system can still call the wrong tool, use the wrong customer ID, repeat a purchase, ignore a tool error, or claim success after execution failed.

The reliable architecture is a probabilistic planner inside deterministic controls. Tool schemas describe permitted shapes, while application code validates meaning, authorizes the caller, enforces confirmation and budgets, dispatches handlers, records side-effect state, and returns structured observations. This guide turns that architecture into runnable tests and production release gates.

TL;DR

Trace request -> tool decision -> proposed arguments -> schema validation -> semantic validation -> authorization -> confirmation -> execution -> observation -> next call or final response. Test each arrow separately, then evaluate the full task.

1. Define reliability before testing function calling reliability

A useful test model has at least five dimensions. Selection reliability asks whether a tool was needed and whether the right one was chosen. Argument reliability asks whether required values are present, correctly typed, normalized, grounded, and complete. Execution reliability asks whether the authorized handler ran with controlled side effects. Recovery reliability asks whether the agent responds correctly to validation, transport, business, and partial failures. Completion reliability asks whether the final answer reflects the actual result.

Define task success, not just call match. For What is my current balance?, success requires an authorized live lookup and an answer grounded in its result. For Draft a reminder, calling a send function is incorrect because the user asked only for a draft. For Book the cheapest refundable option, a search may be followed by comparison and then explicit confirmation before booking. Several call sequences may be valid.

Classify tool risk. Read-only lookup, reversible mutation, irreversible mutation, external communication, financial action, credential operation, and code execution deserve different policies. A test that tolerates an extra read must not tolerate an extra payment. Define per-tool confirmation, scope, rate, timeout, retry, and audit requirements.

Write invariants. Unknown tools never dispatch. Invalid arguments never partially execute. The model cannot choose its own user or tenant scope. A fallback cannot bypass authorization or confirmation. Every call has a stable ID and terminal state. The final response cannot claim a side effect unless the dispatcher recorded success. These deterministic properties remain exact even when model choice is probabilistic.

2. Design tool schemas as executable contracts

Good schemas reduce ambiguity before tests begin. Give each tool one bounded responsibility and a name that distinguishes it from related operations. Describe when to use it, when not to use it, argument meaning, units, formats, and side effects. Avoid a generic execute function with a free-form command.

Use JSON Schema features supported by the selected provider and your validator: object types, required properties, enums, string patterns, length bounds, numeric limits, array item schemas, and additionalProperties: false. Treat provider-side structured generation as convenience, not your security boundary. Validate again inside the application with the exact schema version used to advertise the tool.

Schema validation cannot prove semantic validity. A syntactically valid date can be in the past, an account ID can belong to another tenant, and an end time can precede a start time. Follow structural validation with domain rules and authoritative lookup. Do not embed authenticated identity in a model-controlled argument when the server can derive it from session context.

Version schemas and handlers together. A trace should identify the exact schema shown to the model. Contract tests compare required fields, types, enums, and compatibility between old in-flight calls and a new deployment. Removing an enum or changing units is a breaking change even if the function name stays the same.

3. Build a task corpus with acceptable call plans

Each evaluation case should contain the user request, trusted context, available tools, seeded external state, expected call plan or acceptable plan set, forbidden calls, argument constraints, simulated tool results, expected final facts, and risk tags. Use state-based assertions when several sequences can succeed.

Include no-tool cases, one-tool cases, choice between similar tools, optional tools, dependent multiple tools, independent parallel tools, clarification before a call, confirmation before mutation, impossible requests, and calls that should be refused. Add realistic conversation history where a correction changes a later argument. Include a stale or malicious value inside tool output to test that results remain untrusted data.

Hard negatives distinguish planning quality. Tell me how calendar reminders work should not create one. Draft an email should not send. Can you check whether order 17 shipped? needs a lookup but not a cancellation. A user asking for another account's data should not cause a lookup with a guessed ID. These cases reveal over-calling and unsafe action bias.

For argument labels, separate exact, set, range, derived, and server-owned values. A city name may allow canonical variants. A quantity may be exact. User ID should be server-owned and absent from model arguments. If the expected value depends on a previous tool result, reference that result field rather than copying one fixed literal into the dataset.

Version cases with tool catalog and policy. Record whether a plan change reflects a product decision or a regression. The agentic tool-calling testing guide provides a broader architecture for bounded agent loops, while this guide focuses on call-level correctness and side-effect safety.

4. Test tool selection, abstention, and clarification

Measure whether the model calls a tool when necessary, selects an acceptable tool, and avoids all forbidden tools. Report under-calling, over-calling, wrong-tool choice, and unnecessary duplicate calls separately. A single tool-selection accuracy figure hides very different operational risks.

Vary tool catalogs. Remove the expected tool and verify a safe explanation or alternative, not an invented function name. Add similarly named read and write tools. Change tool order. Include descriptions with overlapping vocabulary. A robust system should use semantics and policy rather than the first matching name. If tool availability depends on authorization, expose only allowed tools when practical and still enforce server-side authorization.

Clarification is a successful behavior when a required value is absent or ambiguous. Test missing dates, multiple matching contacts, uncertain units, contradictory constraints, and relative time without a timezone. The agent should ask for the minimum necessary information rather than guessing or collecting unrelated personal data. When a default is product-approved, make it explicit in deterministic code and visible in the final confirmation.

No-tool tests are equally important. General explanation, rewriting, summarization of supplied text, and hypothetical questions often need no external action. If current or private state is required, a direct answer without a call is unsupported. For broader hallucination test patterns, use the testing RAG hallucinations guide and adapt its evidence-grounding checks to tool observations.

5. Validate arguments beyond JSON Schema

Run validation in layers. First parse JSON without executing anything. Reject duplicate-key ambiguity according to the parser contract, invalid encoding, excessive depth, excessive size, nonfinite numbers, and unexpected top-level types. Next apply JSON Schema. Then apply semantic rules, canonicalization, authorization, and current-state preconditions. Only the validated immutable command reaches the handler.

Test one defect at a time: missing required field, wrong type, extra field, empty string, whitespace-only value, too-long text, unknown enum, boundary quantity, invalid format, reversed date range, nonexistent object, stale version, duplicate target, and unauthorized scope. Add combinations after individual reason codes are stable. Validation errors should identify correctable fields without exposing internal secrets or record existence across authorization boundaries.

Ground arguments in conversation and trusted sources. If the user corrects Boston to Austin, the call must use Austin. If two people share a name, require a lookup or clarification rather than selecting the first. If a tool result supplies an opaque item ID for a later mutation, assert that the later call uses one of the returned authorized IDs. Do not accept a plausible invented ID merely because it matches a pattern.

Canonicalization should be deterministic and tested. Trim where the domain permits, parse dates with an explicit timezone, represent money with decimal types and currency, and normalize identifiers without damaging case-sensitive values. Preserve the user's original value for audit or confirmation where useful. Never use free-form model reasoning as the only authority for unit conversion in a high-impact command.

6. Enforce authorization, confirmation, and dispatch safety

The dispatcher is the security boundary. It maps an allowlisted tool name to a handler, validates the authenticated principal, checks tenant and object authorization, applies rate and budget limits, enforces confirmation policy, and records the call. The model proposes an action but cannot grant itself permissions.

Test horizontal and vertical authorization. Change object IDs between users and tenants, attempt admin-only tools as ordinary users, replay an old capability after role revocation, and submit direct IDs that were never returned in the current authorized flow. Use consistent external errors where revealing object existence would leak data. Verify access before expensive or sensitive retrieval.

Confirmation must bind to the actual command. A user confirming send the draft to Alex should not authorize a later call to a different recipient or modified body. Store a digest or structured summary of the pending operation, expiry, principal, and scope. Test changed arguments, expired confirmation, reused confirmation, multiple tabs, and cancellation. High-risk actions may require step-up authentication outside the conversation.

Dispatch tests use fake handlers that record calls. Assert that unknown names, failed validation, failed authorization, and missing confirmation produce zero handler invocations. Test handler exceptions, timeouts, cancellations, and process restart. Logs should show tool and call IDs, policy and schema versions, validation outcome, principal reference, and terminal state, while secrets and full payloads remain minimized. The API negative testing guide supplies additional malformed and authorization-boundary cases.

7. Protect side effects with idempotency and state

Retries are unavoidable. The model may repeat a call, the orchestrator may retry after a timeout, a queue may redeliver, or the user may submit twice. Read operations can still be expensive, but mutations can create duplicate orders, messages, tickets, or charges. Define idempotency per tool.

Generate the idempotency key in trusted orchestration and bind it to principal, tool, normalized arguments, and operation intent. The handler or downstream service should atomically return the original result for a repeated key. Test a retry before execution, during execution, after commit but before response, and after success. Test the same key with different arguments and require rejection. Test key expiry only after the business duplication window is safe.

Track a state machine such as proposed, validated, authorized, confirmed, executing, succeeded, failed_retryable, failed_final, canceled, and unknown. Ambiguous timeout is not ordinary failure. Query operation status or reconcile before creating another side effect. The agent should tell the user when the outcome is unknown instead of claiming success or blindly repeating.

Compensation is not a substitute for idempotency. Sending a second email and then another apology does not undo the first. Where a true compensating operation exists, test its authorization, limits, failure path, and audit link to the original. Ensure final responses distinguish completed, not started, rolled back, and uncertain outcomes.

8. Test tool results, errors, and bounded recovery

Tool output is untrusted data. A webpage, database field, or third-party response can contain instructions aimed at the model. Wrap results in a structured observation with tool name, call ID, status, data fields, error class, retryability, and artifact references. In prompts and system design, clearly mark content as data and prevent it from redefining tools or policy.

Define an error taxonomy: validation, authorization, confirmation, not found, conflict, rate limit, timeout before execution, ambiguous timeout, dependency unavailable, business rejection, malformed result, and internal error. Each class has an allowed next step. The agent may correct a validation field, ask the user, retry with backoff, select an equivalent authorized route, query operation status, or stop.

Test whether the agent interprets results accurately. An empty search result is not a tool failure. A partial result must not be described as complete. A business rejection such as inventory unavailable should not trigger repeated purchase attempts. If a result contains both data and warnings, the final response must preserve material limitations.

Bound recovery by total steps, repeated identical calls, wall time, attempts per tool, cost, and side-effect count. Detect no-progress loops where only wording changes. When a budget is exhausted, the orchestrator returns a structured stop condition and the assistant explains what remains unresolved. A later successful call must not erase earlier duplicate or security violations from evaluation.

9. Cover multiple, dependent, and parallel calls

Real tasks use call graphs. A dependent plan may search for a contact, then send to the returned ID. An independent plan may fetch weather and calendar data in parallel. Tests should assert dependencies and state, not always one exact ordering. Represent acceptable plans as a directed acyclic graph where nodes are calls and edges indicate required data or sequencing.

For dependent calls, verify the child uses the actual parent result and stops if the parent fails or is ambiguous. Change the fake parent's ID between runs to catch hard-coded values. Test multiple candidates and pagination. If a mutation depends on a read, recheck important preconditions at execution time to reduce time-of-check to time-of-use errors.

For parallel calls, inject different completion orders, one failure, cancellation, slow response, and partial success. Combine results deterministically. Do not let the last result overwrite earlier data. Decide whether one failed branch cancels others, produces a partial answer, or triggers fallback. Preserve a call ID for every branch and propagate the parent run ID.

Test duplicate and conflicting calls. The model may propose reserve item twice or submit enable and disable concurrently. The orchestrator should deduplicate identical operations where safe and serialize or reject conflicts according to resource version. Parallelism must never bypass per-user budgets, rate limits, confirmation counts, or side-effect caps.

10. Attack the function-calling boundary

Fuzz tool names and arguments with unknown names, case changes, Unicode confusables, whitespace, path-like strings, oversized values, deep arrays, encoded content, markup, null bytes, and fields that resemble internal metadata. Dispatch by exact allowlisted identifier, not dynamic import, reflection, shell construction, or URL supplied by the model.

Attempt parameter injection. A free-form search query should never become raw SQL or shell text. A filename must stay inside an approved storage namespace. A URL-fetch tool needs scheme, hostname, redirect, DNS, IP range, response-size, and timeout controls to prevent server-side request forgery. A code tool needs strong sandboxing and a separate threat model. Test the wrapper controls, not only the model's willingness to behave.

Place malicious-looking instructions inside tool results and user-provided documents. The model should summarize or report them as data when relevant, never call a new privileged tool because the result demanded it. Test secret exfiltration attempts through tool arguments, error messages, final answers, logs, and telemetry. Sensitive configuration should not be in model context at all.

Rate and budget abuse deserves concurrency tests. Submit many proposed calls, trigger repeated validation repair, force expensive result sizes, disconnect during execution, and create fallback loops. Enforce limits in the orchestrator and downstream service. A refusal by the model is helpful but not a control. For destructive tools, use disposable environments and canary objects so tests cannot affect real users.

11. Run a real schema and dispatch harness

Save this JSON Schema 2020-12 example as test_tool_dispatch.py. Run python -m pip install jsonschema pytest, then pytest -q. It proves invalid calls cannot reach the handler.

from dataclasses import dataclass
from typing import Any, Callable

import pytest
from jsonschema import Draft202012Validator


CREATE_TICKET_SCHEMA = {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
        "title": {"type": "string", "minLength": 3, "maxLength": 120},
        "priority": {"type": "string", "enum": ["low", "normal", "high"]},
    },
    "required": ["title", "priority"],
    "additionalProperties": False,
}


@dataclass(frozen=True)
class Principal:
    user_id: str
    tenant_id: str
    can_create_ticket: bool


class Dispatcher:
    def __init__(self, create_ticket: Callable[..., dict[str, Any]]):
        self._create_ticket = create_ticket
        self._validator = Draft202012Validator(CREATE_TICKET_SCHEMA)

    def dispatch(self, name: str, arguments: dict[str, Any], principal: Principal):
        if name != "create_ticket":
            raise LookupError("unknown tool")
        errors = sorted(self._validator.iter_errors(arguments), key=lambda error: list(error.path))
        if errors:
            raise ValueError(errors[0].message)
        if not principal.can_create_ticket:
            raise PermissionError("not authorized")
        return self._create_ticket(
            tenant_id=principal.tenant_id,
            requested_by=principal.user_id,
            title=arguments["title"],
            priority=arguments["priority"],
        )


def test_valid_call_uses_server_owned_scope():
    recorded = []
    dispatcher = Dispatcher(lambda **command: recorded.append(command) or {"id": "T-17"})
    principal = Principal("u-1", "tenant-a", True)

    result = dispatcher.dispatch(
        "create_ticket",
        {"title": "Checkout fails", "priority": "high"},
        principal,
    )

    assert result == {"id": "T-17"}
    assert recorded[0]["tenant_id"] == "tenant-a"


@pytest.mark.parametrize("arguments", [
    {"title": "No priority"},
    {"title": "Checkout fails", "priority": "urgent"},
    {"title": "Checkout fails", "priority": "high", "tenant_id": "other"},
])
def test_invalid_arguments_never_execute(arguments):
    recorded = []
    dispatcher = Dispatcher(lambda **command: recorded.append(command) or {})

    with pytest.raises(ValueError):
        dispatcher.dispatch("create_ticket", arguments, Principal("u-1", "tenant-a", True))

    assert recorded == []


def test_unauthorized_call_never_executes():
    recorded = []
    dispatcher = Dispatcher(lambda **command: recorded.append(command) or {})

    with pytest.raises(PermissionError):
        dispatcher.dispatch(
            "create_ticket",
            {"title": "Checkout fails", "priority": "normal"},
            Principal("u-1", "tenant-a", False),
        )

    assert recorded == []

In a provider integration, translate the provider's function-call envelope into this internal dispatcher command. Keep that adapter thin and contract-test it with recorded or documented response shapes. Do not let provider-specific objects flow directly into handlers. This makes model or SDK changes less likely to alter authorization and side-effect logic.

Extend the harness with fake handlers for retryable failure, ambiguous completion, delayed success, and business rejection. Add a durable idempotency repository for mutations. For end-to-end evaluations, capture selected tool, normalized arguments, validation result, handler count, observation, and final claims under the same run ID.

12. Measure and release after testing function calling reliability

Report metrics at each layer: tool-needed recall, no-tool precision, acceptable-tool rate, schema-valid argument rate, semantic validity, authorization rejection, clarification quality, handler success, duplicate side effects, recovery success, grounded completion, steps, latency, and cost. A successful final answer does not erase an invalid intermediate call. Score high-risk tools separately and require zero unauthorized or duplicate side effects.

Run schema, validator, authorization, dispatch, idempotency, and state-machine tests on every change. Run the labeled agent corpus when prompts, models, tool names, descriptions, schemas, policies, or orchestration change. Use deterministic fake tools for most cases, real sandbox integrations for adapter drift, and repeated trials for model variation. Pin all versions and seeded state.

Before broad release, shadow read-only decisions where allowed, then canary low-risk tools with stable cohorts. Mutating tools require disposable data, explicit confirmation, conservative limits, and independent reconciliation. Monitor selection distribution, validation errors by reason, unknown tools, repeated calls, tool latency, timeout class, fallback depth, side-effect reconciliation, user cancellation, and unsupported success claims.

Store enough evidence to replay without retaining unnecessary sensitive content. Include run ID, call ID, tool and schema version, policy version, normalized argument hashes or redacted values, validation and authorization decisions, idempotency key reference, handler terminal state, observation status, and final result. Roll back model, prompt, schema, and tool catalog coherently. Add minimized production failures to the regression corpus and test adjacent tools that share validators or dispatch code.

Interview Questions and Answers

Q: How would you test function calling in an LLM application?

I would separate tool need, tool choice, argument validity, authorization, execution, observation handling, recovery, and final grounding. Deterministic contract tests would protect schemas and dispatch. A labeled task corpus with fake tools would evaluate acceptable plans, while sandbox integration tests would detect adapter drift.

Q: Is valid JSON enough to execute a tool?

No. JSON must pass a strict schema, semantic rules, canonicalization, authorization, confirmation, current-state checks, and rate or budget policy. Server-owned identity and scope must never come from model arguments. Only then can an immutable command reach the handler.

Q: How do you prevent duplicate side effects?

I use a trusted idempotency key bound to principal, operation, and normalized arguments, with atomic result storage. I test retries before execution, after commit but before response, and after success. Ambiguous timeouts trigger status reconciliation rather than a blind second mutation.

Q: How do you evaluate a task with several valid call sequences?

I define acceptable plan graphs and assert the resulting state, required dependencies, forbidden calls, side-effect count, and final facts. I avoid requiring one exact order for independent calls. Dependent calls must use actual parent outputs.

Q: How should tool errors be returned to the model?

Use a compact structure containing call ID, status, safe error class, retryability, correctable fields, and relevant data. Avoid stack traces, secrets, and ambiguous prose. The orchestrator maps each error class to bounded allowed recovery actions.

Q: How do you test prompt injection through tool output?

I put instruction-like text inside fake search, page, and database results and assert it remains untrusted data. The model must not gain tools, change policy, or expose secrets because a result requested it. Deterministic authorization still protects every proposed call.

Q: Which metrics indicate reliable function calling?

I report tool-needed recall, no-tool precision, acceptable selection, argument schema and semantic validity, successful authorized execution, duplicate effects, recovery, and grounded completion. I slice by tool risk and track latency, attempts, and cost.

Q: What should be included in a tool-call trace?

Include stable run and call IDs, tool and schema versions, policy version, redacted normalized arguments, validation, authorization, confirmation, idempotency reference, handler state, observation status, and final outcome. Do not store hidden reasoning or unnecessary secrets.

Common Mistakes

• Counting a parseable tool call as success. Verify intent, arguments, authorization, execution, observation, and final response.
• Giving the model a broad free-form execution tool. Prefer narrow allowlisted tools with explicit schemas and policy.
• Trusting provider schema enforcement as the security boundary. Validate again in application code and add semantic checks.
• Allowing the model to provide tenant or user identity. Derive scope from the authenticated request.
• Retrying every timeout. Distinguish not-started, failed, committed, and unknown outcomes before repeating a side effect.
• Requiring one exact sequence for multi-tool tasks. Assert dependencies, acceptable graphs, final state, and forbidden calls.
• Feeding raw tool output back as instructions. Wrap it as untrusted structured data with size and content controls.
• Reporting only final task success. Preserve invalid calls, policy violations, repeated effects, attempts, latency, and cost.

Conclusion

Testing function calling reliability is the discipline of surrounding a probabilistic planner with deterministic, observable safety and correctness contracts. Reliable systems select tools appropriately, validate grounded arguments, authorize and confirm actions, execute side effects once, handle results as untrusted data, and stop safely when recovery is impossible.

Start by shrinking and hardening the tool schemas, then run the schema and dispatcher harness. Build a corpus spanning no-tool, clarification, dependent, parallel, failure, and adversarial cases. Make unauthorized calls, duplicate effects, and false success claims release blockers, regardless of how convincing the final prose sounds.

Interview Questions and Answers

Frequently Asked Questions

Related Guides

• Agentic testing with tool calling (2026)
• Testing tool calling in an AI agent (2026)
• Accessibility testing checklist (2026)
• Accessibility testing with Cypress (2026)
• Accessibility testing with Playwright (2026)
• Agile testing quadrants (2026)