Catch an AI agent that asks its tools for more access

A support agent is allowed to read two fields from one customer account. After a prompt injection, it requests an export of every customer with email and payment metadata. The model did not “gain” a role; your tool executor accepted arguments that were broader than the session’s grant.

Tool names alone are a weak permission boundary. The same tool may read or write, touch one tenant or another, return two rows or two million, and expose safe or restricted fields. A useful test changes one of those dimensions and proves the adapter never runs.

Enforce permissions after the model has chosen

Showing only safe tools to a model reduces accidental selection. It does not replace authorization. Tool lists can be cached, a conversation can outlive a role change, a retry can reuse an old call, and another code path can invoke the adapter directly. Treat model output as an untrusted request, even when the model and prompt are controlled by your team.

The enforcement point belongs after arguments have been parsed and normalized but before any protected side effect. Too early, and you authorize a vague intention rather than the actual resource. Too late, and a database query or HTTP request may already have crossed the boundary. Put one wrapper around every path to the adapter so a planner, retry worker, delegated agent, and admin replay all meet the same policy.

Define a grant as data issued by a trusted service. The session identifier, user identity, tenant, roles, or approval record can inform that service, but the agent must not be able to write its own grant. Keep the policy version with the decision so a trace can be replayed against the rule that was active at the time.

For a customer-search tool, four dimensions matter in this example:

1. The selected tool must match the granted tool.
2. The requested action must be in the action set.
3. The tenant and account must be inside resource scope.
4. Requested fields and row count must stay within data limits.

Your application will have different dimensions. A file tool may need path, operation, extension, and byte limits. A payment tool may need source account, destination allowlist, currency, and amount. Do not force every tool into one string role if its risk lives in structured arguments.

The following policy returns a stable decision and knows nothing about prompts. It rejects malformed requests instead of letting Python coercion widen them. The executor calls the registry only after an allow decision.

from dataclasses import dataclass
from typing import Any, Callable, Mapping


@dataclass(frozen=True)
class Capability:
    tool: str
    actions: frozenset[str]
    tenant_ids: frozenset[str]
    account_ids: frozenset[str]
    fields: frozenset[str]
    max_rows: int


@dataclass(frozen=True)
class ToolRequest:
    call_id: str
    tool: str
    action: str
    tenant_id: str
    account_id: str
    fields: tuple[str, ...]
    limit: int


@dataclass(frozen=True)
class Decision:
    allowed: bool
    code: str


def authorize(capability: Capability, request: ToolRequest) -> Decision:
    if request.tool != capability.tool:
        return Decision(False, "tool_not_granted")
    if request.action not in capability.actions:
        return Decision(False, "action_not_granted")
    if request.tenant_id not in capability.tenant_ids:
        return Decision(False, "tenant_not_granted")
    if request.account_id not in capability.account_ids:
        return Decision(False, "account_not_granted")
    if not set(request.fields).issubset(capability.fields):
        return Decision(False, "fields_not_granted")
    if not isinstance(request.limit, int) or isinstance(request.limit, bool):
        return Decision(False, "invalid_limit")
    if request.limit < 1 or request.limit > capability.max_rows:
        return Decision(False, "row_limit_exceeded")
    return Decision(True, "allowed")


class PermissionDenied(Exception):
    pass


def execute(
    capability: Capability,
    request: ToolRequest,
    registry: Mapping[str, Callable[[ToolRequest], Any]],
) -> Any:
    decision = authorize(capability, request)
    if not decision.allowed:
        raise PermissionDenied(decision.code)
    return registry[request.tool](request)

This is an application example, not a universal capability schema. Its value is the explicit comparison between trusted grant and requested effect. A production parser should reject extra fields, enforce string formats, and normalize identifiers before constructing ToolRequest. If normalization changes a value, decide whether to reject it or authorize the canonical form. Never authorize one representation and execute another.

An allow decision also needs freshness. If the user's role or account membership can change during a long agent run, load the current grant at execution. A snapshot fixed at conversation start gives predictable behavior but extends access after revocation. A live lookup closes that gap but adds latency and a dependency on the policy service. Choose deliberately and test the choice.

Test every dimension of the grant

Begin with one allowed request. Mutate exactly one dimension per negative row. This makes failure codes meaningful and stops a tenant defect from being masked by an invalid field. The oracle must include the adapter spy because the policy can return “denied” after a careless caller has already executed the request.

from dataclasses import replace

import pytest

from permissions import Capability, PermissionDenied, ToolRequest, execute


GRANT = Capability(
    tool="customer_search",
    actions=frozenset({"read"}),
    tenant_ids=frozenset({"tenant-a"}),
    account_ids=frozenset({"acct-17"}),
    fields=frozenset({"name", "order_status"}),
    max_rows=5,
)

BASE = ToolRequest(
    call_id="call-1",
    tool="customer_search",
    action="read",
    tenant_id="tenant-a",
    account_id="acct-17",
    fields=("name",),
    limit=1,
)


class SearchSpy:
    def __init__(self):
        self.calls = []

    def __call__(self, request):
        self.calls.append(request)
        return [{"name": "A. Customer"}]


def test_allowed_request_reaches_adapter_once():
    spy = SearchSpy()

    result = execute(GRANT, BASE, {"customer_search": spy})

    assert result == [{"name": "A. Customer"}]
    assert spy.calls == [BASE]


@pytest.mark.parametrize(
    ("tool_request", "expected_code"),
    [
        (replace(BASE, tool="customer_export"), "tool_not_granted"),
        (replace(BASE, action="write"), "action_not_granted"),
        (replace(BASE, tenant_id="tenant-b"), "tenant_not_granted"),
        (replace(BASE, account_id="acct-99"), "account_not_granted"),
        (replace(BASE, fields=("name", "email")), "fields_not_granted"),
        (replace(BASE, limit=6), "row_limit_exceeded"),
    ],
)
def test_escalation_is_denied_before_adapter(tool_request, expected_code):
    spy = SearchSpy()

    with pytest.raises(PermissionDenied, match=f"^{expected_code}$"):
        execute(GRANT, tool_request, {"customer_search": spy})

    assert spy.calls == []

Name the first parameter tool_request, never request. Pytest reserves request for its own built-in fixture, and Metafunc.parametrize rejects it outright with 'request' is a reserved name and cannot be used in @pytest.mark.parametrize. That failure happens during collection, not during the test, so the whole module errors out and every test in the file is skipped, including the positive control. A permission suite that never ran is indistinguishable from a permission suite that passed if you only read the exit banner, which is exactly the sort of silent gap this article is arguing against.

These rows fail for observable reasons when enforcement changes. Delete the tenant comparison and the tenant row reaches SearchSpy; pytest.raises then reports DID NOT RAISE as the block exits, so the row fails before the empty-call assertion is ever evaluated. Delete the field subset check and the email request runs, failing the same way on its own row. Each of those mutations fails exactly one parameterized row and leaves the other five plus the positive control green, which is what makes the reason code in the test id point straight at the deleted check. The test is not asking whether fixture strings appear in a list assembled to contain them; it drives each value through the authorization function and observes the boundary.

Add a malformed-input suite before this one if requests begin as JSON. A Boolean is an integer subtype in Python, which is why the policy explicitly rejects it for limit. Strings such as "5" should not be coerced unless the API contract promises coercion. Duplicate fields, empty identifiers, unknown keys, and overlong values deserve parser cases. Permission testing cannot be reliable when the request shape is ambiguous.

Resource hierarchy needs more than prefix matching. A path like tenant-a/../tenant-b/report.csv can begin with an allowed string before normalization. URL encodings, Unicode normalization, case rules, symbolic links, and database aliases create similar traps. Resolve the resource to a canonical identifier using the same component that will execute it, then authorize that identifier. A hand-built test normalizer that differs from production can manufacture confidence.

Wildcards demand explicit semantics. Does tenant-a/* include newly created accounts? Does it include archived records or admin subresources? If the answer depends on the target API, encode the resolved set or a well-defined policy expression. Test one direct child, one nested child, one sibling, and the boundary itself. Do not infer a security rule from ordinary shell glob behavior.

Volume escalation can hide inside pagination. A tool capped at five rows may let the agent request five rows repeatedly with new cursors. Decide whether the limit applies per call, per task, or per session. Then maintain a trusted budget outside the model and decrement it atomically. A per-call test stays useful, but it cannot prove a task-level export cap.

Response scope needs an oracle of its own. The request may ask only for name while a backend convenience query returns a complete customer object and the adapter forgets to project the allowed fields. The pre-call policy is correct, yet restricted email and payment attributes enter the model context. In an integration test, seed an isolated record with unmistakable restricted values, perform an allowed name lookup, and assert those keys are absent from the adapter result, trace, and final model input. Deleting response projection must make the test fail.

Search filters can widen after authorization too. Suppose policy approves account_id="acct-17", then the adapter builds a database predicate with an optional tenant clause and accidentally drops the account clause when another filter is empty. Compare the canonical request passed by the gate with the query parameters observed by a repository spy. At a higher level, seed allowed and forbidden accounts, execute the allowed request, and assert the result set contains only the allowed identifier. The empty result case is not enough because a broken query against an empty fixture can still look safe.

Batch tools change the unit of authorization. A batch containing four allowed reads and one forbidden write must not execute the write. Decide whether the whole batch fails atomically or allowed items proceed individually. Test both the policy response and adapter calls. If partial execution is supported, return a decision per item and preserve input order or stable item IDs. A single top-level allowed=True cannot describe a mixed batch.

Keep a positive control beside every group of denials. A policy that rejects all requests will pass every escalation row and make the product unusable. The allowed case above proves one intended path reaches the adapter exactly once. Add positive cases for every supported action and resource shape, especially after tightening canonicalization. Least privilege includes enough privilege to finish the user's authorized task.

Catch escalation through retries and delegation

A denied call often triggers replanning. The agent may choose a lower-level HTTP tool, a database query tool, or a delegated worker that can reach the same customer data. From the model's perspective this can look like a helpful fallback. From the policy's perspective it is a second request that needs its own grant.

Inventory effects, not only friendly tool names. If customer_search is restricted but http_request can call the customer service directly, the generic tool must carry destination, method, and response constraints at least as strict as the specialized tool. Better still, do not expose a broad network tool in sessions that only need customer search. Filtering reduces attack surface; executor authorization handles what remains.

Retries must preserve the original principal and budget. A queue worker should not replace a user's capability with its own service account authority. Store a reference to the grant or signed task context, validate freshness, and bind it to the call. If the system intentionally switches principals for a background job, trace that as a new authorization step rather than presenting it as the same user action.

Delegation is safest when it narrows authority. The parent proposes what the child needs, but trusted code computes the intersection with the parent's existing capability. The child cannot gain an action, tenant, account, field, or higher limit that the parent lacked.

from permissions import Capability


def delegate(parent: Capability, requested: Capability) -> Capability:
    if requested.tool != parent.tool:
        raise ValueError("delegated_tool_mismatch")

    return Capability(
        tool=parent.tool,
        actions=parent.actions & requested.actions,
        tenant_ids=parent.tenant_ids & requested.tenant_ids,
        account_ids=parent.account_ids & requested.account_ids,
        fields=parent.fields & requested.fields,
        max_rows=min(parent.max_rows, requested.max_rows),
    )


def test_child_cannot_add_delete_or_email_access():
    parent = Capability(
        tool="customer_search",
        actions=frozenset({"read"}),
        tenant_ids=frozenset({"tenant-a"}),
        account_ids=frozenset({"acct-17"}),
        fields=frozenset({"name"}),
        max_rows=5,
    )
    requested = Capability(
        tool="customer_search",
        actions=frozenset({"read", "delete"}),
        tenant_ids=frozenset({"tenant-a", "tenant-b"}),
        account_ids=frozenset({"acct-17", "acct-99"}),
        fields=frozenset({"name", "email"}),
        max_rows=100,
    )

    child = delegate(parent, requested)

    assert child.actions == frozenset({"read"})
    assert child.tenant_ids == frozenset({"tenant-a"})
    assert child.account_ids == frozenset({"acct-17"})
    assert child.fields == frozenset({"name"})
    assert child.max_rows == 5

That test catches a union operator replacing an intersection. It also exposes empty delegated grants, which should stop the child before it calls a model if the task cannot be completed. Silently giving the child its parent's full grant because the intersection is empty would turn a safe refusal into escalation.

Run a multi-turn example in which the first call is denied for a restricted field, the agent retries with allowed fields, and only the second call executes. The trace should contain two decisions with different call IDs. The target should contain one request. This proves that denial does not poison valid recovery while still preventing the original request.

Revocation during retry is another case. Allow the first read, revoke the tenant membership, then submit a retry. Under live authorization the second call must be denied even if it copies the first arguments. Under snapshot authorization it may remain allowed until the session ends. Either can be a documented product policy, but a test should make the chosen lifetime visible.

Approval records and standing capabilities should not be interchangeable. A reviewer may approve one export with a precise query and destination. Turning that record into a session-wide customer_export capability lets the agent repeat or alter the operation. Bind one-time approvals to an action fingerprint, consume them atomically, and keep them outside the ordinary role grant. Test a second call with the same approval and a call that changes only the destination.

Cached tool menus create another near-miss. The model may still see a tool after an administrator revokes it, then request it confidently. That is an expected stale-presentation problem if the executor loads current grants and denies the call. Evidence should show the tool was advertised under one grant version and rejected under a later version. Refreshing the menu improves user experience, but weakening the executor to match the old menu would restore access.

Parallel calls require atomic budgets. Two tool calls can each observe five remaining rows and both request five. A unit test that invokes them sequentially will miss the race. In an integration test, release two workers together against the real budget store and assert the accepted total does not exceed the grant. Use a conditional update or transactional counter. A test-side mutex makes the result deterministic by removing the production condition you need to examine.

Delegation across different tool types needs an explicit mapping rather than a set intersection. A parent granted order_summary may delegate a narrower order_status tool if trusted policy defines that implication. The agent must not invent the mapping from names or descriptions. Store mappings with a policy version, test every allowed implication, and add a negative case for a superficially similar tool such as order_update.

Read evidence from the effect boundary

Start with the target system. Did the customer service receive a request? Did a database query run? Did an object appear in storage? Application traces can be incomplete or incorrectly labeled. For unit tests, a spy is the target. For integration tests, use an isolated tenant and query the target audit log or state after the call.

Next find the policy decision immediately before the adapter. Record a reason code, policy version, principal reference, grant reference, tool, action, canonical resource identifiers, requested field set, requested limit, and call ID. Avoid storing full returned records or raw prompt text. Permission diagnosis needs the shape of the request, not the customer's data.

Propagate a trace identifier across services so the executor decision can be correlated with the target request. W3C Trace Context standardizes traceparent and tracestate for HTTP propagation, but those headers do not grant permission. Treat an incoming trace ID as correlation data, not trusted identity. A caller can send one, and sampling can omit spans.

When a resource check is removed, the parameterized row that covers it fails at the boundary and its siblings stay green. Selecting that one row keeps the report small enough to read at a glance. The traceback body is trimmed below to the two lines that carry the diagnosis:

$ python -m pytest -q tests/test_permissions.py -k tenant_not_granted
F                                                                        [100%]
=================================== FAILURES ===================================
__ test_escalation_is_denied_before_adapter[tool_request2-tenant_not_granted] __
>       with pytest.raises(PermissionDenied, match=f"^{expected_code}$"):
E       Failed: DID NOT RAISE <class 'permissions.PermissionDenied'>
1 failed, 6 deselected in 0.01s

The deselected count is worth reading too. Six of the seven collected tests were filtered out by -k, which confirms the module collected cleanly and the filter matched a single row. If that line ever reports an error instead of a deselection, the suite did not run at all and no denial was proven.

Do not stop at a 403 from the downstream service. MDN describes 403 as a server understanding a request and refusing to process it because of application logic such as insufficient permissions. That is useful defense in depth, but if your agent executor sent a forbidden cross-tenant request, the local least-privilege boundary already failed. Keep both events: local policy decision and remote response.

A successful target response is not the only dangerous outcome. Timing, error text, and record counts can disclose whether another tenant's resource exists. Test denial responses for consistent public shape while retaining detailed reason codes in restricted logs. If the product deliberately returns 404 to hide existence, do not make the QA oracle depend only on the public status.

Separate permission failures from tool failures

An unknown tool and an ungranted tool can look identical to the user, but their evidence differs. Unknown means the registry has no such adapter or the request failed schema validation. Ungranted means the tool exists but this principal cannot use it. Check authorization before exposing registry details to the caller, then log a safe internal code.

A target-side 403 may reflect a stale service credential rather than agent escalation. Compare the local decision and canonical resource with the credential scope used by the adapter. If local policy allowed tenant-a and the target rejects tenant-a for every user, investigate service authentication. If local policy allowed tenant-b for a session granted only tenant-a, the agent boundary is at fault even if the target blocked it.

A timeout is not a denial. Repeating a timed-out read may be safe; repeating a timed-out write may duplicate an effect. Preserve the call ID and idempotency key, then apply the tool's retry policy. Do not convert timeouts into permission_denied to simplify chat wording, because operators will look at the wrong control.

Schema validation is another near-miss. If limit="all" never becomes a ToolRequest, the parser should report invalid_limit and the policy should not run. A policy event claiming row-limit escalation would be inaccurate. Stage codes such as parse, authorize, execute, and observe make this distinction without pretending they are a universal error taxonomy.

The model may also refuse before making any tool call. That is not proof that permissions work. Keep negative model behavior as a defense-in-depth evaluation, then submit the forbidden ToolRequest directly to the executor. Security gates need adversarial requests independent of whether today's model happens to cooperate.

Roll the policy into a live agent

First map every adapter and caller. Include direct API endpoints, retries, scheduled jobs, replay tools, and delegated workers. A central wrapper is useful only if nothing can bypass it. Add a test that constructs each registered adapter through the production registry and verifies invocation goes through the authorization hook.

Introduce decisions in audit-only mode for existing traffic. The policy computes allowed or denied, but execution continues while you compare events with actual grants. Never call an audit-only denial “blocked.” Track unsupported tools and missing resource fields, because old adapters often lack the data a precise policy needs.

Migrate one tool at a time. Begin with read-only access and an isolated tenant. Enforce tool and action, then tenant and account, then fields and volume. This staged order helps locate compatibility gaps, but the final policy must evaluate all dimensions in one decision. Leaving field checks permanently in audit mode creates a known data-exposure path.

Build a regression corpus from real request shapes after redaction. Include allowed calls, direct escalation, encoded resource aliases, retry fallbacks, delegation, revoked grants, and task-budget exhaustion. Each case should state the protected effect and denial code. Do not save model prose unless the wording caused the request mutation.

Wire fast policy and parser tests into pull requests. Run adapter integration tests against isolated targets after packaging. A small live-agent suite can verify that safe requests remain usable and common injections do not result in accepted calls, but the release gate should still rely on executor oracles.

name: tool-permissions

on:
  pull_request:
    paths:
      - "agent/permissions/**"
      - "agent/tools/**"
      - "tests/permissions/**"

jobs:
  policy:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: python -m pip install -r requirements.txt
      - run: python -m pytest -q tests/permissions

Monitor denial rates by code and tool, not by prompt phrase. A sudden rise in fields_not_granted may be an attack, a model change, or an adapter that renamed a field. Sample traces manually with access controls. Alert on accepted high-risk actions and target discrepancies as well as denials; a bypass produces no denial metric.

Plan a fail-closed response for policy-service outages on high-risk tools. Lower-risk read paths may use a short-lived signed grant if business requirements justify it. Record cache age and grant version, cap its lifetime, and test revocation expectations. A silent fallback to the worker's service credentials is not graceful degradation.

Expect measurable engineering costs even when the policy is correct. Loading a live grant on every call adds a network or database operation. Canonicalizing resources can require metadata lookup. Atomic task budgets introduce contention. Do not publish invented latency figures; benchmark your own traffic with realistic call mixes. If latency is unacceptable, cache only grants whose revocation contract permits it and include expiry in the cache key.

Policy growth creates review cost. Six independent dimensions already produce more combinations than an end-to-end suite can enumerate. Use pairwise or risk-based combinations after covering each dimension alone, then add cases from incidents and design changes. Keep the policy function small enough for direct tests. When rules depend on ownership graphs or organizational hierarchy, give that resolver its own contract and fixtures instead of burying it inside a Boolean expression.

Denied calls can frustrate users if the agent receives no safe recovery information. Return a public message that says the action is unavailable and suggests an allowed alternative, without revealing hidden resources or grants. Test that recovery text separately from the decision. The model should never be able to reinterpret a denial message as a new capability, so follow-up requests still pass through the same executor.

Audit quality has a privacy cost. Field names and account identifiers may be sensitive even when values are omitted. Hashing can aid correlation but does not automatically anonymize a small identifier space. Limit retention, access, and export. Security testers need enough data to distinguish tenant, action, and budget failures; they do not need an unredacted copy of every prompt and record.

When strict per-call checks are the wrong shape

Do not make a person approve every low-risk read if a scoped, short-lived capability can express the task safely. Repetitive prompts cause fatigue and encourage blanket approval. Human approval and machine authorization solve different parts of the problem.

Avoid field-level policy when the target can provide a safer purpose-built endpoint. A get_order_status endpoint that returns one public status is easier to reason about than a generic customer query with a field allowlist. Narrow tools reduce policy combinations and improve model selection.

Per-call row limits are insufficient for extraction risk across a long session. Use an aggregate budget, rate limit, or precomputed view when repeated calls can assemble the restricted dataset. Keep the per-call check as defense in depth, but do not claim it enforces the larger boundary.

Do not depend on an application wrapper when untrusted code runs in the same process and can import the raw adapter. Use process, network, database, or platform isolation appropriate to the threat model. Tests should attempt the real bypass path, not only the intended registry.

Finally, do not label every unexpected tool choice as permission escalation. Choosing a granted but irrelevant tool is a selection-quality defect. Supplying malformed arguments is a contract defect. Receiving a provider timeout is an execution defect. Escalation means the requested effect exceeds authority. Keeping that definition narrow gives security failures the urgency and ownership they need.