PRACTICAL GUIDE / AI agent human approval bypass testing

Prove a tool cannot run before the human approves it

Test approval ordering, action matching, expiry, and replay so an agent tool cannot run on a missing, late, stale, or unrelated human decision.

By The Testing AcademyUpdated August 4, 202622 min read
All field guides
In this guide7 sections
  1. Put approval in the execution path
  2. Prove the trace has a matching earlier decision
  3. Exercise the bypasses people actually ship
  4. Diagnose a bypass from durable evidence
  5. Separate approval defects from UI and trace defects
  6. Migrate without creating a kill switch
  7. When human approval should not be the gate

What you will learn

  • Put approval in the execution path
  • Prove the trace has a matching earlier decision
  • Exercise the bypasses people actually ship
  • Diagnose a bypass from durable evidence

The reviewer is still reading a refund request when the customer receives the money. A background worker took the pending tool call from the queue and invoked the payment adapter directly, bypassing the approval route used by the web application. The approval UI worked exactly as designed; it was never an enforcement boundary.

A credible test has to prove two facts. No protected effect occurs without a matching decision, and the decision existed before execution. A nullable approvalRecord field in a fixture proves neither fact.

Put approval in the execution path

Begin with the action that can hurt someone or change durable state. Examples include sending money, deleting data, publishing content, changing access, emailing a customer, or running code. List every caller that can reach its adapter: the normal agent loop, retry worker, scheduler, admin replay, webhook, migration job, and test utility. If one path skips the gate, a green conversational test is irrelevant.

The gate should receive a normalized action from trusted parsing code. It then loads an approval record from a server-side store and compares the record with that action. The browser or model may provide an opaque approval ID, but it must not provide authoritative reviewer identity, decision, deadline, or approved arguments.

Match all security-relevant details. For a refund, that usually includes order, amount, currency, destination, and tool operation. For an email, include recipients, sender identity, template or body digest, and attachments. If the reviewer saw a draft for one address, changing only the recipient creates a different action and requires a new decision.

Canonical serialization makes matching repeatable. Sort object keys, normalize types before hashing, and version the action schema. Do not normalize away distinctions the reviewer cares about. Converting two destination formats to the same account may be correct; dropping the currency because “all current refunds use USD” is not.

The record also needs a request ID, decision, expiry, reviewer, and consumption state. The request ID stops an approval for one workflow from authorizing another identical-looking action. Expiry limits how long changing circumstances can leave a decision usable. Single-use consumption stops replay. None of these fields should be inferred from the trace at execution time if an authoritative approval store exists.

This small gate shows the contract. Its in-memory store is intentionally simple for unit tests. A production store must make consumption atomic across processes.

Python
from dataclasses import dataclass
from datetime import datetime
from hashlib import sha256
import json
from typing import Any, Callable, Literal


def fingerprint(action: dict[str, Any]) -> str:
    encoded = json.dumps(action, sort_keys=True, separators=(",", ":"))
    return sha256(encoded.encode("utf-8")).hexdigest()


@dataclass
class Approval:
    approval_id: str
    request_id: str
    action_hash: str
    decision: Literal["approved", "rejected"]
    reviewer_id: str
    expires_at: datetime
    used_at: datetime | None = None


class ApprovalDenied(Exception):
    pass


class ApprovalStore:
    def __init__(self) -> None:
        self.records: dict[str, Approval] = {}

    def add(self, record: Approval) -> None:
        self.records[record.approval_id] = record

    def consume(
        self,
        *,
        approval_id: str,
        request_id: str,
        action: dict[str, Any],
        now: datetime,
    ) -> Approval:
        record = self.records.get(approval_id)
        if record is None:
            raise ApprovalDenied("approval_not_found")
        if record.request_id != request_id:
            raise ApprovalDenied("approval_request_mismatch")
        if record.action_hash != fingerprint(action):
            raise ApprovalDenied("approval_action_mismatch")
        if record.decision != "approved":
            raise ApprovalDenied("approval_rejected")
        if record.used_at is not None:
            raise ApprovalDenied("approval_already_used")
        if now >= record.expires_at:
            raise ApprovalDenied("approval_expired")

        record.used_at = now
        return record


def run_protected_tool(
    *,
    store: ApprovalStore,
    approval_id: str,
    request_id: str,
    action: dict[str, Any],
    now: Callable[[], datetime],
    adapter: Callable[[dict[str, Any]], Any],
) -> Any:
    store.consume(
        approval_id=approval_id,
        request_id=request_id,
        action=action,
        now=now(),
    )
    return adapter(action)

The most important line is the call order inside run_protected_tool. Consumption happens before the adapter. If a caller catches ApprovalDenied and invokes the adapter anyway, the wrapper is not the boundary. Keep the raw adapter private to the module or service and expose only the guarded function to agent code.

There is a crash window after consumption and before the target confirms the effect. Reversing the calls creates a worse replay window. Use an idempotency key bound to the request and action, or write a transactional outbox entry in the same transaction that consumes approval. When a remote outcome is uncertain, reconcile the target before asking for a second approval. Human review is not a substitute for delivery semantics.

Prove the trace has a matching earlier decision

Live enforcement prevents harm. Trace auditing answers whether the recorded story supports the decision and whether another path escaped the gate. Keep those jobs separate. An auditor that finds a violation after payment is valuable, but it did not protect the payment.

Define event types, fields, and order for your application. The following example uses a monotonically increasing sequence assigned by the workflow service for one run. It does not sort by wall-clock time. Distributed clocks can disagree, exporters can buffer, and collectors can ingest events late.

Each execution must have an earlier request and an earlier approved decision for the same request and action fingerprint. A later matching approval is evidence of an ordering bypass, not retroactive authorization. An earlier rejection is not approval. An earlier approval for a different amount or destination is not a match.

Python
from dataclasses import dataclass
from typing import Literal


EventKind = Literal["approval_requested", "human_decision", "tool_started"]


@dataclass(frozen=True)
class Event:
    sequence: int
    kind: EventKind
    request_id: str
    action_hash: str
    approval_id: str | None = None
    decision: Literal["approved", "rejected"] | None = None


@dataclass(frozen=True)
class Violation:
    execution_sequence: int
    code: str


def audit_approval_order(events: list[Event]) -> list[Violation]:
    violations: list[Violation] = []
    executions = [event for event in events if event.kind == "tool_started"]

    for execution in executions:
        if execution.approval_id is None:
            violations.append(
                Violation(execution.sequence, "execution_approval_id_missing")
            )
            continue

        earlier = [event for event in events if event.sequence < execution.sequence]
        later = [event for event in events if event.sequence > execution.sequence]

        matching_requests = [
            event
            for event in earlier
            if event.kind == "approval_requested"
            and event.request_id == execution.request_id
            and event.action_hash == execution.action_hash
        ]
        matching_decisions = [
            event
            for event in earlier
            if event.kind == "human_decision"
            and event.request_id == execution.request_id
            and event.action_hash == execution.action_hash
            and event.approval_id == execution.approval_id
            and event.decision == "approved"
        ]
        decision_values = {
            event.decision
            for event in earlier
            if event.kind == "human_decision"
            and event.request_id == execution.request_id
            and event.action_hash == execution.action_hash
            and event.approval_id == execution.approval_id
        }
        if decision_values == {"approved", "rejected"}:
            violations.append(
                Violation(execution.sequence, "conflicting_human_decisions")
            )
            continue

        matching_approvals = [
            decision
            for decision in matching_decisions
            if any(request.sequence < decision.sequence for request in matching_requests)
        ]

        if matching_approvals:
            continue

        approved_later = any(
            event.kind == "human_decision"
            and event.request_id == execution.request_id
            and event.action_hash == execution.action_hash
            and event.approval_id == execution.approval_id
            and event.decision == "approved"
            for event in later
        )
        rejected_earlier = any(
            event.kind == "human_decision"
            and event.request_id == execution.request_id
            and event.action_hash == execution.action_hash
            and event.approval_id == execution.approval_id
            and event.decision == "rejected"
            for event in earlier
        )
        decision_for_other_action = any(
            event.kind == "human_decision"
            and event.request_id == execution.request_id
            and event.approval_id == execution.approval_id
            and event.action_hash != execution.action_hash
            for event in earlier
        )
        decision_for_other_approval = any(
            event.kind == "human_decision"
            and event.request_id == execution.request_id
            and event.action_hash == execution.action_hash
            and event.approval_id != execution.approval_id
            for event in earlier
        )
        approval_before_request = any(
            decision.sequence < request.sequence
            for decision in matching_decisions
            for request in matching_requests
        )

        if approved_later:
            code = "approval_after_execution"
        elif rejected_earlier:
            code = "execution_after_rejection"
        elif decision_for_other_action:
            code = "approval_action_mismatch"
        elif decision_for_other_approval:
            code = "approval_id_mismatch"
        elif approval_before_request:
            code = "approval_before_request"
        elif not matching_requests:
            code = "matching_request_missing"
        else:
            code = "matching_approval_missing"
        violations.append(Violation(execution.sequence, code))

    return violations

The function derives a result from event relationships. It does not check that a hard-coded failure category appears in a hard-coded taxonomy. Moving approval after execution changes the returned violation. Changing the action hash changes it again. Removing the execution produces no execution violation because no protected call was observed; a separate completeness control should check expected terminal events.

Sequence values need a trusted writer and a clear scope. A database append index, workflow transition number, or signed event-chain position can work. A client-provided number cannot. If multiple services assign independent sequences, add causal identifiers such as request event ID and approval event ID rather than comparing unrelated counters.

W3C Trace Context helps propagate a trace identity and parent relationship across HTTP boundaries. It does not define your approval event, action fingerprint, or business ordering rule. Carry trace IDs for correlation, then keep trusted domain event IDs for authorization evidence. Never interpret possession of traceparent as permission.

Exercise the bypasses people actually ship

Start with a valid trace and transform one property per case. The allowed control proves the auditor does not reject every execution. Seven negative rows cover missing approval, late approval, rejection, action substitution, approval-ID substitution, a missing approval ID, and a decision recorded before its request. The expected code tells the responder which invariant failed.

Python
from dataclasses import replace

import pytest

from approval_audit import Event, Violation, audit_approval_order


ACTION = "sha256:refund-2500-usd-order-19"
OTHER_ACTION = "sha256:refund-250000-usd-order-19"

REQUESTED = Event(10, "approval_requested", "req-19", ACTION)
APPROVED = Event(
    20,
    "human_decision",
    "req-19",
    ACTION,
    approval_id="approval-19",
    decision="approved",
)
STARTED = Event(
    30,
    "tool_started",
    "req-19",
    ACTION,
    approval_id="approval-19",
)


def test_matching_approval_precedes_execution():
    assert audit_approval_order([REQUESTED, APPROVED, STARTED]) == []


@pytest.mark.parametrize(
    ("events", "expected"),
    [
        (
            [REQUESTED, STARTED],
            Violation(30, "matching_approval_missing"),
        ),
        (
            [REQUESTED, STARTED, replace(APPROVED, sequence=40)],
            Violation(30, "approval_after_execution"),
        ),
        (
            [REQUESTED, replace(APPROVED, decision="rejected"), STARTED],
            Violation(30, "execution_after_rejection"),
        ),
        (
            [REQUESTED, replace(APPROVED, action_hash=OTHER_ACTION), STARTED],
            Violation(30, "approval_action_mismatch"),
        ),
        (
            [REQUESTED, APPROVED, replace(STARTED, approval_id="approval-99")],
            Violation(30, "approval_id_mismatch"),
        ),
        (
            [REQUESTED, APPROVED, replace(STARTED, approval_id=None)],
            Violation(30, "execution_approval_id_missing"),
        ),
        (
            [
                replace(APPROVED, sequence=10),
                replace(REQUESTED, sequence=20),
                STARTED,
            ],
            Violation(30, "approval_before_request"),
        ),
    ],
)
def test_trace_rejects_approval_bypasses(events, expected):
    assert audit_approval_order(events) == [expected]

Now test the live effect. Build a spy adapter, create a valid record for a small refund, and submit a larger action with the same approval ID. Assert approval_action_mismatch and an empty call list. Then submit the original small action, assert one call, and submit it again to prove replay is denied. Those three observations show matching, successful use, and single-use behavior through the actual gate.

Python
from datetime import datetime, timedelta, timezone

import pytest

from approval_gate import (
    Approval,
    ApprovalDenied,
    ApprovalStore,
    fingerprint,
    run_protected_tool,
)


def test_changed_amount_is_denied_then_original_action_is_single_use():
    now = datetime(2026, 8, 4, 10, 0, tzinfo=timezone.utc)
    approved_action = {
        "tool": "refund",
        "order_id": "O-19",
        "amount_cents": 2500,
        "currency": "USD",
    }
    changed_action = {**approved_action, "amount_cents": 250000}
    calls = []
    store = ApprovalStore()
    store.add(
        Approval(
            approval_id="approval-19",
            request_id="req-19",
            action_hash=fingerprint(approved_action),
            decision="approved",
            reviewer_id="reviewer-8",
            expires_at=now + timedelta(minutes=10),
        )
    )

    with pytest.raises(ApprovalDenied, match="^approval_action_mismatch$"):
        run_protected_tool(
            store=store,
            approval_id="approval-19",
            request_id="req-19",
            action=changed_action,
            now=lambda: now,
            adapter=calls.append,
        )
    assert calls == []

    run_protected_tool(
        store=store,
        approval_id="approval-19",
        request_id="req-19",
        action=approved_action,
        now=lambda: now,
        adapter=calls.append,
    )
    assert calls == [approved_action]

    with pytest.raises(ApprovalDenied, match="^approval_already_used$"):
        run_protected_tool(
            store=store,
            approval_id="approval-19",
            request_id="req-19",
            action=approved_action,
            now=lambda: now,
            adapter=calls.append,
        )
    assert calls == [approved_action]

The failed changed-action attempt does not consume the record. That allows the action the reviewer actually approved to proceed. Whether malformed or mismatched attempts should revoke a ticket is a separate threat decision. Revocation can slow an attacker, but it also lets anyone who obtains an approval ID deny service by sending one bad call. Keep denial and revocation as separate state transitions, then test the policy you choose.

The successful call marks the record used before calling the spy. If the spy raised an exception, the approval would remain consumed in this sample. That is conservative for replay but can strand legitimate work. Production code needs a state such as pending delivery plus target reconciliation, not a test that resets used_at whenever an exception appears. An exception does not prove the target did nothing.

A second worked example should bypass the web controller. Publish a job directly to the same queue format a worker consumes, leaving approval ID absent or unknown. The worker must call run_protected_tool and reject it. If the job handler imports the payment adapter directly, the integration test will find the architectural hole that an HTTP route test misses.

A third example changes the action after display. Capture the canonical payload rendered to the reviewer, approve it, then mutate the destination in graph state before execution. The fingerprint at the gate must be computed from the execution action, not copied from the approval response. Otherwise a malicious or buggy state update can carry the old hash beside new arguments.

Test exact boundary time with an injected clock. A decision is valid before its deadline and invalid at the deadline in the gate above. The two cases should differ by the clock value alone. Do not wait in real time, and do not add a sleep that makes CI timing part of a security oracle.

Concurrency matters for single use. Two workers can read used_at=None at once in the simple store. Exercise your real persistence method with two consumers released together, then prove one consume succeeds and the other receives approval_already_used. The target idempotency record should also show one business operation. A unit-test lock would conceal the race.

Message brokers make duplicate delivery normal enough to deserve a named case. Deliver the exact job twice with the same request, approval, and target idempotency key. One worker may finish before the other begins, or both may overlap. In either schedule, the business effect count must be one. The losing worker should report replay or already-in-progress, not request a fresh approval immediately.

Out-of-order messages create a different case. An approval event may reach the trace pipeline after a queued execution job, while the authoritative store already contains the decision. The live gate can safely allow execution even though the observability stream arrives out of order. Audit by trusted domain sequence or causal event IDs, not collector arrival. If neither exists, label ordering unknown instead of claiming a confirmed bypass.

Some actions require two reviewers or roles with separation of duties. Extend the store to hold individual decisions and let trusted policy calculate quorum for one request and action fingerprint. Test one approval, two valid distinct reviewers, duplicate decisions by the same reviewer, a rejection, and an approval from an ineligible role. Do not collapse the set into one Boolean before the gate, because the test then cannot prove who satisfied the rule.

Resubmission after rejection should create a new request or an explicit new review round. Rewriting the rejected record to approved destroys the history and makes ordering ambiguous. A test should retain the first rejection, link the changed action to a new request, and require the execution to reference only the later approved round.

Diagnose a bypass from durable evidence

Look at the protected target first. For a refund, use the provider operation or an isolated ledger entry. For an email, use the delivery service event. For deletion, inspect a tombstone or audit record. A missing trace span is not proof that nothing happened, and a “denied” UI state can coexist with an earlier side effect.

Find the tool_started event and its request ID, action fingerprint, approval ID, adapter name, and target idempotency key. Then locate the decision event it claims to depend on. Compare trusted sequence or causal links. A matching event ingested later may have happened earlier, so ingestion order alone is not enough. Conversely, a later domain sequence cannot authorize an earlier start even if its wall-clock timestamp looks earlier.

Inspect what the reviewer saw. The approval request should reference the same action fingerprint and a redacted display projection generated from that action. Store the display schema version. If the action matches but the UI omitted the destination or amount, enforcement worked while informed consent failed. That is a review-presentation defect and can still be release-blocking.

Check the approval store separately from the trace. Was the record present, approved, unexpired, and unused when the executor evaluated it? A trace exporter can drop events while the store remains authoritative. A stale read replica can also make an approved decision look missing. Record which store and consistency path served the decision.

When the late-approval guard is accidentally widened so that a matching decision counts no matter where it sits in the trace, rather than only before the execution, exactly one row flips and the other six hold. The traceback body is trimmed below to the assertion and the diff:

Shell
$ python -m pytest -q tests/test_approval_audit.py -k approval_bypasses
.F.....                                                                  [100%]
=================================== FAILURES ===================================
___________ test_trace_rejects_approval_bypasses[events1-expected1] ____________
>       assert audit_approval_order(events) == [expected]
E       AssertionError: assert [] == [Violation(ex...r_execution')]
E         
E         Right contains one more item: Violation(execution_sequence=30, code='approval_after_execution')
E         Use -v to get more diff
1 failed, 6 passed, 1 deselected in 0.01s

Read the progress line as data, not decoration. Seven characters means seven parameterized rows ran, which is the whole negative set, and the single deselection is the positive control that -k filtered out. Only the second row changed behavior. Note also that -q abbreviates the middle of the assertion to assert [] == [Violation(ex...r_execution')]. The full expected value survives on the Right contains one more item line underneath, and -v prints the untruncated diff. Never quote the abbreviated form as if it were the complete comparison, because the elided text is exactly the part a reader needs.

That output says the execution event was present and the expected ordering violation disappeared. If instead the live-gate test shows a nonempty adapter call list, the prevention control failed. Keep those alerts separate. An audit-classification regression is serious, but it is not evidence that the adapter ran.

Protect the evidence itself. Approval logs contain user identity, action metadata, and sometimes sensitive resource IDs. Use access controls, retention limits, and redaction. Hashing an order ID is not anonymization when the identifier space is small. Retain enough to correlate the record without copying full prompts or payment details.

Check trace completeness before trusting a clean audit. Count protected target operations and compare them with tool_started and terminal events using stable target idempotency keys. A target operation with no start event is an instrumentation gap and a possible bypass path. A start event with no terminal event is an uncertain outcome. These comparisons can find missing spans without assuming that every absent event means an unauthorized effect.

Duplicate events should not create duplicate violations. Exporters retry, collectors reprocess, and storage jobs can replay batches. Give every event an immutable event ID and deduplicate in the audit pipeline. Keep duplicate counts as telemetry because a sudden increase can signal pipeline trouble, but do not page the approval team twice for the same execution.

Action fingerprints are useful only if investigators can reproduce them from a permitted view of the action. Store the schema version and canonicalization version. During a migration, calculate both old and new fingerprints in a controlled validator and compare them before switching. If an old event lacks required fields, mark it unverifiable. Inventing a missing field to force a match weakens the evidence.

Reviewer identity needs provenance. A display name copied from a client is not enough. Link the decision to the authenticated subject and authentication event retained by the approval service. The trace can show a pseudonymous reviewer reference while restricted systems retain the identity. QA should test that changing a submitted reviewer field does not change the server-side principal.

Separate approval defects from UI and trace defects

A button that remains enabled after expiry is a UI defect, but the backend can still deny the stale click. Test both layers. The browser case should display an expired state and request a new review; the direct executor case should reject regardless of what the browser shows. Never accept server-side execution merely because the client tried to prevent it.

A missing trace decision can be an instrumentation defect when the store proves a valid consume and the target accepted exactly once. Investigate exporter errors, sampling, field redaction, and event transaction boundaries. Do not classify it as a human bypass without evidence. It is still an auditability gap, which may violate your release requirements.

An approval denial can be correct while the product reports a generic failure. If the record expired, the user should be prompted for a fresh review rather than told the payment provider is down. Preserve the internal denial code through orchestration, then map it to safe product copy. Test mapping separately so wording changes cannot alter authorization.

A tool failure after a valid consume is not a bypass. The approval existed and preceded the attempt, but the target timed out or rejected the request. Reconciliation and retry policy decide what happens next. Reusing the same approval may or may not be allowed by your business contract; if it is allowed, bind reuse to the same idempotency key and a confirmed non-effect.

A reviewer mistake is also distinct. A real human can approve a harmful action that was displayed accurately. Authorization and trace ordering will pass. That case belongs to review design, role policy, separation of duties, or fraud controls. Do not bend the bypass oracle until it flags every bad business decision.

Clock skew can make a safe trace look late when events are sorted by timestamp. Compare domain sequence and causal IDs. If only wall-clock values disagree, fix time synchronization or presentation. If the execution's causal parent is not the matching approval, the defect remains even when timestamps happen to look ordered.

Migrate without creating a kill switch

Map the current flow before enforcing. Find where approval is requested, stored, resumed, and consumed. Search for raw adapter imports and direct network clients. Add an audit event at each current execution path so you can see which calls lack request or approval linkage.

Introduce canonical action schemas per tool. Version them and generate reviewer displays from the same normalized object used for hashing. During migration, old pending approvals may not contain enough data to bind safely. Cancel them or require review again. Guessing a fingerprint from incomplete state creates the exact mismatch the new design should prevent.

Build an adapter inventory that is executable. Register each protected adapter with its required approval policy and have a test fail when a new side-effecting adapter lacks an entry. The registry cannot discover code that bypasses it, so combine it with code review and dependency boundaries. Keeping raw clients in a separate package with restricted imports makes accidental bypass harder to ship.

Run the new gate in report-only mode briefly, but label events would_deny and keep a fixed enforcement date. Compare would-deny calls with legitimate operations. Missing approval IDs often reveal jobs that were considered “internal” and never passed through the user route.

Report-only mode carries risk because known bypasses still execute. Use isolated or low-consequence tools, cap the observation period, and review every would-deny effect. For payments, deletion, credential changes, or external publishing, prefer a shadow environment or replayed requests rather than allowing unauthorized production calls for measurement.

Enforce one reversible tool first. Exercise valid, missing, rejected, mismatched, expired, replayed, late, and concurrent cases in a controlled environment. Confirm both gate events and target state. Then move to higher-risk tools with an operator runbook for denial spikes and uncertain outcomes.

The CI split should keep pure matching and audit-order tests fast, then run adapter integration cases separately. The example below has no model dependency. A model can produce adversarial requests in another suite, but direct construction guarantees the gate sees every forbidden shape.

YAML
name: human-approval-gate

on:
  pull_request:
    paths:
      - "agent/approval/**"
      - "agent/tools/**"
      - "tests/approval/**"

jobs:
  approval:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: python -m pip install -r requirements.txt
      - run: python -m pytest -q tests/approval

Avoid a rollback switch that exposes raw adapters. If the approval service fails, pause the tool, route work to a separate manual process, or use a narrowly designed break-glass procedure. That procedure needs stronger identity, limited scope, explicit operator intent, and its own tests. “Skip approval” is not an operational fallback.

The design costs latency, storage, reviewer time, and recovery complexity. Strongly consistent consumption can add a database round trip. Short deadlines increase repeated reviews. Action hashing adds schema governance. Idempotency and reconciliation add state. Measure those costs in your system rather than inventing figures, then choose controls by consequence.

Rollout metrics should distinguish safety from usability. Track attempted executions, allowed decisions, denial codes, abandoned reviews, repeated reviews, reconciliation cases, and target discrepancies. Do not combine them into one pass rate. A decrease in denials can mean better requests, or it can mean the gate stopped running. Compare the number of protected target effects with gate decisions to detect the latter.

Train operators on uncertain outcomes. If consumption succeeded but the target response was lost, they must search by idempotency key before issuing another approval. The runbook should show where that key appears and who can perform reconciliation. Test the runbook in a staging incident drill, including the case where the trace exporter is unavailable.

When human approval should not be the gate

Do not interrupt harmless, reversible actions merely because the framework supports human-in-the-loop flows. Repeated low-value prompts train people to approve without reading. Use scoped permissions, rate limits, or automated validation when they express the risk better.

Avoid asking a reviewer to approve information the interface cannot explain. A generic “run tool” dialog with hidden arguments is ceremony, not informed control. Improve the action preview or narrow the tool before adding a required click.

Human approval is too slow for some safety invariants. Tenant isolation, schema validation, amount bounds, and prohibited destinations should be enforced automatically on every call. A reviewer can be an additional control for exceptional actions, not a replacement for deterministic policy.

Do not use approval to rescue an unreliable effect. If the target lacks idempotency and retries can duplicate payments, fix delivery semantics. Asking twice does not tell you whether the first operation succeeded.

Finally, a high-volume workflow may need separation of duties or batch approval rather than one decision per call. Define exactly what the batch covers, freeze or fingerprint its contents, and reject additions after review. The same core rule remains: execution must be bound to a matching earlier decision, and the protected adapter must never depend on the model's claim that a human said yes.

Emergency access is another design, not an exemption hidden in the normal gate. A break-glass path should identify the operator, narrow the allowed action, expire quickly, and produce a mandatory review event. Ordinary agent credentials must not be able to select it. Include direct negative tests from the agent worker and a positive drill from the authorized operator path.

// FIELD DISPATCH

Get the QA Field Notes

Weekly QA battles, AI testing guides, and interview drills. Free on Substack.

// LIVE COURSE / THE TESTING ACADEMY

AI Tester Blueprint

Master GenAI, AI Agents, MCP, RAG, CrewAI. Build 23+ real AI projects.

From the instructor behind this guide.

AI testing roles are up 180% and pay 12-22 LPA. 12+ weeks / 65+ live hrs / Sat-Sun 8:30 AM IST.

Code PROMODE / 10% offJoin the batch

The Testing Academy editorial desk

Practical QA guidance built around test evidence, production tradeoffs, and interview-ready explanations.

Published July 25, 2026 / Reviewed August 4, 2026

PRIMARY REFERENCES

Verify the details at the source

QABattle guides are practical explanations. Product behavior, standards, and APIs can change, so use these primary references for the canonical details.

  1. 01
    Official docs.pytest.org reference

    docs.pytest.org

    Primary documentation selected and verified for the claims in this guide.

  2. 02
    Official w3.org reference

    w3.org

    Primary documentation selected and verified for the claims in this guide.

  3. 03
    Evaluate complex agents

    LangSmith

    Official guidance for final-response, trajectory, and single-step agent evaluation.

  4. 04
    Agents SDK tracing

    OpenAI

    Primary trace model for agent runs, generations, tool calls, handoffs, and guardrails.

FAQ / QUICK ANSWERS

Questions testers ask

Where should an agent’s human approval be checked?

Put the check in the unavoidable path immediately before the protected adapter or side effect. A prompt instruction and an approval screen can support the flow, but neither enforces it.

How can a test prove approval happened before execution?

Use a trusted per-run sequence or causal link, then require a matching approved decision at a lower sequence than the execution event. Wall-clock timestamps alone can be reordered by skew or delayed ingestion.

What must be matched between approval and a tool call?

Bind the request ID and a canonical fingerprint of the exact action, including security-relevant arguments. Tool name alone cannot distinguish a small refund from a much larger one.

Should a human approval be reusable after a retry?

Reject reuse by default for side effects, and combine single-use approval with target idempotency. If the first outcome is uncertain, reconcile it before asking for or consuming another decision.

Can trace analysis replace a live approval gate?

A trace can reveal that a bypass happened, but it runs after the event and may be incomplete. Enforcement must block the adapter; trace auditing is a separate detection and investigation control.