Can another tester run this case without calling the author?

A new tester reaches step four and cannot find “Submit claim.” The button was renamed “Send for review” three releases ago, and everyone on the original team has been translating the old wording from memory. The case has been passing because its regular runners know when to ignore it.

What a cold runner exposes that the author cannot

An author reads intent into shorthand. “Create a standard account” includes a remembered role, region, feature flag, balance, and cleanup routine. “Verify the page works” points to a mental list of expected fields. Another tester receives only the words on the page. That gap is the real subject of handoff review.

Grammar matters, but it is not the gate. A polished case can still depend on yesterday's data, an earlier suite case, an undocumented admin tool, or one person's interpretation of success. A useful review asks whether the document functions as an executable test asset when the author is unavailable.

Freeze the revision before the dry run. Record the case ID, version, linked requirement or risk, target build, environment profile, and reviewer. Otherwise the author can silently edit steps while the runner is executing, and nobody knows which text passed. Corrections are welcome after the attempt, but they belong in a new revision with the observed problem attached.

Choose a reviewer who understands testing and can access the product, but did not participate in writing the case. Total product ignorance creates onboarding questions that no case should carry. Deep familiarity creates the opposite problem: the reviewer fills gaps from memory. The useful cold runner knows the domain at the level expected of the next owner and lacks private context about this asset.

During the first run, the author does not coach. The runner records every question, assumption, blocked action, label mismatch, data substitution, and judgment call. If safety requires stopping, stop. If a harmless ambiguity can be explored, record the chosen interpretation before continuing. An unrecorded hint makes a fragile case appear transferable.

The outcome is more specific than pass or fail. A case can be executable and reveal a product defect. It can be blocked by environment or test data. It can be wrong because the requirement changed. It can be ambiguous because two outcomes both satisfy the words. Classifying the outcome prevents a documentation defect from being filed against the product or a real product defect from being dismissed as poor instructions.

Give the runner a review sheet separate from the case. For each step, it records start state, action taken, observed result, evidence reference, question or assumption, and whether author help was needed. The sheet also records setup time, but time is diagnostic rather than a universal pass threshold. A complex recovery case can legitimately take longer than a simple profile edit. The failure is undocumented work, not slowness by itself.

Agree on stop conditions before the run. Stop when proceeding could send real communication, change shared configuration, consume another team's fixture, create an irreversible financial effect, or destroy evidence. Mark the exact state reached and the missing instruction. Telling a runner to use judgment without boundaries shifts risk from the document to the individual.

Handoff acceptance can then use observable criteria. The stated environment and data can be prepared. Every action identifies a reachable target. Each expected result supports one verdict. Required evidence can be collected. Cleanup or retention works after pass, fail, and early stop. The runner needs no undocumented author intervention. These criteria can fail against a real case revision, unlike a review checkbox that is marked complete by the author.

The ISTQB Foundation Level syllabus describes test cases in natural language as involving necessary preconditions, inputs, and postconditions, and says acceptance tests should be understandable to stakeholders. The Advanced Test Analyst syllabus adds concrete emphasis on preconditions, input data, expected results, postconditions, traceability, and keeping testware current. Those are useful ingredients. The product requirement remains the source for what a particular result should be.

Decide readiness from the evidence a runner needs

Start with one behavior and one reason to fail. “Customer can apply a valid promotion to an eligible cart” is coherent. “Customer signs in, edits a profile, applies a promotion, checks out, downloads an invoice, and logs out” contains several independent rules. When it fails, the title does not locate the defect and rerunning later steps may be impossible.

One behavior does not mean one action. A state transition may require preparation, trigger, verification, and cleanup. The boundary is whether the case supports one diagnosable verdict. A refund workflow can contain several steps if the rule under test is that one approved refund moves an order and ledger through a defined transition.

Preconditions describe state, not history. “Run TC-041 first” hides what TC-041 created. Replace it with the required order status, customer type, feature state, and a repeatable way to obtain them. If a planned workflow intentionally passes an artifact between cases, define that dependency at suite level, name the artifact, and provide a restart point. Pretending every sequence is independent can be as misleading as hiding all coupling.

Test data earns its place by covering a partition, boundary, decision rule, or known risk. A value such as 42 is not useful merely because it is concrete. State why the quantity is the maximum, why the account is ineligible, or why the date sits immediately after expiry. Record whether the runner generates, reserves, or reuses the value and how collisions are prevented.

Steps use current terms and identify the target without prescribing irrelevant mechanics. “On the Review order page, choose Express shipping” is clearer than “click the second radio button.” It remains usable with a keyboard and survives harmless layout movement. If two controls share a label, add the section or item that distinguishes them.

Expected results are observable and local to the action that produces them. “System handles invalid quantity correctly” asks the runner to invent an oracle. “Quantity remains 10, an inline message says Maximum quantity is 10, and the cart total remains unchanged” supports one verdict. If the important effect is server-side, include the approved inspection route rather than assuming the UI is the whole system.

Cleanup states what must remain for evidence and what must be restored. A failed case may need its order preserved until triage. A consumed invitation, locked account, changed feature flag, or queued email can affect later tests. Name cleanup ownership and the procedure for both pass and early-stop paths.

Traceability points both ways. The case links to a versioned requirement, risk, decision table, or defect. The requirement or coverage view should let maintainers find affected cases when behavior changes. A bare ticket URL is weak if the ticket can be rewritten without history. Record the relevant revision or acceptance-criteria version.

Ownership completes the handoff. An owner reviews product changes, data dependencies, and stale evidence. A review date alone creates calendar work without a trigger. Add event triggers such as a changed requirement, renamed workflow, escaped defect, test-data migration, environment replacement, or repeated runner question.

Work through three cases that fail for different reasons

The first case has a vague oracle. Its title says Check maximum cart quantity. Its only step says Set quantity to 11 and verify the validation works. One runner expects the field to reset to 10. Another expects an error while 11 remains visible. Both can claim they followed the case.

The rewritten example below is illustrative test data, not a claim about a real store. The fictional requirement CART-204 says one line permits at most 10 units, retains the last accepted value, shows a specific inline message, and does not change the total after rejection. The case makes the data route, trigger, visible result, durable check, and cleanup explicit.

id: TC-CART-142
version: 4
title: Reject a quantity above the per-line maximum
requirement: CART-204@rev-7
risk: An oversized line bypasses inventory and pricing rules
environmentProfile: commerce-regression-v3
owner: checkout-qa
preconditions:
  - Product MUG-ALPINE is enabled with perLineMaximum 10
  - Product MUG-ALPINE has at least 11 units in the test inventory
  - A new cart belongs to the reserved customer account
testData:
  sku: MUG-ALPINE
  acceptedQuantity: 10
  rejectedQuantity: 11
  currency: INR
  unitPriceMinor: 2500
  expectedAcceptedTotalMinor: 25000
steps:
  - action: Add MUG-ALPINE to the cart and set its quantity to 10
    expected: The line quantity is 10 and the cart total is INR 250.00
  - action: Replace the line quantity with 11 and submit the change
    expected: The line quantity remains 10 and an inline message says "Maximum quantity is 10"
  - action: Refresh the cart from the server
    expected: The stored line quantity is 10 and the cart total remains INR 250.00
cleanup:
  - Delete the cart through the Test Data Portal after evidence is collected
evidence:
  - Screenshot containing the line, quantity, message, and total
  - Cart identifier and server-rendered line state

The extra detail has a cost. If copy changes intentionally, the exact message needs maintenance. That is justified when the message itself is an acceptance criterion. If the requirement permits several messages, assert the approved meaning and placement rather than freezing one sentence that product never promised.

The second case depends on suite order. TC-INVITE-018 says Open the invitation created in TC-INVITE-017. A runner selects only the expiry case in a focused regression and has no URL. A parallel run consumes the shared invitation first. The failure alternates between blocked, already used, and expired depending on history.

The fix is not merely to copy TC-INVITE-017 into the precondition. State that this case needs an unused invitation whose expiry is in the past, name the approved fixture recipe that creates it, reserve a unique recipient, and record the invitation ID. Cleanup should remove the recipient and invitation if the environment supports it. If the product only allows expiry through a clock-controlled service, the environment profile must expose that capability rather than asking the runner to wait.

This case also exposes a near-miss. A veteran may replace the unavailable expired invitation with an already-used one and see the same generic Invalid invitation page. The visible text matches, but a different rule was exercised. Evidence must include the fixture state or reason code available through the approved inspection surface rather than only a screenshot of the shared error page.

The third case uses stale interface language. The step says Choose Submit claim, while the current build says Send for review. A familiar tester maps the names automatically. A new tester stops or selects Submit another claim elsewhere on the page. Updating the label fixes today's block, but the handoff also needs a maintenance trigger linked to the claims workflow and an owner who sees design changes.

Screenshots are useful supporting evidence and poor primary instructions. They age quickly, can show one locale or viewport, and encourage coordinate-based actions. Keep a crop when visual placement matters, but write the role, current label, section, and expected result in text. When copy is intentionally localized, identify the locale and use the approved translation for that environment.

Tell a product defect from a defective case

Begin triage with the frozen case revision and the runner's execution log. Identify the first point where observed state diverged from the written precondition, action, or expectation. Later failures may be consequences. If setup did not produce the required account role, there is no value debating the checkout result.

A case defect exists when the requirement is clear but the document omits or contradicts it. Examples include a stale label, a hidden flag, an unavailable data recipe, two expected outcomes, or cleanup that destroys required evidence. A product defect exists when the preconditions are true, the runner performs the documented action, and the observed behavior conflicts with the current requirement.

Environment failures have their own proof. Record build, deployment, feature configuration, integration status, browser or device where relevant, locale, and time zone. “Works on my machine” is not a verdict. Compare the failed environment profile with the case's required profile and isolate the mismatched capability.

Data failures are state failures, not generic blockers. Capture the record identifier and the fields that matter, without exposing personal or secret data. An account can look valid in the UI while its invitation is consumed or its regional eligibility differs. A reusable test-data portal should show reservation, state, owner, and cleanup status.

Use a known-good control when several layers remain plausible. For the quantity example, first submit the allowed boundary and confirm the fixture, page, and update route work. Then submit the value above the boundary. If both attempts fail before validation, the environment or setup owns the first problem. If the allowed value succeeds and the rejected value is accepted, the business rule is implicated. The control is useful because a product change can make either observation differ; it is not a check over values that the case itself hard-coded to agree.

Date cases need particular care. “Invitation expired yesterday” changes with clock, time zone, and execution date. Record the system clock source, the fixture's timestamp with offset, the user's zone where it affects display, and the boundary rule from the requirement. A screenshot showing Yesterday does not establish the server's expiry comparison. Preserve the stored timestamp or approved inspection result alongside the visible message.

When two cold runners disagree, compare the first diverging interpretation. If one reads Delete account as deleting a profile and another as disabling sign-in, the title or requirement link may be underspecified. If both execute the same state and see the same result but assign different verdicts, the expected result is ambiguous. If both reach the same failing verdict from independent data, confidence in a product defect increases, although shared misunderstanding of an obsolete requirement remains possible.

Consider a bug report rejected at triage:

“Express checkout total wrong. Add items, select express, and check total. Expected correct total. Actual wrong total.”

The developer cannot tell which items, quantities, currency, destination, promotion, build, or calculation rule applied. The report may describe a real defect, but it does not provide a reproducible comparison. “Cannot reproduce” in this triage means insufficient evidence, not proof that the system is correct.

The rewrite below uses an illustrative fixture. Its values are test inputs and expected calculations, not measurements from an experiment. It ties the report to the case revision and makes the first wrong transition visible.

title: Express shipping is omitted after the delivery state changes on review
case: TC-CHECKOUT-233@version-6
requirement: SHIP-88@rev-4
build: qa-2026.08.04.3
environment: commerce-regression-v3
locale: en-IN
customer: reserved fixture customer CUST-SHIP-17
cart:
  sku: COURSE-KIT
  quantity: 1
  subtotalMinor: 180000
steps:
  - Add COURSE-KIT and continue to delivery
  - Enter the fixture address in Karnataka and select Express
  - Continue to review, then change only the state to Maharashtra
  - Return to the review page
expected:
  shippingMinor: 12000
  totalMinor: 192000
  rule: Express fee is recalculated from the current delivery state
actual:
  shippingMinor: 0
  totalMinor: 180000
  storedShippingService: EXPRESS
evidence:
  - cart ID CART-SHIP-17
  - screenshot of the review breakdown
  - sanitized pricing decision showing state Maharashtra and no shipping rule
cleanup:
  - cart retained for triage and expires through the fixture policy

This report gives triage several ways to falsify the claim. The requirement can show that Express is free, the fixture can show a different state, the pricing decision can show a valid rule, or the stored order can show the UI alone is stale. A report that can be disproved by evidence is stronger than one whose expected result is always “correct.”

After fixing the case, use a fresh runner if practical. The original runner now knows the missing label, data source, or business rule and can pass from memory. If a fresh reviewer is unavailable, require the runner to follow only the revised text and note prior knowledge as a limitation.

Close triage with two linked outcomes when necessary. A product defect can coexist with a case defect, such as a checkout total error discovered through a stale label and undocumented fixture. Fixing only the document leaves the product wrong. Fixing only the product leaves the next runner dependent on coaching. Link both records to the frozen execution so future reviewers understand why one run created two pieces of work.

Do not convert every question into another paragraph. Repeated product concepts belong in an environment profile, glossary, data recipe, or workflow overview with versioning and ownership. The case should reference shared material and list its case-specific state. Copying a page of common setup into hundreds of cases creates hundreds of stale copies.

Migrate a living suite without freezing its mistakes

Inventory the suite by execution frequency, business risk, recent defects, ownership, age, and dependency. Start with cases used for releases, onboarding, regulated evidence, destructive workflows, and automation candidates. A rarely used low-risk case can wait, while a frequently coached payment case cannot.

Sample cold runs before changing the template. The questions runners ask reveal which fields the team genuinely needs. Adding every conceivable field produces forms that authors complete with Not applicable or copied boilerplate. Make a field mandatory only when its absence has caused or could plausibly cause a wrong execution or verdict.

Convert hidden dependencies into state descriptions and fixture recipes. Where a suite is an intentional journey, create a suite record that names order, shared artifacts, failure recovery, and cleanup. This preserves coherent end-to-end work without making individual cases lie about independence.

Assign owners by product area or component, not by the person who happened to write the case years ago. Trigger review from requirement and interface changes where tooling permits. Add a visible last-executed build and last-reviewed revision, but do not use age alone to declare a case correct.

Use explicit lifecycle states during rollout: draft, ready for cold run, changes requested, handoff approved, and retired. Approval belongs to a particular revision. Editing an expected result or setup after approval moves the case back to review, while a spelling correction can follow the team's lighter policy. Without revision-aware status, an approved badge survives changes nobody executed.

Pilot the template with one high-risk workflow and one routine workflow. The high-risk case tests whether evidence, data, and cleanup fields are strong enough. The routine case reveals whether the format is too expensive for ordinary regression. Review runner questions and author interventions from actual pilots before migrating the rest of the suite. Do not report a success percentage until the sample and classification rules are defined.

For distributed teams, test handoff across a real boundary where possible. A case that transfers inside one office may rely on shared calendars, network access, locale assumptions, or spoken knowledge. Give the revision to the team expected to own it next, confirm their environment access before the clock starts, and treat unavailable institutional access separately from missing case instructions.

Static validation can enforce structure during migration. The following Python program reads YAML cases, reports duplicate IDs, missing required fields, empty steps, and expected results that end on an evaluative word instead of an observable state, and exits nonzero when it finds a problem. It uses PyYAML's safe_load and expects that dependency to be declared by the repository. It deliberately does not claim to judge business correctness.

Write the vague-phrase rule against the cases you already know are bad, then run it against them. A short list of banned literals looks convincing and catches almost nothing: a pattern matching only works correctly, handled correctly, successful as expected, and verify success passes all three of the weak oracles quoted earlier in this article, because none of them contains any of those exact strings. The shape those three share is different. Each one ends on a word that grades the outcome rather than describing it: ... verify the validation works, ... invalid quantity correctly, ... the page works. Anchoring the evaluative vocabulary to the end of the expected result catches that shape while leaving concrete results alone, since a result naming a quantity, a message, an HTTP code, or a heading ends on the observable thing rather than on the verdict.

from __future__ import annotations

import re
import sys
from pathlib import Path

import yaml

REQUIRED = {
    "id",
    "version",
    "title",
    "requirement",
    "owner",
    "preconditions",
    "steps",
    "cleanup",
}
# An expected result that ends on an evaluative word names no observable state,
# which is the shape of every vague oracle above. A few phrases are vague
# wherever they appear, so they are matched without the end anchor.
VAGUE = re.compile(
    r"\b(?:works?|working|worked|correctly|properly|successfully"
    r"|as\s+expected|as\s+designed"
    r"|without\s+(?:any\s+)?(?:issues?|errors?|problems?))"
    r"\s*[.!]*$"
    r"|\b(?:works correctly|handled correctly|successful as expected"
    r"|verify success|no (?:issues?|errors?|problems?))\b",
    re.IGNORECASE,
)


def lint_case(path: Path) -> tuple[str | None, list[str]]:
    data = yaml.safe_load(path.read_text(encoding="utf-8"))
    errors: list[str] = []
    if not isinstance(data, dict):
        return None, ["document must contain one mapping"]

    missing = sorted(REQUIRED - data.keys())
    if missing:
        errors.append("missing fields: " + ", ".join(missing))

    steps = data.get("steps")
    if not isinstance(steps, list) or not steps:
        errors.append("steps must be a non-empty list")
    else:
        for index, step in enumerate(steps, start=1):
            if not isinstance(step, dict) or not step.get("action"):
                errors.append(f"step {index} needs an action")
                continue
            expected = str(step.get("expected", "")).strip()
            if not expected:
                errors.append(f"step {index} needs an expected result")
            elif VAGUE.search(expected):
                errors.append(
                    f"step {index} has a vague expected result: {expected!r}"
                )

    case_id = data.get("id")
    return str(case_id) if case_id else None, errors


def main(root: Path) -> int:
    seen: dict[str, Path] = {}
    failed = False
    paths = sorted(root.rglob("*.yaml"))
    if not paths:
        print(f"{root}: no YAML case files found", file=sys.stderr)
        return 1

    for path in paths:
        case_id, errors = lint_case(path)
        if case_id in seen:
            errors.append("duplicate id also used by " + str(seen[case_id]))
        elif case_id:
            seen[case_id] = path

        for error in errors:
            failed = True
            print(f"{path}: {error}", file=sys.stderr)
    return 1 if failed else 0


if __name__ == "__main__":
    raise SystemExit(main(Path(sys.argv[1])))

Every branch can fail against a real document change. Removing an expected result, duplicating an ID, or ending a step on a verdict rather than an observation produces an error. Prove it on the two documents this article already contains: the rewritten TC-CART-142 above exits 0, and the original version, whose three steps expect the validation to work, the quantity to be handled correctly, and the page to work, exits 1 and names all three steps. The script does not compare a hard-coded fixture with itself.

Be precise about the remaining gap, because a linter that is oversold gets trusted past its evidence. The end anchor is what keeps false positives low, and it is also the limit: The validation works correctly and the total updates ends on a concrete clause, so only the literal-phrase half of the pattern catches it, and a sentence that buries a verdict in the middle without using one of those literals passes. The check also cannot detect that 10 is the wrong maximum, that a requirement link points to obsolete meaning, or that an expected result is specific but wrong. Static analysis narrows what a reviewer must read. It does not replace the cold-runner trial.

Wire the linter to changed case files and keep semantic approval in the review workflow:

name: manual-case-contract

on:
  pull_request:
    paths:
      - "content/manual-cases/**"
      - "tools/manual-case-lint/**"

permissions:
  contents: read

jobs:
  lint-cases:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip
          cache-dependency-path: tools/manual-case-lint/requirements.txt
      - run: python -m pip install -r tools/manual-case-lint/requirements.txt
      - run: python tools/manual-case-lint/lint_cases.py content/manual-cases

The migration costs author and reviewer time. Independent dry runs consume environments and data. Explicit expected results require requirement access. Ownership creates maintenance work. Offset those costs with risk-based depth, shared versioned setup, and retirement of duplicate or obsolete cases. Do not compensate by turning every case into a novel.

Track handoff evidence directly: author interventions, runner questions, blocked setup, unplanned assumptions, verdict disagreements, and corrections by category. These are observations from actual reviews. Report counts only when collected from a defined sample and period. Do not invent a target improvement and present it as measured team performance.

When formal handoff review is the wrong level of control

Exploratory charters should leave room for investigation. Forcing every observation into numbered steps before learning the system can destroy the point of exploration. Hand off the mission, risks, data boundaries, time box, and evidence expectations, then convert stable regressions into cases after useful behavior is known.

A short-lived prototype may change faster than detailed cases can be maintained. Use acceptance examples and a focused risk checklist until the interaction settles. Critical safety, money, privacy, and migration risks still deserve explicit oracles even during rapid change.

Do not split one coherent rule into tiny cases merely to satisfy one behavior per case. A state transition can require several observations to establish one verdict. Split when failures have different causes, setup, owners, or recovery, not at every click.

Avoid replacing current written instructions with screenshots. Images can support visual assertions, but they hide labels from search, become stale, and may not match locale or viewport. Keep the executable meaning in text and version any reference image.

Do not treat a green linter as handoff approval. Structure is necessary and insufficient. A file can contain every field while its data is impossible, its requirement is wrong, and its expected result permits disagreement. The independent dry run remains the proof of transfer.

Finally, retire cases instead of preserving them as historical clutter in the active suite. Archive the last revision, reason, linked requirement, and replacement where one exists. A smaller owned suite gives a new tester a better handoff than thousands of apparently complete cases that veterans know not to trust.