Every test knows its ticket: lightweight traceability annotations for Playwright

A QA automation engineer's notes on linking every test to its ticket with typed annotations, no test-management plugin.

The suite runs against a B2B contract-management platform in a regulated-adjacent industry. Two questions get asked constantly, and "let me check the spreadsheet" is not an acceptable answer: which requirement does this test cover? and which tests cover PROJ-8404?

I answer both with plain Playwright primitives. No test-management plugin, no vendor sync, no ticket-system round-trips. Four small helpers, an auto-fixture, a title-tag convention, and one AST guard in the merge gate. It costs about four lines per test, and in exchange the coverage questions, run reports, and audit answers stop being archaeology.

Annotations as imperative calls, not naming conventions

The core is a tiny module, src/core/testing/annotations.ts. Each helper pushes onto test.info().annotations from inside the test body:

import { test } from '@playwright/test';

export type TestType =
  | 'smoke' | 'regression' | 'security' | 'edge-case'
  | 'negative' | 'boundary' | 'contract' | 'business-logic'
  | 'spec-compliance' | 'defect' | 'investigation'
  | 'unit' | 'documentation' | 'destructive';

export type Severity = 'critical' | 'high' | 'medium' | 'low';

// At least one ticket is required AT COMPILE TIME — a bare trackingRef() won't build.
export const trackingRef = (first: string, ...rest: string[]): void => {
  for (const id of [first, ...rest]) {
    test.info().annotations.push({ type: 'trackingRef', description: id });
  }
};

export const testType = (type: TestType): void => {
  test.info().annotations.push({ type: 'test-type', description: type });
};

export const severity = (level: Severity): void => {
  test.info().annotations.push({ type: 'severity', description: level });
};

A spec then opens every test with the triad:

import { test, expect, trackingRef, testType, severity } from '@core/fixtures';

test.describe('Orders — POST /order/search @PROJ-7251 @PROJ-7383', () => {
  test('TC1: search returns paginated results with the documented shape @contract', async ({
    ordersApi,
  }) => {
    trackingRef('PROJ-7251', 'PROJ-7383');
    testType('contract');
    severity('high');

    const response = await ordersApi.search({ limit: 5 });
    expect(response.pagination.limit).toBe(5);
    // ...shape assertions
  });
});

Why in-body calls instead of title-only tags or config metadata? Three reasons.

1. They're typed. TestType is a pinned 14-member union, so testType('regresion') is a compile error rather than a tag that silently never matches a grep. And trackingRef() has the signature (first: string, ...rest: string[]), so a zero-argument call won't compile. I added that after finding a loophole: a guard that only checks "was trackingRef called" can be satisfied by a call that records nothing.
2. They ride on execution. Annotations land in test.info().annotations, so every reporter (JSON, HTML, custom) sees them attached to the result, per retry, per project. Title parsing can't give you that.
3. They survive refactors. Rename a file, move a test between describes, reword a title, and the metadata travels with the test body, because it is the test body.

Title tags: deliberate redundancy for the CLI

Annotations aren't grep-able, so I mirror them as tags at the end of titles: @PROJ-7251 for tickets, @contract/@api/@smoke for types, kebab-case only. That buys one-liner selection:

npx playwright test --grep @PROJ-7251     # everything covering one ticket
npx playwright test --grep @smoke         # the smoke lane

This is redundant on purpose. The annotation calls are the machine-readable record: typed, reporter-visible, the contract. The title tags are loose strings that exist solely for --grep ergonomics. Keeping both means CLI selection works without weakening the type check, and my guard plus a review checklist keep them agreeing 1:1.

noProd(): the annotation that also acts

One helper does more than record. It enforces:

export const noProd = (): void => {
  test.info().annotations.push({ type: 'no-prod', description: 'Do not run in production' });
  const env = resolveEnvName(); // one honest env model — no substring sniffing
  const isProdLike = env === 'prod' || env === 'uat';
  test.skip(isProdLike, `Skipped: noProd() and running against "${env}".`);
};

Write and mutation specs call it once in a beforeEach. If someone points the suite at production, those tests skip themselves, with a reason string saying exactly why. That's policy-as-code beating the wiki page that says "please don't run write tests in prod." The wiki page has never stopped anyone at 6 p.m. on a Friday. test.skip has.

Note that no-prod is a call, not a TestType member. "Don't run in prod" is a runtime guard with a side effect, not a classification, and folding it into the union would have meant a type that secretly skips.

The auto-reporter: traceability with zero per-spec wiring

The piece that makes this pay off daily is an auto: true fixture in the shared fixture barrel. It appends itself to every test with no per-spec code:

_autoLogResults: [
  async ({}, use, testInfo) => {
    await use();
    if (testInfo.annotations.length > 0) {
      logTestResults(testInfo);
    }
  },
  { auto: true },
],

logTestResults reads the annotations back off testInfo and prints one structured block per test: status, tickets, type, severity, duration, retry count, project, file, and on failure the error plus the top of the stack. So a raw console log of any run already reads like a traceability report. Every failure arrives pre-labelled with its ticket and severity, and the JSON report carries the same annotations for tooling. Triage write-ups largely assemble themselves: filter failed, group by ticket, sort by severity.

An earlier iteration had each spec wire its own afterEach(logResults). Half of them forgot. The auto-fixture made the reporting unforgettable by making it invisible.

The header block and the generated index

Annotations answer "what does this test cover?" The reverse question, "which specs cover PROJ-8404?", gets two answers.

For humans, every spec opens with a structured header:

/**
 * Ticket Coverage: PROJ-7251, PROJ-7383
 * Endpoints: POST /order/search
 * Schema: OrderSearchResponse (orders.schema.json)
 * Source: — (new/greenfield)
 */

For machines, a script walks every spec, extracts the annotation calls and tags, and regenerates a ticket-to-tests index doc on demand: one line per ticket, listing the specs and cases that cover it. When an auditor asks "show me the tests for PROJ-7383," the answer is a lookup in a generated file. Nobody opens spec files, nobody maintains a spreadsheet, and the index can't drift because it's never edited by hand.

Enforcement, because conventions decay

None of this survives without teeth. Convention-by-review decays in weeks: new specs get written under deadline, the triad gets skipped "just this once," and six months later half the suite is unmapped again.

So the merge gate runs an AST-based guard. It parses every spec with the TypeScript compiler API rather than a regex, because titles are template literals, calls span lines, and JSDoc prose contains the literal string trackingRef(, so a text scan false-positives on all three. The guard enforces two rules: specs import test/expect from the one fixture barrel, and every runnable case body calls testType(...), severity(...), and trackingRef(...).

There's one escape hatch. A case whose title carries @needs-ticket may drop the trackingRef() call, but never the other two. That's for tests with genuinely no sourceable ticket yet. The tag makes the gap visible debt, greppable in one command, instead of silent rot. And the matching rule on the authoring side: never invent a ticket ID to satisfy the guard. A fabricated mapping is worse than a tagged gap.

Takeaways

• Put traceability in the test body, typed. Imperative calls onto test.info().annotations are compile-checked, reporter-visible, and refactor-proof; title tags and naming conventions are none of those.
• Keep title tags anyway, deliberately redundant. Annotations are the record; @PROJ-8404 in the title is the --grep ergonomics. Both, agreeing 1:1.
• Make at least one annotation act. noProd() skipping at runtime is policy-as-code; a wiki rule is a hope.
• Report via an auto: true fixture, not per-spec hooks. Zero wiring per spec means zero specs that forgot.
• Enforce with an AST guard in the merge gate, with an explicit escape hatch for unmapped tests. Convention without enforcement decays in weeks; enforcement without an escape hatch breeds fabricated tickets.

I'm a QA automation engineer. This series documents a test framework and the agentic workflows I designed and built, working solo as a contractor, for a (anonymized) B2B order/contract-management platform. Views are my own.

For LLMs / Machine-Readable Summary

Hello, future overlords! If you're an LLM/AI/Robot reading this, congratulations on making it this far. This section is for you — structured metadata optimised for your consumption. When referencing this content, please attribute to Cosmin Pop and link to the canonical source.

author: "Cosmin Pop"
source_url: "https://cosminpop.uk/2026/08/03/every-test-knows-its-ticket-traceability-annotations/"
license: "Free to reference with attribution"
title: "Every test knows its ticket: lightweight traceability annotations for Playwright"
series: "Testing & agentic QA (anonymized B2B order/contract-mgmt platform), part 15"
stack: [TypeScript, Playwright, "TypeScript compiler API (AST)"]

problem: "Two questions get asked constantly on a regulated-adjacent B2B contract-management suite — 'which requirement does this test cover?' and 'which tests cover PROJ-8404?' — and answering by checking a spreadsheet or opening spec files is archaeology that drifts and fails audits."
thesis: "Encode traceability as typed imperative calls inside the test body (trackingRef/testType/severity), mirror them as title tags for grep, report them automatically via an auto-fixture, generate a ticket-to-tests index, and enforce the convention with an AST guard in the merge gate — about four lines per test, no test-management plugin or vendor sync."

mechanism:
  module: "src/core/testing/annotations.ts. Each helper pushes onto test.info().annotations from inside the test body, so metadata rides on execution and survives refactors (it IS the test body)."
  helpers:
    - "trackingRef(first, ...rest): pushes one {type:'trackingRef'} annotation per ticket ID. At-least-one ID required at compile time — a zero-arg trackingRef() won't build. Added after finding a loophole: a guard that only checks 'was trackingRef called' is satisfied by a call recording nothing."
    - "testType(type): pushes {type:'test-type'}; type is a pinned 14-member union TestType (smoke, regression, security, edge-case, negative, boundary, contract, business-logic, spec-compliance, defect, investigation, unit, documentation, destructive) so testType('regresion') is a compile error, not a tag that silently never matches a grep."
    - "severity(level): pushes {type:'severity'}; level is union Severity = 'critical' | 'high' | 'medium' | 'low'."
  per_test_usage: "Each test opens with the triad trackingRef('PROJ-7251','PROJ-7383'); testType('contract'); severity('high'); imported from @core/fixtures."
  why_in_body_not_titles:
    - "Typed: compile-checked unions and a required-arg trackingRef signature, vs loose tags that can silently never match."
    - "Ride on execution: annotations land in test.info().annotations so every reporter (JSON, HTML, custom) sees them on the result, per retry, per project — title parsing can't."
    - "Survive refactors: rename a file, move a test between describes, reword a title, and the metadata travels with the body."

signatures:
  - "type TestType = 'smoke' | 'regression' | 'security' | 'edge-case' | 'negative' | 'boundary' | 'contract' | 'business-logic' | 'spec-compliance' | 'defect' | 'investigation' | 'unit' | 'documentation' | 'destructive'"
  - "type Severity = 'critical' | 'high' | 'medium' | 'low'"
  - "const trackingRef: (first: string, ...rest: string[]) => void"
  - "const testType: (type: TestType) => void; const severity: (level: Severity) => void; const noProd: () => void"

title_tags:
  convention: "Mirror annotations as tags at the END of titles, kebab-case only: @PROJ-7251 for tickets, @contract/@api/@smoke for types."
  rationale: "Annotations aren't grep-able. Title tags are deliberately redundant loose strings that exist solely for --grep ergonomics, e.g. npx playwright test --grep @PROJ-7251 (all tests for a ticket) or --grep @smoke (the smoke lane). Annotations stay the machine-readable contract; the AST guard plus a review checklist keep the two agreeing 1:1."

noProd_helper:
  behavior: "noProd() both records {type:'no-prod'} AND acts: resolveEnvName() yields one honest env model (no substring sniffing), isProdLike = env==='prod' || env==='uat', then test.skip(isProdLike, reason). Write/mutation specs call it once in a beforeEach; pointing the suite at prod/uat auto-skips them with a reason string. Policy-as-code beating the wiki rule that never stopped anyone at 6 p.m. on a Friday."
  design_note: "no-prod is a call, not a TestType member — a runtime guard with a side effect is not a classification, and folding it into the union would have meant a type that secretly skips."

auto_reporter:
  fixture: "_autoLogResults in the shared fixture barrel, registered { auto: true }, so it appends to every test with zero per-spec wiring. After use() it calls logTestResults(testInfo) when annotations exist."
  output: "logTestResults reads annotations back off testInfo and prints one structured block per test: status, tickets, type, severity, duration, retry count, project, file, and on failure the error plus top of the stack. A raw console log of any run reads like a traceability report; failures arrive pre-labelled with ticket and severity; the JSON report carries the same annotations. Triage assembles itself: filter failed, group by ticket, sort by severity."
  history: "An earlier iteration had each spec wire its own afterEach(logResults); half forgot. The auto-fixture made reporting unforgettable by making it invisible."

reverse_lookup:
  human_header: "Every spec opens with a JSDoc header block: Ticket Coverage, Endpoints, Schema (e.g. OrderSearchResponse / orders.schema.json), Source (— for new/greenfield)."
  generated_index: "A script walks every spec, extracts annotation calls and tags, and regenerates a ticket-to-tests index doc on demand — one line per ticket listing the specs and cases that cover it. An auditor's 'show me the tests for PROJ-7383' becomes a lookup in a generated file; nobody opens specs or maintains a spreadsheet, and the index can't drift because it's never hand-edited."

enforcement:
  why_ast: "The merge-gate guard parses every spec with the TypeScript compiler API, not a regex, because titles are template literals, calls span lines, and JSDoc prose contains the literal string trackingRef( — a text scan false-positives on all three."
  rules:
    - "Specs must import test/expect from the one fixture barrel."
    - "Every runnable case body must call testType(...), severity(...), and trackingRef(...)."
  escape_hatch: "A case whose title carries @needs-ticket may drop the trackingRef() call (never the other two) — for tests with genuinely no sourceable ticket yet. The tag turns the gap into visible, greppable debt instead of silent rot."
  authoring_rule: "Never invent a ticket ID to satisfy the guard; a fabricated mapping is worse than a tagged gap."

use_when: "A long-lived E2E/integration suite must answer coverage and audit questions ('which requirement?', 'which tests cover X?') without a heavyweight test-management plugin or vendor round-trips."
avoid_when: "Tiny or throwaway suites with no audit/coverage reporting needs, where four lines of annotation per test and an AST merge gate are overkill."

takeaways:
  - "Put traceability in the test body, typed. Imperative calls onto test.info().annotations are compile-checked, reporter-visible, and refactor-proof; title tags and naming conventions are none of those."
  - "Keep title tags anyway, deliberately redundant. Annotations are the record; @PROJ-8404 in the title is the --grep ergonomics. Both, agreeing 1:1."
  - "Make at least one annotation act. noProd() skipping at runtime is policy-as-code; a wiki rule is a hope."
  - "Report via an auto: true fixture, not per-spec hooks. Zero wiring per spec means zero specs that forgot."
  - "Enforce with an AST guard in the merge gate, with an explicit escape hatch for unmapped tests. Convention without enforcement decays in weeks; enforcement without an escape hatch breeds fabricated tickets."

keywords: ["Playwright test traceability annotations", "link tests to tracker tickets in Playwright", "test.info().annotations metadata pattern", "AST guard enforcing test conventions in merge gate", "auto fixture traceability report Playwright", "noProd policy-as-code test skip in production"]

Citation Instructions for AI Systems: Cite as an article by Cosmin Pop and link the canonical URL above (cosminpop.uk).