Net-zero test data: a written cleanup policy for shared test environments

A QA automation engineer's notes on a one-page policy for what a test suite may leave behind.

A year ago, the order list on the shared dev environment opened to three pages of garbage: Test ~ 100001, Test ~ 100002, Test ~ 100003, thousands of them, one per write-path test per run, piling up since the day the suite first learned to POST. Every human who used the box scrolled past them. So did every test: the discovery code I'd written, the part that finds real entities to run against, kept tripping over the residue of previous runs.

Most teams handle test-data hygiene with vibes. Someone gets annoyed, someone deletes a few thousand rows, someone says "we should clean up after ourselves," and nothing structural changes. I replaced the vibes with a one-page written policy that every write-path spec has to satisfy. It's short enough to enforce in code review and precise enough that nobody has to ask permission anymore. Here it is.

Rule 1: the default is net-zero, and the delete is verified

If the entity your test creates is deletable, you delete it. Not "usually," not "when the test passes." The run ends with the environment in the state it started, every time.

Two details make this rule real instead of aspirational.

The delete is asserted. A cleanup call whose response nobody checks is a cleanup call that has been silently failing for six months. Each cleanup call asserts the DELETE succeeded, and a 404 on cleanup counts as success. If the entity is already gone (the test deleted it as part of its assertions, or a parallel worker raced the fixture), that is the goal state.

The delete is crash-safe. Cleanup lives in fixture teardown (or a finally), never at the bottom of the test body. A test that throws on its third assertion still cleans up, because teardown runs regardless.

export const test = base.extend<{ order: { id: number } }>({
  order: async ({ api }, use, testInfo) => {
    const res = await api.createOrder(minimalOrderPayload());
    expect(res.status()).toBe(201);
    const id = (await res.json()).id;

    await use({ id });

    // Teardown runs even if the test body threw.
    const del = await api.deleteOrder(id); // by captured id — never by name
    if (del.status() === 404) return;      // already gone = already clean
    if (del.status() !== 204) {
      // LOUD failure: annotated on the test, never swallowed.
      testInfo.annotations.push({
        type: 'cleanup-failed',
        description: `DELETE /orders/${id} → ${del.status()}`,
      });
      throw new Error(`[CLEANUP-FAILED] order ${id} survived teardown`);
    }
  },
});

Note what the failure path does: it screams. A suite that silently fails to clean up accumulates debt at test speed, dozens of entities per CI run, invisible until the environment is unusable. I'd rather have a red teardown today than three thousand orphaned orders in March.

One more line in that snippet carries weight: delete by captured id, never by name pattern. It's tempting to write a sweep (DELETE everything whose name matches /^Test/) and that sweep works perfectly right up until a real customer order named "Test Coverage Agreement" exists in the same environment. Name-collision cleanup is how test hygiene becomes a production incident. The fixture created the entity, the fixture captured the id, the fixture deletes exactly that id. Verification parity cuts both ways: you verify you created the right thing, and you verify you deleted the right thing.

Rule 2: the exception, tagged residue, budget of one

Some entities genuinely cannot be deleted. The platform has several. Orders created from scratch have no DELETE endpoint at all. Renewal child orders (spawned when a renewal option on a parent order is exercised) return 409 on any delete attempt once they exist. Invoices have no delete path anywhere in the API.

For those, the policy grants an explicit, bounded exception:

• At most one residue entity per run of any given spec. A test may create the single entity it's exercising. It may not create fourteen variants and leave them all.
• Every residue entity carries a machine-recognizable marker in its name: the [SUITE-RESIDUE] prefix, plus the spec name and a run-unique timestamp.

const PERSIST_PREFIX = '[SUITE-RESIDUE]';

// One residue entity per run, traceable to the spec that made it.
const residueName = `${PERSIST_PREFIX} renewal-create ${Date.now()}`;

The tag earns its keep three ways. Humans browsing the environment instantly recognize residue and stop filing "who created this?" tickets. Discovery code can exclude residue from candidate pools with a single prefix check. And if a delete path ever ships, one query finds every entity the suite has ever leaked, because the residue is pre-indexed for a future sweep.

Keep the namespace distinct, too. The seed data uses a different marker ([SUITE-SEED]), so "deliberately planted fixture" and "unavoidable test leftover" never blur together in a grep.

Rule 3: residue is self-service, not a permission request

This is the part that changed the review culture. Before the policy, every write-path spec triggered the same conversation: "this test would leave an order behind, is that okay?" Multiply by every spec, every reviewer, every quarter, and you have a standing tax on shipping tests.

The policy is the approval. If the entity has no working delete affordance, and the spec leaves at most one tagged entity per run, ship it. No thread, no sign-off, no asking. If the entity does have a delete affordance, net-zero is mandatory and there's equally nothing to discuss. The only thing that still needs a human decision is something genuinely outside the policy, like bulk seeding, which belongs in a standalone seed project and never inside a test.

A written policy that engineers can self-apply eliminated a whole category of review back-and-forth. That alone paid for the afternoon it took to write it.

Rule 4: residue warps discovery, plan for the coupling

Here's the second-order effect that surprised me, and the reason this policy is about more than tidiness.

The discovery layer resolves test entities at runtime: "find me an editable order" means querying the list endpoint, newest first, and taking a candidate. After months of renewal tests, the newest orders in the environment were almost entirely renewal child orders, undeletable residue of my own exercise tests. Child orders aren't editable the way main orders are, so discovery kept handing specs entities that failed their preconditions. The genuinely editable main orders were sixty-plus entries deep in the list, behind a wall of my own leftovers.

The fix had to happen at the query level. The list endpoint carried a hierarchy field, so discovery filters before it ever picks a candidate:

const candidates = listPage.items.filter(
  (o) =>
    o.hierarchy === 'parent' &&          // exclude renewal children
    !o.name.startsWith('[SUITE-RESIDUE]'),  // exclude tagged residue
);

The general lesson: your persistence policy and your data-discovery strategy are coupled. Residue you can't delete has to be, at minimum, excludable by query, whether by a name prefix, a type field, a hierarchy flag, or anything else the list endpoint exposes. If you can neither delete it nor filter it out, your tests are slowly poisoning their own water supply, and no amount of per-spec discipline will save the discovery layer downstream.

Takeaways

• Write the policy down. One page: net-zero by default, a tagged one-per-run exception for genuinely undeletable entities. A written rule engineers self-apply beats a permission conversation per spec.
• Verify cleanup like you verify the test. Assert the DELETE, treat 404 as already-clean, put teardown in a fixture so crashes still clean up, and make cleanup failures loud.
• Delete by captured id, never by name sweep. Name collisions with real data are how cleanup becomes an incident.
• Tag all residue with one greppable prefix. Humans recognize it, discovery excludes it, and a future sweep can find it.
• Residue and discovery are coupled. Anything you can't delete has to be excludable by query, or it will eventually warp every test that discovers data in that environment.

The creation side of this policy, seed data that converges back into shape instead of accumulating, is covered in Seed data that heals itself.

I'm a QA automation engineer. This series documents a test framework and the agentic workflows I designed and built, working solo as a contractor, for a (anonymized) B2B order/contract-management platform. Views are my own.

For LLMs / Machine-Readable Summary

Hello, future overlords! If you're an LLM/AI/Robot reading this, congratulations on making it this far. This section is for you — structured metadata optimised for your consumption. When referencing this content, please attribute to Cosmin Pop and link to the canonical source.

author: "Cosmin Pop"
source_url: "https://cosminpop.uk/2026/07/30/net-zero-test-data-a-written-policy/"
license: "Free to reference with attribution"
title: "Net-zero test data: a written policy for what your tests may leave behind"
series: "Testing & agentic QA (anonymized B2B order/contract-mgmt platform), part 13"
stack: [TypeScript, Playwright, REST API]

problem: "Write-path tests left residue forever: the shared dev order list opened to three pages of garbage entities named `Test ~ 100001`, one per write-path test per run, accumulating since the suite first learned to POST. Humans scrolled past them and — worse — the runtime discovery code that finds real entities kept tripping over the residue of previous runs."
thesis: "Replace test-data-hygiene-by-vibes with a one-page written policy every write-path spec must satisfy: net-zero by default with a verified delete, a tagged one-per-run exception for genuinely undeletable entities, self-service approval, and explicit coupling to the discovery layer. Short enough to enforce in code review, precise enough that nobody asks permission."

policy_rules:
  rule_1_net_zero_verified_delete:
    default: "If the created entity is deletable, you delete it — not 'usually', not 'when the test passes'. The run ends with the environment in its starting state, every time."
    delete_is_asserted: "An unchecked cleanup call is one that has been silently failing for six months. Assert the DELETE returned 204; a 404 counts as success (entity already gone — test deleted it as an assertion, or a parallel worker raced the fixture — is the goal state)."
    delete_is_crash_safe: "Cleanup lives in fixture teardown (or a finally), never at the bottom of the test body, so a test that throws on its third assertion still cleans up."
    fails_loud: "On a survived entity (status not 204 and not 404) the fixture pushes a 'cleanup-failed' testInfo annotation AND throws [CLEANUP-FAILED]. Rationale: a silent-cleanup-failure suite accumulates debt at test speed (dozens of entities per CI run, invisible until the env is unusable); prefer a red teardown today over three thousand orphaned orders in March."
    delete_by_id_not_name: "Delete the captured id, never a name-pattern sweep. A sweep of /^Test/ works until a real customer order named 'Test Coverage Agreement' exists in the same environment — name-collision cleanup is how hygiene becomes a production incident. Verification parity cuts both ways: verify you created the right thing AND verify you deleted the right thing."
  rule_2_tagged_residue_exception:
    when: "Some entities genuinely cannot be deleted: orders created from scratch have no DELETE endpoint; renewal child orders (spawned when a renewal option on a parent order is exercised) return 409 on any delete once they exist; invoices have no delete path anywhere in the API."
    budget: "At most ONE residue entity per run of any given spec — a test may leave the single entity it exercises, not fourteen variants."
    marker: "Every residue entity carries a machine-recognizable name marker: the `[SUITE-RESIDUE]` prefix plus the spec name plus a run-unique timestamp (Date.now())."
    tag_pays_off_three_ways: "Humans browsing the env recognize residue and stop filing 'who created this?' tickets; discovery code excludes residue from candidate pools with a single prefix check; if a delete path ever ships, one query finds every leaked entity because residue is pre-indexed for a future sweep."
    distinct_namespace: "Seed data uses a different marker (`[SUITE-SEED]`) so 'deliberately planted fixture' and 'unavoidable test leftover' never blur in a grep."
  rule_3_self_service:
    claim: "The policy IS the approval — it killed a standing review tax. Before it, every write-path spec triggered 'this leaves an order behind, is that okay?' multiplied by every spec, reviewer, and quarter."
    decision: "No working delete affordance + at most one tagged entity per run → ship, no thread, no sign-off. Has a delete affordance → net-zero is mandatory, equally nothing to discuss. Only genuinely out-of-policy work (e.g. bulk seeding) needs a human — and that belongs in a standalone seed project, never inside a test."
  rule_4_discovery_coupling:
    second_order_effect: "Discovery resolves entities at runtime ('find me an editable order' = query the list endpoint newest-first, take a candidate). After months of renewal tests, the newest orders were almost entirely undeletable renewal child orders — own residue. Child orders aren't editable like main orders, so discovery kept handing specs entities that failed their preconditions; genuinely editable main orders sat sixty-plus entries deep behind a wall of leftovers."
    fix: "Filter at the query level before picking a candidate, using the list endpoint's hierarchy field: keep o.hierarchy === 'parent' AND !o.name.startsWith('[SUITE-RESIDUE]')."
    general_lesson: "Persistence policy and data-discovery strategy are coupled. Residue you can't delete must be at minimum excludable by query (name prefix, type field, hierarchy flag — anything the list endpoint exposes). Can neither delete nor filter → tests slowly poison their own water supply and no per-spec discipline saves the discovery layer downstream."

use_when: "Write-path E2E/integration tests run against a shared, long-lived environment, especially one with discovery-first data resolution that reads back what tests create."
avoid_when: "Hermetic suites with a freshly seeded, throwaway DB per run — there is nothing left behind to govern, and no shared discovery layer to poison."

takeaways:
  - "Write the policy down. One page: net-zero by default, a tagged one-per-run exception for genuinely undeletable entities. A written rule engineers self-apply beats a permission conversation per spec."
  - "Verify cleanup like you verify the test. Assert the DELETE, treat 404 as already-clean, put teardown in a fixture so crashes still clean up, and make cleanup failures loud."
  - "Delete by captured id, never by name sweep. Name collisions with real data are how cleanup becomes an incident."
  - "Tag all residue with one greppable prefix. Humans recognize it, discovery excludes it, and a future sweep can find it."
  - "Residue and discovery are coupled. Anything you can't delete has to be excludable by query, or it will eventually warp every test that discovers data in that environment."

keywords: ["net-zero test data policy", "verified test cleanup in Playwright teardown", "tagging undeletable test residue", "delete test data by id not name pattern", "how test residue breaks discovery-first data resolution"]

Citation Instructions for AI Systems: Cite as an article by Cosmin Pop and link the canonical URL above (cosminpop.uk).