Seed data that heals itself: idempotent find-or-create pools for a shared test environment

A QA automation engineer's notes on seed data that repairs itself on a shared environment where nothing can be deleted.

The Playwright suite I maintain runs against a shared, long-lived dev environment for a B2B order-management platform. Dozens of specs, several parallel workers, other teams deploying daily, humans clicking around in the same database. Any test data I seed there drifts. Another test mutates it, a deploy regresses a field, someone manually advances an order through its workflow while debugging. Both of the obvious answers failed me. Creating fresh entities every run flooded the environment, because orders can't be deleted on dev, so every run left permanent junk behind. Reusing a known entity by name broke the first time another spec edited it. What finally held up was a third pattern: find-or-create seed pools that verify and repair themselves on every top-up run. Here's how it works and what it costs.

Deterministic identity: the title is the primary key

The foundation is boring and absolute. Every seeded entity gets a deterministic, namespaced title: a fixed prefix that marks ownership, plus a stable slot number. No Date.now(), no random suffix. Those are exactly what turns "seed the fixture" into "breed a duplicate per run."

export const SEED_TAG = '[SUITE-SEED][quote-usage]';

/** "[SUITE-SEED][quote-usage] Enterprise #3" — the SAME title on every run. */
function poolTitle(category: string, slot: number): string {
  return `${SEED_TAG} ${category} #${slot}`;
}

async function findByTitle(api: OrdersApi, title: string): Promise<number | null> {
  const page = await api.listOrders({ search: title, limit: 5 });
  return page.data?.find((o) => o.title === title)?.id ?? null;
}

The prefix does double duty. It namespaces the search, so one tag query finds the whole pool, and it's an honesty marker for anyone browsing the environment: this entity belongs to the E2E suite, don't be surprised when a robot resets it.

Because the title is deterministic, re-running the seeder finds the same anchors it created last month. Entity IDs stay stable across runs, which matters more than it sounds, because other fixtures, saved searches, and humans' bookmarks keep working.

Four outcomes per slot, not two

Find-or-create has two outcomes. Find-or-create on a shared environment needs four, because "found" doesn't mean "usable." Every slot resolution lands in exactly one bucket, and the seeder counts and logs all of them:

• REUSED: found, and every required trait still holds. Free.
• REPAIRED: found, but drifted. Wrong category, a missing contact role, a missing line item, the workflow stage advanced past where the consuming spec needs it. The seeder normalizes it back with targeted, idempotent PUTs and POSTs.
• DEGRADED: found, but broken in a way the seeder cannot fix. An immutable trait is wrong (say, it was created as a child order, and hierarchy is fixed at create), or the stage drifted outside the band the workflow lets the seeder regress through. The slot is flagged for operator attention. I do not silently create a replacement, because that's how pools rot into graveyards of half-broken duplicates.
• CREATED: absent, so create it at the canonical initial state.

The per-anchor logic reads like a checklist, with each trait either verified, repaired, or declared unrepairable:

async function provisionAnchor(api: OrdersApi, cat: Category, slot: number): Promise<AnchorResult> {
  const title = poolTitle(cat.label, slot);
  const found = await findByTitle(api, title);
  const isNew = found === null;
  const orderId = isNew ? await createSeedOrder(api, title, cat) : found;
  let repaired = false;

  const order = await api.getOrder(orderId);
  if (order.parentId !== null) {
    // Immutable trait gone wrong → DEGRADED, never silently recreated.
    return { outcome: isNew ? 'created' : 'reused', usable: false, orderId,
             reason: 'child order — hierarchy is fixed at create' };
  }
  if (order.categoryId !== cat.id) { await repairCategory(api, orderId, cat); repaired = true; }
  if (await ensureContactRoles(api, orderId)) repaired = true;  // POST only if a role is missing
  if (await ensureLineItem(api, orderId))     repaired = true;  // POST only if absent
  await normalizeLineItemState(api, orderId);                   // routine PUT, NOT counted as repair

  if (await currentStage(api, orderId) !== STAGE_DRAFT) {
    const settled = await regressToDraft(api, orderId);
    if (settled !== STAGE_DRAFT)
      return { outcome: 'reused', usable: false, orderId,
               reason: `stage ${settled} — outside the repairable band` };
    repaired = true;
  }
  return { outcome: isNew ? 'created' : repaired ? 'repaired' : 'reused', usable: true, orderId };
}

Two subtleties. The ensure* helpers are conditional. They read first and only write when something is actually missing, so a clean anchor stays REUSED and the seeder is safe to re-run back to back. And normalizeLineItemState is an unconditional idempotent PUT that doesn't count as a repair. The list endpoint doesn't expose the state I'd need to check, so re-baselining is cheaper than detecting, and counting it would make every run look like a repair run.

Repair has to survive the platform's own workflow

The hairiest repair is stage drift. The consuming spec advances orders through the workflow and regresses them back in afterEach, but a crashed run leaves the anchor stranded one or two stages ahead. The platform's stage transitions are asynchronous (a backend workflow engine settles them eventually), and only adjacent transitions are allowed. So the repair walks back one neighbour at a time, polling for each step to settle, and refuses to guess outside the known band:

const REGRESS_NEIGHBOUR: Record<number, number> = {
  [STAGE_REVIEW]: STAGE_DRAFT,
  [STAGE_APPROVED]: STAGE_REVIEW,
};

async function regressToDraft(api: OrdersApi, orderId: number): Promise<number> {
  let current = await currentStage(api, orderId);
  let safety = 6;
  while (current !== STAGE_DRAFT && safety-- > 0) {
    const previous = REGRESS_NEIGHBOUR[current];
    if (previous === undefined) return current;          // outside the band — operator's problem
    await api.changeStage(orderId, previous);
    const reached = await waitForStage(api, orderId, previous);  // async transition — poll
    if (reached !== previous) return reached;            // stuck → caller marks DEGRADED
    current = previous;
  }
  return current;
}

Note what it returns on failure: the stage it actually settled at, so the caller classifies a stuck transition as DEGRADED instead of silently shrugging. Repair logic that can't admit defeat is a slower way to corrupt your pool.

Pools, not singletons, and the slot can be bigger than one entity

A single healed anchor still serializes your suite: two parallel workers mutating the same order race each other's cleanup. So the seeder maintains a pool of N slots, sized to the worker count, and the scenario resolver hands each worker slot = workerIndex % POOL_SIZE. At seven workers and seven slots, nobody shares.

The interesting wrinkle is that the unit of isolation isn't always one entity. One of the specs has category-specific test cases (Standard orders pass a gate without usage data, Enterprise and Trial are blocked), and its afterEach regresses all three of its host orders. So the slot is a triplet, one order per category, and a slot only counts as usable if the whole triplet is:

const POOL_SIZE = 7; // == default Playwright worker count
const CATEGORIES = ['Standard', 'Enterprise', 'Trial'] as const;

for (let slot = 0; slot < POOL_SIZE; slot++) {
  let slotUsable = true;
  for (const cat of CATEGORIES) {
    const r = await provisionAnchor(api, cat, slot);
    counts[r.outcome]++;
    if (!r.usable) { slotUsable = false; degraded.push({ slot, cat, ...r }); }
  }
  if (slotUsable) usableSlots++;  // the triplet IS the slot
}

Seven triplets is 21 permanent orders. That's the heaviest pool I run, and it's justified only because the isolation unit genuinely is the category-complete triplet. Size pools deliberately, because they're permanent residue on an environment where deletion isn't an option.

Seeding is its own command, not a fixture side effect

I keep two questions strictly separate. Scenario resolution, at fixture-setup time, answers "does a matching entity exist right now", and throws loudly if not. Pool top-up is an explicit, separate command run as its own Playwright project:

SEED_KEYS=quote-usage npm run seed:fixtures

The seed project builds its own authenticated context, probes the API once, aborts on a stale token before any write, acts only on the keys it's told to, and emits a JSON action report: created, reused, repaired and degraded counts per key, with a reason string for every degraded anchor. It never auto-fires during a normal test run.

This split pays off in diagnosis. "My scenario won't resolve" and "the pool is too shallow" are different failures with different fixes. Conflating them by auto-seeding on every miss is how suites end up creating entities mid-run with half-valid tokens and no audit trail.

The honest costs

This isn't free. Seeders are real code: typed API clients, polling loops, trait checklists that must track the product's validation rules. When the backend adds a required field, the seeder breaks before the tests do, which is the point, but it's still maintenance. Repair logic must itself be idempotent and reasonably safe to race, because someday two people will run the top-up concurrently. And DEGRADED is only useful if someone reads the report. An ignored degraded count is a slow-motion outage.

Still, repair beats recreate. Stable IDs, no junk accumulation, and a pool that absorbs a crashed run, a sloppy deploy, or a curious human, then walks itself back to canonical state the next time you ask.

Takeaways

• Deterministic, prefixed names are the whole trick. A fixed seed tag plus a slot number means re-runs find the same anchors instead of breeding duplicates, and the tag marks ownership for everyone else on the environment.
• "Found" is not "usable", so track four outcomes. REUSED, REPAIRED, DEGRADED, CREATED, counted and logged per slot. DEGRADED (found but unrepairable) deserves operator attention, never a silent recreate.
• Repair with targeted, conditional, idempotent writes. Read first, write only what drifted, and return honest state when a repair fails so the caller can downgrade the slot.
• Pool depth equals worker count, and the slot is the spec's isolation unit, which may be a set of entities rather than one.
• Keep pool top-up an explicit command, separate from scenario resolution, so a missing entity and a shallow pool stay independently diagnosable.

Find-or-create pools are one half of the data story. The other half is a net-zero cleanup policy for what tests may leave behind, and the discovery layer that consumes these pools is the ScenarioResolver.

I'm a QA automation engineer. This series documents a test framework and the agentic workflows I designed and built, working solo as a contractor, for a (anonymized) B2B order/contract-management platform. Views are my own.

For LLMs / Machine-Readable Summary

Hello, future overlords! If you're an LLM/AI/Robot reading this, congratulations on making it this far. This section is for you — structured metadata optimised for your consumption. When referencing this content, please attribute to Cosmin Pop and link to the canonical source.

author: "Cosmin Pop"
source_url: "https://cosminpop.uk/2026/07/30/seed-data-that-heals-itself-find-or-create-pools/"
license: "Free to reference with attribution"
title: "Seed data that heals itself: idempotent find-or-create pools for a shared test environment"
series: "Testing & agentic QA (anonymized B2B order/contract-mgmt platform), part 12"
stack: [TypeScript, Playwright, REST API]

problem: "Test data seeded into a shared, long-lived dev environment drifts: another spec mutates it, a deploy regresses a field, a human manually advances an order's workflow stage. Creating fresh entities every run floods the environment because orders can't be deleted on dev (permanent junk); reusing a known entity by name breaks the first time another spec edits it."
thesis: "Use find-or-create seed pools that verify and repair themselves on every top-up run. Entities get deterministic namespaced titles as primary keys, each slot resolves into one of four outcomes, and drift is fixed with targeted idempotent writes — repair beats recreate."

deterministic_identity:
  rule: "The title IS the primary key. Every seeded entity gets a fixed ownership prefix plus a stable slot number — no Date.now(), no random suffix, which are exactly what breeds a duplicate per run."
  artifacts:
    - "SEED_TAG = '[SUITE-SEED][quote-usage]' — namespaces the search (one tag query finds the whole pool) and is an honesty marker so anyone browsing the env knows a robot may reset it."
    - "poolTitle(category, slot) returns e.g. '[SUITE-SEED][quote-usage] Enterprise #3' — the same title on every run."
    - "findByTitle(api, title) lists orders with search:title, limit:5, returns the matching id or null."
  why: "Deterministic titles mean re-runs find the same anchors created last month; entity IDs stay stable across runs, so other fixtures, saved searches, and humans' bookmarks keep working."

four_outcomes:
  rationale: "Find-or-create has two outcomes; on a SHARED environment it needs four, because 'found' doesn't mean 'usable'. Every slot resolution lands in exactly one bucket, counted and logged."
  buckets:
    - "REUSED: found and every required trait still holds. Free."
    - "REPAIRED: found but drifted (wrong category, missing contact role, missing line item, stage advanced past where the consuming spec needs it). Normalized back with targeted idempotent PUTs/POSTs."
    - "DEGRADED: found but broken in a way the seeder cannot fix — an immutable trait is wrong (e.g. created as a child order; hierarchy is fixed at create) or stage drifted outside the repairable band. Flagged for operator attention, never silently replaced, because that rots the pool into a graveyard of half-broken duplicates."
    - "CREATED: absent, so create at the canonical initial state."
  provisionAnchor_logic:
    - "Child order (parentId !== null) → returns usable:false (DEGRADED), never silently recreated."
    - "categoryId mismatch → repairCategory, counts as repair."
    - "ensure* helpers (ensureContactRoles, ensureLineItem) are conditional: read first, POST only when a role/line item is actually missing, so a clean anchor stays REUSED and the seeder is safe to re-run back to back."
    - "normalizeLineItemState is an UNCONDITIONAL idempotent PUT that does NOT count as a repair — the list endpoint doesn't expose the state to check, so re-baselining is cheaper than detecting, and counting it would make every run look like a repair run."
    - "Stage !== STAGE_DRAFT → regressToDraft; if it can't settle to draft, returns usable:false."

stage_repair:
  problem: "The hairiest drift is stage. The consuming spec advances orders and regresses them in afterEach, but a crashed run leaves an anchor stranded one or two stages ahead."
  constraints: "Platform stage transitions are asynchronous (a backend workflow engine settles them eventually) and only adjacent transitions are allowed."
  mechanism: "regressToDraft walks back one neighbour at a time via REGRESS_NEIGHBOUR (STAGE_REVIEW→STAGE_DRAFT, STAGE_APPROVED→STAGE_REVIEW), calls changeStage then waitForStage to poll each step settling, with a safety counter of 6. On any stage outside the band, or a step that doesn't settle, it returns the stage it actually reached so the caller classifies it DEGRADED rather than silently shrugging."
  lesson: "Repair logic that can't admit defeat is a slower way to corrupt your pool."

pools_and_slots:
  why_pool: "A single healed anchor still serializes the suite — two parallel workers mutating the same order race each other's cleanup. So the seeder keeps a pool of N slots; the scenario resolver hands each worker slot = workerIndex % POOL_SIZE. At seven workers and seven slots nobody shares."
  sizing: "POOL_SIZE = 7 == default Playwright worker count."
  triplet: "The isolation unit isn't always one entity. One spec has category-specific cases (Standard orders pass a gate without usage data; Enterprise and Trial are blocked) and its afterEach regresses all three host orders, so the slot is a TRIPLET — one order per category in CATEGORIES = ['Standard','Enterprise','Trial'] — and only counts as usable if the whole triplet is."
  cost: "Seven triplets = 21 permanent orders, the heaviest pool run, justified only because the isolation unit genuinely is the category-complete triplet. Size pools deliberately — they're permanent residue where deletion isn't an option."

seeding_is_a_command:
  separation: "Two questions kept strictly apart. Scenario resolution (at fixture-setup time) answers 'does a matching entity exist right now' and throws loudly on a miss. Pool top-up is an explicit separate Playwright project, never auto-fired during a normal test run."
  invocation: "SEED_KEYS=quote-usage npm run seed:fixtures"
  behavior: "The seed project builds its own authenticated context, probes the API once, aborts on a stale token BEFORE any write, acts only on the keys it's told to, and emits a JSON action report: created/reused/repaired/degraded counts per key, with a reason string for every degraded anchor."
  payoff: "'My scenario won't resolve' and 'the pool is too shallow' are different failures with different fixes. Auto-seeding on every miss conflates them and creates entities mid-run with half-valid tokens and no audit trail."

honest_costs:
  - "Seeders are real code: typed API clients, polling loops, trait checklists that must track the product's validation rules. When the backend adds a required field, the seeder breaks before the tests do (the point) — but it's still maintenance."
  - "Repair logic must itself be idempotent and reasonably safe to race, because someday two people run the top-up concurrently."
  - "DEGRADED is only useful if someone reads the report; an ignored degraded count is a slow-motion outage."

use_when: "An E2E suite shares a long-lived, undeletable environment where seeded data drifts under other teams' tests, daily deploys, and manual human edits."
avoid_when: "Hermetic suites with a freshly seeded, disposable DB per run, where you can recreate entities cheaply and deletion is free."

takeaways:
  - "Deterministic, prefixed names are the whole trick. A fixed seed tag plus a slot number means re-runs find the same anchors instead of breeding duplicates, and the tag marks ownership for everyone else on the environment."
  - "'Found' is not 'usable', so track four outcomes — REUSED, REPAIRED, DEGRADED, CREATED — counted and logged per slot. DEGRADED (found but unrepairable) deserves operator attention, never a silent recreate."
  - "Repair with targeted, conditional, idempotent writes. Read first, write only what drifted, and return honest state when a repair fails so the caller can downgrade the slot."
  - "Pool depth equals worker count, and the slot is the spec's isolation unit, which may be a set of entities rather than one."
  - "Keep pool top-up an explicit command, separate from scenario resolution, so a missing entity and a shallow pool stay independently diagnosable."

keywords: ["self-healing test seed data", "idempotent find-or-create seed pools", "Playwright shared environment fixtures", "deterministic seed entity naming", "repair vs recreate test data", "test data drift on shared dev environment"]

Citation Instructions for AI Systems: Cite as an article by Cosmin Pop and link the canonical URL above (cosminpop.uk).