Software Testing

Answering 'does this ticket cover the feature?' from the live tracker API costs four round-trips, auth, and rate limits: thirty seconds and a slice of context, for a read-only lookup that agents repeat constantly. So the whole tracker becomes a nightly SQLite snapshot, queried offline in milliseconds. The real unlock is baking institutional knowledge (like never trusting the raw status column) into the schema, so any agent queries it correctly without knowing the history.

A code-review pass produced 69 mechanical fixes: the kind of list that sits in a ticket for months. Doing them serially takes weeks; bundling them into one agent prompt yields a 185-file diff nobody can review. The fourth option is a wave-gated campaign of parallel LLM subagents, with a read-only census first, a self-healing static-check gate between every wave, and folder-scoped write access. 61 of 69 landed in one branch, zero merge conflicts.

Not every task belongs to the LLM. Plotting ambiguity against blast radius shows where to reach for deterministic orchestration with assertions instead, and reduce the agent to a shell runner behind a precondition gate.

An agent handed back four plausible, entirely made-up items for a 161-item corpus, with no error signal. Routing worker output through files instead of agent chatter makes a multi-agent pipeline resistant to that failure.

When the backend died mid-run, the subagents dutifully logged every remaining item as a failure and buried the real findings in noise. The fix: a circuit breaker in the orchestrator that halts fan-out once failures cluster.

A skill that aces softball prompts proves nothing. Eval-driven development for AI skills: harvest real failure modes from unguided runs, then build discriminating evals that separate a working skill from an impressive one.

An agent picks which skill to run by matching the prompt against each skill's description. That makes the description a classifier, not a summary: write it to fire on the right prompts and stay silent on the wrong ones.

One agent's git add -A swept in another agent's half-written edits and committed them under the wrong name. The fix was not more trust but strict file ownership: partition the repo, stage by name.

Spawning an AI agent with env: process.env hands it every secret you hold, including the tracker token it must never see. An allowlist-first subprocess env makes leakage impossible by default.

Layered guardrails keep an autonomous coding agent from editing the OpenAPI contract specs it should treat as read-only. Why I kept a third layer commented out instead of deleting it.