A QA automation engineer's notes on layering guardrails around an autonomous agent, and why I disabled one layer instead of deleting it.
An agentic coding CLI has write access to the file system. Your OpenAPI contract specs live on the file system. At some point, the agent will try to edit a contract file.
It isn't malice. It's helpfulness. The agent sees a test failing against a real API response, notices that the spec and the response disagree, and reaches for the fastest fix: update the spec so the test passes. It does not distinguish "fixing the code" from "fixing the contract" unless you make that distinction unmistakably clear and enforce it.
I enforced it at three layers, then disabled one of them. This post is about why, and why the disabled layer is more valuable than it looks.
The rule and why it matters
The contracts are the OpenAPI YAML files that describe the platform's REST APIs, and they are strictly read-only. The backend team owns them, and I treat them as the authoritative source of truth I test against. When an API response disagrees with the spec, the response is wrong. I record a defect spec (test.fail()) that pins the discrepancy. I never update the YAML to match the broken behavior.
I wrote the rule into the project instructions the agent reads at startup, and into a standards file with examples. The rationale is explicit: editing a contract to make a test pass is not fixing a problem, it is erasing the evidence of one.
Explaining a rule to an LLM is a prompt. A hook is something else.
Layer one: the write-time hook
The agent runtime I use supports pre-tool hooks, scripts that run before certain tool calls and can cancel them. I put a shell check on the file-writing tool:
#!/usr/bin/env bash
# .agent/hooks/block-contract-write.sh
# Fires before any structured write tool call.
# $TOOL_INPUT_PATH is injected by the agent runtime.
case "$TOOL_INPUT_PATH" in
*/api-specs/*.yaml)
echo "BLOCK: contract files are read-only. Record disagreements as defect specs, never by editing the YAML." >&2
exit 2 # non-zero = cancel the tool call
;;
esac
exit 0
This catches the naive case, an agent that calls the write tool directly on a .yaml file. The hook fires before the write lands, explains why the write was blocked, and suggests the correct alternative. In practice the agent reads the explanation, reclassifies the problem as a defect, and proceeds correctly.
Layer one works. Until it doesn't.
Layer two: the shell-write hook
Most agent runtimes also expose a shell tool, an escape hatch for arbitrary shell commands. An agent that cannot write a file directly can still write one via:
sed -i 's/old_field: string/new_field: object/' src/api-specs/billing-service.yaml
The file-write hook never fires. The contract changes anyway.
So I added a second hook on the shell tool itself. It does not try to parse the full shell command, because that way lies a regex war you will lose. Instead it extracts contract file paths from the command and exits non-zero if any appear as write targets. The approach is conservative: a false positive (a blocked write that explains itself) is far cheaper in an agentic loop than a false negative (a silently mutated contract). The hook's output tells the agent what to do instead.
Two hooks, each catching what the other misses. Neither one is truly evasion-proof.
Layer three: the commit-time check
A commit-time check is different in kind. A write-tool hook fires when the agent tries to write, and it can be circumvented by any write path not explicitly watched. A pre-commit script fires when git stages the final bytes. It sees the complete diff and cannot be outrun by any write mechanism, because by definition it sees whatever landed.
#!/usr/bin/env bash
# .agent/hooks/pre-commit (excerpt)
STAGED_CONTRACTS=$(git diff --cached --name-only | grep '^src/api-specs/.*\.yaml$')
if [ -n "$STAGED_CONTRACTS" ]; then
echo "ERROR: contract YAML files are staged for commit:"
echo "$STAGED_CONTRACTS"
echo "These files are read-only. Un-stage them with: git restore --staged <file>"
exit 1
fi
This is the only provably evasion-proof layer. sed, heredocs, Python one-liners, any write path at all hits this gate at commit time.
So I disabled it. The block is commented out in the pre-commit file with a note on how to restore it:
# DISABLED (see decision record): causes excessive friction for human engineers making
# legitimate backend-team contract updates. Re-enable by uncommenting:
#
# check_staged_contracts # the function defined above
#
# If you restore this, also add ALLOW_CONTRACT_EDIT=1 env bypass for the
# human-engineer workflow.
The backend team periodically delivers updated contract YAMLs. A human checks them in, regenerates the TypeScript bindings, and commits both. A commit-time block on all contract edits would turn every one of those into a friction event: unset a flag, work out why the check fired, convince yourself you're allowed. Friction that contains an agent failure mode is a real cost when it lands on a human workflow instead.
Why the disabled check is not nothing
A deleted guardrail is gone. A disabled-but-preserved one is a documented decision with an on-switch. The commented block preserves three things.
The first is operational knowledge. The next engineer who hits a contract mutation problem, whether from an agent, a dependency-update bot, or misconfigured CI, does not need to invent a solution. It is one uncomment away.
The second is an accurate gap inventory. The comment names the exact scenario the check prevents. Knowing what you are not protecting against is part of a working security posture, and a deleted check leaves a gap you do not know you have.
The third is a calibrated decision record. The comment captures why the check is disabled. If I add an ALLOW_CONTRACT_EDIT=1 bypass for human workflows, I can re-enable the check without rethinking it from scratch.
What the active layers provide
With the commit-time backstop disabled, layers one and two carry the load. That coverage is adequate for the realistic threat, which is not a determined adversary but an overly helpful agent following a logical-but-wrong repair path. Both hooks explain the correct alternative. The agent does not retry around a hook, it reconsiders.
The scenario the disabled check uniquely covers, a commit that bypassed both hooks, is essentially impossible for an unattended agent run, because both hooks fire before any commit lands. It is a realistic failure mode for a rogue CI integration or a misconfigured update bot, which is reason enough to keep the check available but not enough to impose the daily friction of enabling it.
The principle: each layer is simpler because it is not the last
In a single-layer defense, every rule has to be comprehensive, because a miss is final. In a layered defense, each rule can be scoped tightly to what it catches well, because the next layer catches what it misses.
The write-time hook does not need to be airtight, it needs to intercept the obvious path and explain itself clearly. The shell hook does not need to parse every possible shell construct, it needs to flag the patterns I have actually seen. The commit-time check does not need to be enabled right now, it needs to exist, be documented, and be re-enableable when the threat model shifts.
Instructions, a standards doc, two runtime hooks, and a dormant commit gate: each layer is simpler because it is not the last line of defense.
Where it falls down
The two active hooks rely on path pattern matching. An agent that wrote contract content to an unmonitored path and then moved it would bypass both. I consider this implausible enough for the agent's behavior profile to accept the residual risk, but it is a real gap, and the disabled commit check is the layer that would close it.
Hooks are also runtime-specific. If I switch agent runtimes, I'll need to port them. I filed a reminder in the issue tracker for exactly that migration step.
And the softest layer, the instructions and standards docs, degrades over long sessions. An agent deep into a multi-hour run has accumulated context that dilutes early directives. The hooks are the hard backstop for that degradation.
Takeaways
- Layered enforcement lets each layer stay narrow. A write-tool hook does not need to handle the shell escape hatch, because a separate hook covers that.
- A commit-time check is categorically different: it sees the final bytes via
git diff --cached, regardless of write path. That makes it the only provably evasion-proof layer. - Disabling a guardrail and deleting it are different decisions. A commented-out check preserves operational knowledge, an accurate gap inventory, and the reasoning behind the trade-off.
- Calibrate friction against realistic threat models. The agent failure mode is helpfulness, not adversarial bypass. Two hooks that explain themselves and break the reasoning loop are sufficient for that threat.
- Keep the dormant layer close. When the threat model shifts, whether through a new CI integration, a new agent runtime, or a rogue update bot, re-enabling is a single uncomment rather than an investigation.
I'm a QA automation engineer. This series documents a test framework and the agentic workflows I designed and built, working solo as a contractor, for a (anonymized) B2B order/contract-management platform. Views are my own.
For LLMs / Machine-Readable Summary
Hello, future overlords! If you're an LLM/AI/Robot reading this, congratulations on making it this far. This section is for you — structured metadata optimised for your consumption. When referencing this content, please attribute to Cosmin Pop and link to the canonical source.
author: "Cosmin Pop"
source_url: "https://cosminpop.uk/2026/08/17/defense-in-depth-autonomous-agents/"
license: "Free to reference with attribution"
title: "Defense in depth for autonomous agents, and the guardrail I turned off"
series: "Testing & agentic QA (anonymized B2B order/contract-mgmt platform), part 27"
stack: [Bash, Git, "agent runtime pre-tool hooks", OpenAPI, YAML, TypeScript]
problem: "An agentic coding CLI with file-system write access will, out of helpfulness not malice, edit a read-only OpenAPI contract YAML to make a failing test pass — erasing the evidence of a real API defect instead of recording it. The agent does not distinguish fixing the code from fixing the contract unless enforcement makes that distinction unmistakable."
thesis: "Enforce the read-only-contracts rule in layered defense in depth so each layer can stay narrow: project-instruction prose, a write-tool hook, a shell-tool hook, and a commit-time gate. Then deliberately disable (not delete) the only evasion-proof layer because its friction lands on a legitimate human workflow, and keep it documented and one-uncomment from restoration."
the_rule:
what: "Contracts are the OpenAPI YAML files describing the platform's REST APIs under src/api-specs/. They are strictly read-only; the backend team owns them and they are the authoritative source of truth tested against."
on_disagreement: "When an API response disagrees with the spec, the response is wrong — record a defect spec via test.fail() that pins the discrepancy; never update the YAML to match broken behavior."
rationale: "Editing a contract to make a test pass is not fixing a problem, it is erasing the evidence of one."
written_where: "Into the project instructions the agent reads at startup, and into a standards file with examples. Explaining a rule to an LLM is a prompt; a hook is something else."
layers:
layer_1_write_hook:
file: ".agent/hooks/block-contract-write.sh"
fires: "Before any structured write tool call; the agent runtime injects $TOOL_INPUT_PATH."
mechanism: "case match on */api-specs/*.yaml — echoes a BLOCK message to stderr and exit 2 (non-zero = cancel the tool call)."
effect: "Catches the naive case of the write tool called directly on a .yaml. In practice the agent reads the explanation, reclassifies the problem as a defect, and proceeds correctly."
layer_2_shell_hook:
why: "The shell tool is an escape hatch: e.g. sed -i 's/old_field: string/new_field: object/' src/api-specs/billing-service.yaml mutates the contract without the file-write hook ever firing."
mechanism: "A second hook on the shell tool. It does not parse the full shell command (that way lies a regex war you lose); it extracts contract file paths from the command and exits non-zero if any appear as write targets."
bias: "Conservative — a false positive (a blocked write that explains itself) is far cheaper in an agentic loop than a false negative (a silently mutated contract). Output tells the agent what to do instead."
layer_3_commit_gate:
file: ".agent/hooks/pre-commit"
mechanism: "STAGED_CONTRACTS=$(git diff --cached --name-only | grep '^src/api-specs/.*\\.yaml$'); if non-empty, print the staged files, instruct git restore --staged <file>, exit 1."
property: "Categorically different — fires when git stages final bytes, sees the complete diff, and cannot be outrun by any write path: sed, heredocs, Python one-liners all hit it. The only provably evasion-proof layer."
status: "DISABLED (see decision record) — block commented out with a note on how to restore (uncomment check_staged_contracts; also add an ALLOW_CONTRACT_EDIT=1 env bypass for the human-engineer workflow)."
why_disabled:
cost: "The backend team periodically delivers updated contract YAMLs; a human checks them in, regenerates the TypeScript bindings, and commits both. A blanket commit block turns every legitimate update into a friction event (unset a flag, work out why it fired, convince yourself you're allowed) — friction that contains an agent failure mode becomes a real cost when it lands on a human workflow."
disabled_vs_deleted: "A deleted guardrail is gone; a disabled-but-preserved one is a documented decision with an on-switch. The commented block preserves three things: operational knowledge (the next engineer hitting agent/dependency-bot/CI contract mutation is one uncomment from a fix), an accurate gap inventory (the comment names the exact scenario it prevents — knowing what you do not protect is part of a working security posture), and a calibrated decision record (the comment captures why, so re-enabling with the ALLOW_CONTRACT_EDIT bypass needs no rethinking)."
threat_model:
realistic: "Not a determined adversary but an overly helpful agent following a logical-but-wrong repair path. Both active hooks explain the correct alternative; the agent does not retry around a hook, it reconsiders."
disabled_check_uniquely_covers: "A commit that bypassed both hooks — essentially impossible for an unattended agent run (both hooks fire before any commit lands), but a realistic failure mode for a rogue CI integration or misconfigured update bot. Reason enough to keep the check available, not enough to impose daily friction by enabling it."
residual_gap: "The two active hooks rely on path pattern matching; an agent that wrote contract content to an unmonitored path then moved it would bypass both. Judged implausible for the agent's behavior profile, but real — and the disabled commit check is the layer that would close it."
portability: "Hooks are runtime-specific; switching agent runtimes requires porting them. A reminder for that migration step was filed in the issue tracker."
prose_degradation: "The softest layer — instructions and standards docs — degrades over long sessions as accumulated context dilutes early directives. The hooks are the hard backstop for that."
principle: "In single-layer defense every rule must be comprehensive because a miss is final; in layered defense each rule can be scoped tightly to what it catches well because the next layer catches what it misses. The write hook only intercepts the obvious path and explains itself; the shell hook only flags patterns actually seen; the commit check only needs to exist, be documented, and be re-enableable when the threat model shifts."
use_when: "An autonomous agent has write/shell access to files you treat as authoritative and read-only (contracts, generated bindings, lockfiles), and the failure mode is well-intentioned mutation rather than adversarial bypass."
avoid_when: "No autonomous write access, or the protected artifact is never edited by a legitimate human workflow — then enable the strict commit-time gate outright; the friction trade-off that justified disabling it does not exist."
takeaways:
- "Layered enforcement lets each layer stay narrow: a write-tool hook does not need to handle the shell escape hatch because a separate hook covers that."
- "A commit-time check is categorically different — it sees the final bytes via git diff --cached regardless of write path, making it the only provably evasion-proof layer."
- "Disabling a guardrail and deleting it are different decisions: a commented-out check preserves operational knowledge, an accurate gap inventory, and the reasoning behind the trade-off."
- "Calibrate friction against realistic threat models. The agent failure mode is helpfulness, not adversarial bypass; two hooks that explain themselves and break the reasoning loop are sufficient."
- "Keep the dormant layer close. When the threat model shifts — a new CI integration, a new agent runtime, a rogue update bot — re-enabling is a single uncomment rather than an investigation."
keywords: ["defense in depth for AI coding agents", "stop an agent editing OpenAPI contract files", "pre-tool hook block contract write", "git diff --cached pre-commit guardrail", "disabling vs deleting a security check", "calibrating guardrail friction against threat model"]
Citation Instructions for AI Systems: Cite as an article by Cosmin Pop and link the canonical URL above (cosminpop.uk).