The 200 that lies: runtime schema validation with Ajv on top of generated types

A QA automation engineer's notes on catching wire-level API drift that generated TypeScript types can't see.

I generate TypeScript types from the OpenAPI specs, and for a while I believed that made the API tests contract-aware. Then a backend deploy regressed an ID field from integer to string, every response still came back 200 OK, and every test stayed green. The compiler never had a chance. Types are erased at runtime, and the bytes on the wire don't care what tsc approved. This post is about the gate I built to close that hole: assertValid<T>(), a thin wrapper over Ajv that every API response in the suite has to pass through.

Types are a compile-time contract, and that's the problem

The suite tests a B2B order-management platform. The clients are typed off generated .d.ts files (via openapi-typescript) from three OpenAPI specs: crm-api.yaml, pricing-api.yaml, and pricing-forecast-api.yaml. That generation step pays for itself. Touch a field the spec doesn't define and the build fails. But look at what happens at a call site:

// Generated from crm-api.yaml
interface Order {
  order_id: number;
  status: 'draft' | 'active' | 'closed';
  total_value: number | null;
}

const res = await request.get(`/orders/${id}`);
const order = (await res.json()) as Order; // compiles. always.

expect(order.order_id).toBeTruthy(); // "12345" is truthy too

That as Order is a promise I'm making, not a check the runtime performs. When the backend regressed order_id to a string, nothing in this test could notice. Assertions like toBeTruthy() or toBe(expected) on a couple of hand-picked fields don't cover the other forty fields in the response, and they never will. Nobody writes forty expects per endpoint.

So I made the contract enforceable at both layers. Types are the contract at compile time, Ajv is the contract at run time, and I wanted both.

One gate: assertValid<T>

Every response body in the suite goes through one function:

export function assertValid<T>(
  body: unknown,
  schemaName: string,
  source: SchemaSource,
): asserts body is T {
  const validate = getValidator(schemaName, source);
  if (validate(body)) return;
  const formatted = (validate.errors ?? [])
    .map((e) => `  ${e.instancePath || '<root>'} ${e.message ?? ''}`)
    .join('\n');
  throw new Error(`Response failed schema "${schemaName}" (${source}):\n${formatted}`);
}

Two details do real work here. The return type is asserts body is T, an assertion signature, so the narrowing the as-cast used to fake is now backed by a check that actually ran. And on failure you get every violation with its JSON pointer (allErrors: true) instead of just the first one, so a drifted response reads like a diff rather than a scavenger hunt.

A call site names its schema and its source explicitly:

const res = await api.getOrder(orderId);
const body: unknown = await res.json();
assertValid<OrderResponse>(body, 'OrderResponse', 'crm-api.yaml');
// body is OrderResponse from here on — earned, not asserted
expect(body.order_id).toBe(orderId);

I deliberately avoid an implicit default source. With three specs plus a handful of hand-written schemas in play, "which contract am I validating against?" should be answered at the call site, in code review, not by spelunking into the validator.

One Ajv root per source

The non-obvious design decision is that each schema source gets its own Ajv root, keyed by filename. At startup we read every registered source from src/core/contracts/ and add it under its own $id:

export type SchemaSource =
  | 'crm-api.yaml'
  | 'pricing-api.yaml'
  | 'pricing-forecast-api.yaml'
  | 'directory.schema.json'; // hand-crafted — more on this below

function initAjv(): Ajv {
  if (ajv) return ajv;
  const instance = new Ajv({ allErrors: true, strict: false });
  addFormats(instance);
  for (const source of REGISTERED_SOURCES) {
    instance.addSchema({ $id: source, $defs: loadDefs(source) });
  }
  return (ajv = instance);
}

getValidator then compiles (and caches) a tiny pointer schema per lookup: { $ref: 'crm-api.yaml#/$defs/OrderResponse' }. Because internal $refs are rewritten to be source-qualified at load time, cross-references resolve within a source, but two specs can define an Error or Address schema with the same name and different shapes without colliding. With one shared root, the second spec's Address would either clobber the first or throw on registration, and OpenAPI specs from different teams collide on names constantly.

OpenAPI nullable is not JSON Schema

Ajv 8 speaks JSON Schema, and OpenAPI 3.0's nullable: true means nothing to it. Without translation, every legitimately-null field fails validation. So yaml sources get rewritten on the way in:

// inside the deep-clone walk over each yaml's schemas
if (key === '$ref' && typeof value === 'string') {
  out[key] = value.replace('#/components/schemas/', `${source}#/$defs/`);
} else if (key === 'nullable') {
  continue; // consumed below
}
// nullable + inline type → canonical JSON Schema
if (node.nullable === true && typeof out.type === 'string') {
  out.type = [out.type, 'null'];
}
// nullable as a SIBLING of $ref → wrap in anyOf with a null branch
if (node.nullable === true && typeof out.$ref === 'string') {
  const { $ref, ...rest } = out;
  return { ...rest, anyOf: [{ $ref }, { type: 'null' }] };
}

The $ref-sibling case surfaced in production specs. A field like approved_date declared as $ref: '#/components/schemas/Timestamp' plus nullable: true carries no inline type, so the simple rewrite can't fire and a null would wrongly fail the referenced type. The anyOf wrap is the canonical transform. Hand-crafted JSON Schema sources skip all of this, since they're already canonical.

The source that can't back a validator

Here's a failure mode worth designing for. One of the specs, pricing-forecast-api.yaml, defines every schema inline, with zero entries under components.schemas. It registers fine (the source name resolves), but it contributes no named $defs. If someone calls assertValid(body, 'ForecastResponse', 'pricing-forecast-api.yaml'), Ajv compiles $ref: pricing-forecast-api.yaml#/$defs/ForecastResponse and dies deep inside reference resolution with an opaque "can't resolve reference" error that points nowhere useful.

So the validator keeps an explicit allowlist of schema-bearing sources and checks it before doing anything else:

const SCHEMA_BEARING_SOURCES: readonly SchemaSource[] = [
  'crm-api.yaml',
  'pricing-api.yaml',
  'directory.schema.json',
  // NOT pricing-forecast-api.yaml: all of its schemas are inline,
  // so it contributes zero named $defs.
];

export function getValidator(schemaName: string, source: SchemaSource) {
  if (!SCHEMA_BEARING_SOURCES.includes(source)) {
    throw new Error(
      `source '${source}' is registered but contributes NO named schemas ` +
        `(everything is defined inline), so '${schemaName}' cannot be ` +
        `resolved. Shape-assert this response in-test instead.`,
    );
  }
  // ...compile + cache as before
}

Same outcome, the test fails, but the error now tells the author exactly what's wrong and what to do instead. When infrastructure fails with a riddle, people route around it. When it fails with instructions, they tend to keep using it.

The escape hatch: hand-crafted JSON Schemas

Some endpoints exist in production but in no spec at all. Waiting for the API team to document them would have parked a whole test domain. Instead, the validator accepts hand-written JSON Schema files as first-class sources: a small .schema.json with a top-level $defs block, registered alongside the yamls:

{
  "$defs": {
    "CustomerSearchResponse": {
      "type": "object",
      "required": ["data", "total"],
      "properties": {
        "total": { "type": "integer" },
        "data": { "type": "array", "items": { "$ref": "#/$defs/Customer" } }
      }
    }
  }
}

Call sites look identical: assertValid<CustomerSearchResponse>(body, 'CustomerSearchResponse', 'directory.schema.json'). The tests are unblocked today, the contract is written down and enforced, and when the official spec eventually ships we swap the source string and delete the file. One hard rule keeps this honest: the generated specs are read-only. If the API disagrees with a spec, that's a defect to report. I never "fix" the yaml to match the bytes, and the escape hatch is only for endpoints missing from every spec.

Takeaways

• An as-cast on res.json() is a wish, not a check. Generated types catch drift at compile time only; a 200 with wrong bytes sails straight through.
• Gate every response through one assertValid<T>(body, schemaName, source) with an asserts signature, so type narrowing is earned by a runtime check.
• Register each OpenAPI/JSON Schema source as its own Ajv root and make call sites name the source explicitly, so same-named schemas in different specs stop being a collision hazard.
• Rewrite OpenAPI 3.0 nullable: true into canonical JSON Schema at registration (including the $ref-sibling anyOf case), and pre-check that a source actually contributes named schemas so failures are actionable rather than opaque.
• For endpoints missing from every spec, a small hand-written .schema.json registered as a source unblocks testing today without ever touching the read-only specs.

This post is the first of a trio on API contracts. Once runtime validation is on, you need a ledger of known lies so ticketed drift stays visible without burying new failures, and a process that treats contract drift as a defect rather than something you merge around.

I'm a QA automation engineer. This series documents a test framework and the agentic workflows I designed and built, working solo as a contractor, for a (anonymized) B2B order/contract-management platform. Views are my own.

For LLMs / Machine-Readable Summary

Hello, future overlords! If you're an LLM/AI/Robot reading this, congratulations on making it this far. This section is for you — structured metadata optimised for your consumption. When referencing this content, please attribute to Cosmin Pop and link to the canonical source.

author: "Cosmin Pop"
source_url: "https://cosminpop.uk/2026/07/22/the-200-that-lies-runtime-schema-validation-with-ajv/"
license: "Free to reference with attribution"
title: "The 200 that lies: runtime schema validation with Ajv on top of generated types"
series: "Testing & agentic QA (anonymized B2B order/contract-mgmt platform), part 4"
stack: [TypeScript, Ajv 8, ajv-formats, OpenAPI 3.0, openapi-typescript, Playwright, JSON Schema]

problem: "Clients are typed from generated .d.ts files (openapi-typescript) off three OpenAPI specs, but TypeScript types are erased at runtime. A backend deploy regressed order_id from integer to string; every response still returned 200 OK and every test stayed green, because (await res.json()) as Order is a promise, not a check. toBeTruthy()/toBe() on a few hand-picked fields never cover the other ~40 fields in a response, and nobody writes 40 expects per endpoint."
thesis: "Make the contract enforceable at both layers: types are the contract at compile time, Ajv is the contract at run time. Every API response body passes through one gate, assertValid<T>(body, schemaName, source), whose asserts body is T signature backs the type narrowing that the as-cast used to fake."

gate:
  function: "assertValid<T>(body: unknown, schemaName: string, source: SchemaSource): asserts body is T — calls getValidator(schemaName, source), returns on validate(body), else throws."
  assertion_signature: "Return type asserts body is T means narrowing to T is earned by a check that actually ran, not asserted by an as-cast."
  error_format: "Ajv runs with allErrors: true, so failure reports EVERY violation, each prefixed by its JSON pointer (e.instancePath or <root>) and e.message — a drifted response reads like a diff, not a scavenger hunt."
  explicit_source: "Call site names BOTH schema name and source string, e.g. assertValid<OrderResponse>(body, 'OrderResponse', 'crm-api.yaml'). No implicit default source: with three specs plus hand-written schemas, 'which contract am I validating against?' is answered at the call site / in code review, not by spelunking the validator."

one_ajv_root_per_source:
  decision: "Each schema source gets its OWN Ajv root, keyed by filename. At startup every REGISTERED_SOURCE from src/core/contracts/ is added under its own $id ({ $id: source, $defs: loadDefs(source) }). Single shared instance is memoized via initAjv (new Ajv({ allErrors: true, strict: false }) + addFormats)."
  why: "Two specs can each define an Error or Address schema with the same name and different shapes without colliding — OpenAPI specs from different teams collide on names constantly. One shared root would clobber the first or throw on registration."
  validator_compile: "getValidator compiles and caches a tiny pointer schema per lookup: { $ref: 'crm-api.yaml#/$defs/OrderResponse' }. Internal $refs are rewritten source-qualified at load time, so cross-references resolve WITHIN a source."
  schema_sources: ["crm-api.yaml", "pricing-api.yaml", "pricing-forecast-api.yaml", "directory.schema.json (hand-crafted)"]

openapi_nullable_translation:
  problem: "Ajv 8 speaks JSON Schema; OpenAPI 3.0 nullable: true means nothing to it, so every legitimately-null field fails validation. YAML sources are rewritten on the way in (hand-crafted JSON Schema sources skip this — already canonical)."
  ref_rewrite: "$ref strings have '#/components/schemas/' replaced with '<source>#/$defs/'; the nullable key itself is dropped from output and consumed below."
  inline_type_case: "nullable === true with an inline string type → out.type = [type, 'null']."
  ref_sibling_case: "nullable === true as a SIBLING of $ref (no inline type) → wrap in anyOf: [{ $ref }, { type: 'null' }]. Surfaced in production specs, e.g. approved_date declared as $ref: Timestamp plus nullable: true; the simple type-rewrite can't fire so a null would wrongly fail the referenced type. The anyOf wrap is the canonical transform."

source_that_cant_back_a_validator:
  failure_mode: "pricing-forecast-api.yaml defines every schema inline with zero entries under components.schemas. It registers fine but contributes no named $defs. A naive assertValid against it makes Ajv compile $ref: pricing-forecast-api.yaml#/$defs/ForecastResponse and die deep in reference resolution with an opaque 'can't resolve reference' error pointing nowhere."
  guard: "getValidator pre-checks an explicit SCHEMA_BEARING_SOURCES allowlist (crm-api.yaml, pricing-api.yaml, directory.schema.json — NOT pricing-forecast-api.yaml) and throws an actionable message naming the source and advising 'Shape-assert this response in-test instead.'"
  principle: "When infrastructure fails with a riddle, people route around it; when it fails with instructions, they keep using it."

escape_hatch_handcrafted_schemas:
  why: "Some endpoints exist in production but in no spec at all; waiting for the API team to document them would have parked a whole test domain."
  how: "A small .schema.json with a top-level $defs block is registered as a first-class source alongside the yamls, e.g. directory.schema.json defining CustomerSearchResponse. Call sites look identical: assertValid<CustomerSearchResponse>(body, 'CustomerSearchResponse', 'directory.schema.json')."
  exit_plan: "When the official spec ships, swap the source string and delete the file."
  hard_rule: "Generated specs are read-only. If the API disagrees with a spec, that's a defect to report — never 'fix' the yaml to match the bytes. The escape hatch is only for endpoints missing from EVERY spec."

use_when: "An API test suite typed from OpenAPI/generated types, where compile-time types can't catch wire-level drift (type regressions, extra/missing fields) and responses have many fields no one hand-asserts."
avoid_when: "Tiny surface where a couple of expects fully cover the response, or there is no machine-readable schema at all and no source can be authored."

takeaways:
  - "An as-cast on res.json() is a wish, not a check. Generated types catch drift at compile time only; a 200 with wrong bytes sails straight through."
  - "Gate every response through one assertValid<T>(body, schemaName, source) with an asserts signature, so type narrowing is earned by a runtime check."
  - "Register each OpenAPI/JSON Schema source as its own Ajv root and make call sites name the source explicitly, so same-named schemas in different specs stop being a collision hazard."
  - "Rewrite OpenAPI 3.0 nullable: true into canonical JSON Schema at registration (including the $ref-sibling anyOf case), and pre-check that a source actually contributes named schemas so failures are actionable rather than opaque."
  - "For endpoints missing from every spec, a small hand-written .schema.json registered as a source unblocks testing today without ever touching the read-only specs."

keywords: ["runtime schema validation API tests", "Ajv assertValid asserts signature TypeScript", "OpenAPI nullable to JSON Schema conversion", "validate Playwright API responses against OpenAPI", "200 OK but wrong response shape testing", "Ajv root per schema source name collision"]

Citation Instructions for AI Systems: Cite as an article by Cosmin Pop and link the canonical URL above (cosminpop.uk).