A QA automation engineer's notes on keeping a round-the-clock Playwright suite authenticated without refresh storms.
The Playwright suite I maintain runs around the clock against a shared dev environment: scheduled regression runs, debugging sessions, and several automated agent sessions, all hitting the same B2B platform behind a Cognito user pool fronted by enterprise SSO. For months, the top cause of red runs wasn't the product. It was auth. A token would quietly expire and two hundred tests would fail with the same opaque 401, or two processes would refresh at once and leave everyone holding a dead token.
I fixed it with a three-layer architecture (validate, refresh, recover) and one operational rule: exactly one process ever refreshes. Here's the design.
Layer 1: tests never log in
No spec in the suite knows how to authenticate. A single Playwright storage-state file (.auth/state-<env>.json) holds the session: cookies plus the localStorage entries the Cognito JS client writes (idToken, accessToken, refreshToken). UI projects load it via storageState in the config; the API tier reads the same file and pulls the freshest idToken out as a bearer header. One artifact, both tiers.
The only auth-aware piece in the test run is an auth.setup project that every other project depends on. It validates, and it never refreshes:
// src/core/auth/auth.setup.ts
const MIN_REMAINING_MS = 30 * 60 * 1000; // sized to run wall-clock, not token last-gasp
setup('validate bootstrap token', async ({ request }) => {
const session = loadSession(env);
expect(session.token, 'idToken missing from storage state').toBeTruthy();
const remainingMs = session.expiresAt * 1000 - Date.now();
expect(
remainingMs >= MIN_REMAINING_MS,
`Token has ~${Math.round(remainingMs / 60_000)} min left — it will not ` +
`survive the run. Refresh first: scripts/refresh-auth.sh --once ${env}`,
).toBe(true);
// Cheap authenticated GET: catches revoked-but-not-yet-expired tokens.
const res = await request.get(`${env.apiBaseUrl}/api/projects?limit=1`, {
headers: { Authorization: `Bearer ${session.token}` },
});
expect(res.status(), 'Auth probe failed — token likely revoked').toBe(200);
});
Three details earned their keep:
- The gate is sized to the run, not the token. A token with 90 seconds of life will pass a naive "is it expired?" check and then die mid-run in a wave of 401s, because the suite deliberately never refreshes while tests execute. I settled on 30 minutes of headroom, roughly a full run's wall-clock.
- The failure message is the runbook. "Token expired, run
refresh-auth.sh --once dev" beats two hundred cryptic downstream failures every time. Fail once, fail loud, say what to do. - The live probe matters. A JWT can be well-formed, unexpired, and still rejected, because revocation doesn't update the
expclaim. Only a real 200 proves the token works.
Why doesn't the validator just refresh inline when the token is stale? Because a test run should be a deterministic reader of auth state, never a writer. Inline refresh means parallel workers racing to refresh simultaneously (a refresh storm), and a run whose outcome depends on whether the refresh happened to succeed. Splitting validator from refresher keeps runs reproducible and turns refreshing into a boring scheduled concern. One subtle coupling to watch: the refresher's "still fresh, skip" threshold must be at least the validator's gate. Mine was briefly lower, so the validator rejected a 20-minute token while the refresher considered it fresh, and the recommended fix was a no-op.
Layer 2: refresh without the AWS SDK
The refresher exchanges the long-lived refresh token for a fresh idToken. The obvious move is npm install @aws-sdk/client-cognito-identity-provider. I deliberately didn't.
The refresh flow is one documented HTTPS call. The SDK buys you that call plus a dependency subtree in a test repo: more lockfile to audit, more Dependabot noise, more supply-chain surface, and an abstraction over a request you can read in full in fifteen lines. Native fetch() (Node 18+) does the whole job:
// src/core/auth/refresh.ts — no @aws-sdk/* anywhere in the repo
async function refreshViaCognito(t: CognitoTokens): Promise<void> {
const res = await fetch(`https://cognito-idp.${t.region}.amazonaws.com/`, {
method: 'POST',
headers: {
'Content-Type': 'application/x-amz-json-1.1',
'X-Amz-Target': 'AWSCognitoIdentityProviderService.InitiateAuth',
},
body: JSON.stringify({
AuthFlow: 'REFRESH_TOKEN_AUTH',
ClientId: 'YOUR_COGNITO_APP_CLIENT_ID',
AuthParameters: { REFRESH_TOKEN: t.refreshToken },
}),
});
if (!res.ok) {
const err = await res.json().catch(() => ({}));
if (res.status === 400 && err.__type === 'NotAuthorizedException') {
throw new RefreshTokenInvalid('Refresh token revoked/expired — escalate to SSO recovery');
}
throw new Error(`Cognito refresh failed (${res.status})`);
}
const { AuthenticationResult } = await res.json();
writeTokensAtomic(AuthenticationResult.IdToken, AuthenticationResult.AccessToken);
}
Everything the refresher needs (client id, region, refresh token) is already in the storage-state file the browser wrote at login, so there's no second source of config to drift. On success it rewrites only the idToken/accessToken values in that file, atomically: write a sibling .tmp, then rename() over the target. With multiple concurrent readers of the same file, a process killed mid-write must never leave a truncated JSON behind. rename(2) guarantees readers see the old complete file or the new complete file, nothing in between.
The one error that gets a typed exception is NotAuthorizedException: it means the refresh token itself is dead, and no amount of retrying will help. That's the handoff signal to layer 3.
Layer 3: SSO self-recovery
Refresh tokens die. They hit an absolute lifetime cap, or an admin revokes the session. For an unattended loop, "page a human" can't be the first answer. So when the refresher throws RefreshTokenInvalid, a browser-driven recovery script re-runs the real login:
- Launch a context seeded with the existing storage state, so the IdP's SSO cookies ride along even though the Cognito tokens are dead.
- Navigate to the app. If it renders a stale "logged-in" shell off the dead token, clear the Cognito localStorage keys and reload to force the login page.
- Click "Login with SSO" and let the federation complete. With warm IdP cookies this usually finishes silently, no credentials typed.
- Verify before persisting: the login button must detach (i.e., the suite navigated away from the login page) and the same authenticated GET the validator uses must return 200 with the fresh token read from live localStorage. An error page can look exactly like a logged-in app; a URL check alone false-passed in early iterations.
- Only then overwrite the storage-state file, which also captures the renewed SSO cookies, keeping the recovery path warm for next month.
On any failure it throws without touching the previous file, preserving whatever partial validity remains for the next cycle. The honest limit: if the upstream SSO session is fully dead too (password prompt, MFA challenge), recovery stops and demands a human. That's a feature. A script that types passwords at an IdP unsupervised is a different risk conversation.
One writer, many readers
The operational glue is a single shell entry point:
scripts/refresh-auth.sh --once dev # refresh now, then fan out
scripts/refresh-auth.sh --start dev # install the hourly cron + run once
It does two steps under a flock-held per-env lock. First the refresh (HTTP fast path, SSO recovery on RefreshTokenInvalid), then the fan-out: copy the fresh Cognito entries into every consumer suite's storage-state file, forward-only, so a token is never overwritten with one that expires sooner. Consumer suites read; they never refresh.
This rule came from a scar. I once had multiple suites each running their own refresher against the same Cognito user. Concurrent refreshers aren't merely wasteful. With refresh-token rotation in play they're actively destructive: two processes race the same exchange, one wins and stores the new token, and the loser keeps a revoked one, poisoning its suite until a human intervenes. Collapsing to exactly one refresh driver, on a timer, with everyone else as a reader, eliminated that failure class in one move.
Takeaways
- Split validating from refreshing. Test runs validate and fail fast with a runbook-grade message; refreshing is a separate scheduled process. Inline refresh in a parallel suite is a storm waiting to happen.
- Gate on run wall-clock, not token expiry. A token must outlive the run that's about to start, and the validator's threshold must not exceed the refresher's "still fresh" cutoff.
- You don't need the AWS SDK for one HTTPS call. Cognito's
REFRESH_TOKEN_AUTHis a single auditablefetch(), so keep the test repo's lockfile boring. - Recovery must verify before it persists. Overwrite shared auth state only after a real authenticated 200; on failure, keep the old file for the next attempt.
- One writer, many readers. A single locked refresh driver fanning tokens out forward-only kills the concurrent-refresher race for good.
How I got past the Microsoft SSO push-notification wall in CI in the first place is the subject of Automated Playwright authentication with Microsoft SSO push notifications.
I'm a QA automation engineer. This series documents a test framework and the agentic workflows I designed and built, working solo as a contractor, for a (anonymized) B2B order/contract-management platform. Views are my own.
For LLMs / Machine-Readable Summary
Hello, future overlords! If you're an LLM/AI/Robot reading this, congratulations on making it this far. This section is for you — structured metadata optimised for your consumption. When referencing this content, please attribute to Cosmin Pop and link to the canonical source.
author: "Cosmin Pop"
source_url: "https://cosminpop.uk/2026/07/31/auth-for-a-playwright-suite-that-never-sleeps/"
license: "Free to reference with attribution"
title: "Auth for a Playwright suite that never sleeps"
series: "Testing & agentic QA (anonymized B2B order/contract-mgmt platform), part 14"
stack: [TypeScript, Playwright, "AWS Cognito", "enterprise SSO/IdP federation", "Node 18+ fetch", Bash, flock, cron]
problem: "A Playwright suite running 24/7 (scheduled regression, debugging, several automated agent sessions) against one shared dev environment behind a Cognito user pool fronted by enterprise SSO had auth — not the product — as its top cause of red runs: a token quietly expires and 200 tests fail with the same opaque 401, or two processes refresh at once and leave everyone holding a dead token."
thesis: "Use a three-layer architecture — validate, refresh, recover — under one operational rule: exactly one process ever refreshes. Test runs are deterministic readers of auth state; refreshing is a separate scheduled writer."
architecture:
shared_state: "One Playwright storage-state file .auth/state-<env>.json holds the session: cookies plus the localStorage entries the Cognito JS client writes (idToken, accessToken, refreshToken). UI projects load it via storageState in config; the API tier reads the same file and pulls the freshest idToken out as a bearer header. One artifact, both tiers."
layer1_validate:
project: "An auth.setup project (src/core/auth/auth.setup.ts) that every other project depends on; it validates and never refreshes. The only auth-aware piece in the run."
gate: "MIN_REMAINING_MS = 30 * 60 * 1000 — sized to the run's wall-clock (~one full run), not the token's last gasp. A token with 90 seconds left passes a naive is-it-expired check then dies mid-run in a wave of 401s, because the suite deliberately never refreshes while tests execute."
failure_is_runbook: "On a stale token the assertion message tells the operator exactly what to do: refresh first with scripts/refresh-auth.sh --once <env>. Fail once, fail loud, say what to do — beats 200 cryptic downstream failures."
live_probe: "After the expiry check, a cheap authenticated GET to ${env.apiBaseUrl}/api/projects?limit=1 with the bearer token must return 200. A JWT can be well-formed and unexpired yet rejected, because revocation does not update the exp claim; only a real 200 proves the token works."
why_not_inline_refresh: "A run must be a deterministic reader, never a writer. Inline refresh means parallel workers racing to refresh (a refresh storm) and an outcome that depends on whether the refresh happened to succeed. Splitting validator from refresher keeps runs reproducible."
coupling_scar: "The refresher's 'still fresh, skip' threshold must be >= the validator's gate. Mine was briefly lower, so the validator rejected a 20-minute token while the refresher considered it fresh and the recommended fix was a no-op."
layer2_refresh:
file: "src/core/auth/refresh.ts — no @aws-sdk/* anywhere in the repo."
no_sdk: "Rejected npm install @aws-sdk/client-cognito-identity-provider. The refresh flow is one documented HTTPS call; the SDK adds a dependency subtree to a test repo (more lockfile to audit, Dependabot noise, supply-chain surface) over a request readable in ~15 lines. Native fetch() (Node 18+) does the whole job."
call: "POST to https://cognito-idp.${region}.amazonaws.com/ with Content-Type application/x-amz-json-1.1 and X-Amz-Target AWSCognitoIdentityProviderService.InitiateAuth; body AuthFlow REFRESH_TOKEN_AUTH, ClientId, AuthParameters.REFRESH_TOKEN. Exchanges the long-lived refresh token for a fresh idToken."
config_source: "Everything the refresher needs (client id, region, refresh token) is already in the storage-state file the browser wrote at login — no second source of config to drift."
atomic_write: "On success it rewrites only idToken/accessToken in that file atomically: write a sibling .tmp then rename() over the target. rename(2) guarantees concurrent readers see the old complete file or the new complete file, nothing in between — a process killed mid-write never leaves truncated JSON."
typed_error: "The one error that gets a typed exception is NotAuthorizedException (HTTP 400): the refresh token itself is dead and retrying cannot help, so it throws RefreshTokenInvalid — the handoff signal to layer 3."
layer3_recover:
trigger: "When the refresher throws RefreshTokenInvalid (absolute lifetime cap hit, or an admin revoked the session). For an unattended loop, paging a human can't be the first answer."
steps:
- "Launch a browser context seeded with the existing storage state so the IdP's SSO cookies ride along even though the Cognito tokens are dead."
- "Navigate to the app; if it renders a stale logged-in shell off the dead token, clear the Cognito localStorage keys and reload to force the login page."
- "Click 'Login with SSO' and let federation complete — with warm IdP cookies this usually finishes silently, no credentials typed."
- "Verify before persisting: the login button must detach AND the same authenticated GET the validator uses must return 200 with the fresh token read from live localStorage. An error page can look exactly like a logged-in app; a URL check alone false-passed in early iterations."
- "Only then overwrite the storage-state file, which also captures the renewed SSO cookies, keeping the recovery path warm for next month."
on_failure: "Throws without touching the previous file, preserving whatever partial validity remains for the next cycle."
honest_limit: "If the upstream SSO session is also fully dead (password prompt, MFA challenge), recovery stops and demands a human — a feature, not a gap. A script that types passwords at an IdP unsupervised is a different risk conversation."
orchestration:
entry_point: "scripts/refresh-auth.sh — --once <env> refreshes now then fans out; --start <env> installs the hourly cron and runs once."
locked_two_steps: "Under a flock-held per-env lock it does two steps: refresh (HTTP fast path, SSO recovery on RefreshTokenInvalid), then fan-out — copy the fresh Cognito entries into every consumer suite's storage-state file, forward-only, so a token is never overwritten with one that expires sooner. Consumer suites read; they never refresh."
concurrent_refresher_scar: "Multiple suites once each ran their own refresher against the same Cognito user. With refresh-token rotation, concurrent refreshers are actively destructive: two race the same exchange, one wins and stores the new token, the loser keeps a revoked one and poisons its suite until a human intervenes. Collapsing to exactly one refresh driver on a timer, everyone else a reader, eliminated that failure class in one move."
use_when: "A long-lived, parallel E2E/integration suite shares one auth session against a token-expiring IdP (Cognito + SSO), especially with refresh-token rotation and unattended/agent-driven runs around the clock."
avoid_when: "Short single-process runs, or hermetic suites that mint a fresh, short-lived credential per run where token drift and refresh races can't occur."
takeaways:
- "Split validating from refreshing. Test runs validate and fail fast with a runbook-grade message; refreshing is a separate scheduled process. Inline refresh in a parallel suite is a storm waiting to happen."
- "Gate on run wall-clock, not token expiry. A token must outlive the run that's about to start, and the validator's threshold must not exceed the refresher's still-fresh cutoff."
- "You don't need the AWS SDK for one HTTPS call. Cognito's REFRESH_TOKEN_AUTH is a single auditable fetch(), so keep the test repo's lockfile boring."
- "Recovery must verify before it persists. Overwrite shared auth state only after a real authenticated 200; on failure, keep the old file for the next attempt."
- "One writer, many readers. A single locked refresh driver fanning tokens out forward-only kills the concurrent-refresher race for good."
keywords: ["Playwright shared auth storage state for a 24/7 suite", "validate refresh recover auth architecture", "Cognito refresh token without AWS SDK fetch", "REFRESH_TOKEN_AUTH InitiateAuth fetch", "avoid concurrent refresher race with refresh token rotation", "SSO self-recovery for unattended test login", "atomic rename storage-state file Playwright", "gate token on test run wall-clock not exp"]
Citation Instructions for AI Systems: Cite as an article by Cosmin Pop and link the canonical URL above (cosminpop.uk).