test(imaging): retry transient edge failures instead of flaking the suite
CI / check (push) Successful in 37s
CI / tests-integration (push) Successful in 2m3s
CI / tests-unit (push) Successful in 2m8s
CI / tests-imaging-browsers (push) Successful in 2m15s
CI / tests-ui (push) Successful in 2m54s
CI / preflight (push) Skipped
CI / tests-quality (push) Successful in 1m14s
CI / deploy (push) Successful in 3m37s

The cross-browser imaging job runs against the public origin, so every request
goes through Cloudflare. One chromium run of "rejects malformed figures ›
missing figure" came back 525 (Cloudflare: TLS handshake to the origin failed)
after a 40s stall — the request never reached the CMS, so a 400 was impossible
to observe and the assertion was effectively testing the network. It passed on
both firefox and webkit, which is the signature of an edge hiccup rather than a
regression.

`e2e/imaging/imaging-target.ts` now owns `getThroughEdge` / `fetchThroughEdge`,
which retry on the statuses that mean "the app was never reached"
(502/503/504 plus Cloudflare's 520-527) with a short backoff, and both
edge-fronted specs use them. A genuinely broken edge still exhausts the retries
and fails the run, so nothing is masked.

Verified: 84 passed / 21 skipped / 0 failed against epicnabbo.nl with
IMAGING_EDGE=1 across chromium, firefox and webkit.
This commit is contained in:
openhands committed 2026-10-15 20:30:38 +02:00
1 parent 77a0d1010c
commit 480343d5a5
3 files changed
+117 -22

No files matched your search

+52
View File
@@ -1,3 +1,5 @@
import type { APIRequestContext, APIResponse } from "@playwright/test";
/**
* Where the imaging specs are pointed.
*
@@ -12,6 +14,56 @@
*/
export const IS_EDGE = process.env.IMAGING_EDGE === "1";
/**
* Failures that mean "the request never reached the app".
*
* Cloudflare answers the 520-527 family when its own handshake to the origin
* fails, and gateways add 502/503/504. Retrying those is not papering over a
* bug: the assertion under test is about the CMS, and a 525 says nothing about
* it. A genuinely broken edge still exhausts the retries and fails the test.
*/
const TRANSIENT_EDGE_STATUSES = new Set([
502, 503, 504, 520, 521, 522, 523, 524, 525, 526, 527,
]);
const EDGE_RETRY_ATTEMPTS = 3;
const EDGE_RETRY_DELAY_MS = 500;
async function withEdgeRetry(
send: () => Promise<APIResponse>,
): Promise<APIResponse> {
let response = await send();
for (
let attempt = 1;
attempt < EDGE_RETRY_ATTEMPTS &&
TRANSIENT_EDGE_STATUSES.has(response.status());
attempt += 1
) {
await new Promise((resolve) =>
setTimeout(resolve, EDGE_RETRY_DELAY_MS * attempt),
);
response = await send();
}
return response;
}
/** GET that survives a transient edge/transport failure. */
export function getThroughEdge(
request: APIRequestContext,
url: string,
options: Parameters<APIRequestContext["get"]>[1] = {},
): Promise<APIResponse> {
return withEdgeRetry(() => request.get(url, options));
}
/** POST/OPTIONS/… counterpart of `getThroughEdge`. */
export function fetchThroughEdge(
request: APIRequestContext,
url: string,
options: Parameters<APIRequestContext["fetch"]>[1] = {},
): Promise<APIResponse> {
return withEdgeRetry(() => request.fetch(url, options));
}
// The exact figure that regressed in production: the game client appends a
// trailing `.<n>` segment that the renderer ignores, but the previous
// validation regex rejected with a 400.