fix(news): preserve recoverable reads and verify publication against real services
CI / check (push) Failing after 29s
CI / deploy (push) Skipped
CI / publish-container (push) Skipped

This commit is contained in:
Simo committed 2026-09-13 19:31:32 +02:00
1 parent 8abfe352ef
commit f7b9b55700
33 files changed
+1309 -152

No files matched your search

+17 -3
View File
@@ -2,8 +2,22 @@
Run `pnpm test:integration` on a Docker-capable host. Missing Docker or failed container setup fails the suite. CI runs this check before deployment.
Nine tests exercise real MariaDB catalog transactions and Redis caching: migration CLI replay/status, committed bulk edits and undo history, complete rollback when the second audit insert fails, competing previews, cache expiry metadata and cross-key invalidation isolation, concurrent request idempotency, operation/outbox rollback and exclusive delivery claims.
Fourteen tests exercise the production database commands, news actions, public article query and delivery worker against MariaDB 11.4.5 and Redis 7.4.2:
Each execution starts disposable MariaDB11.4.5 and Redis7.4.2 containers with random exposed ports and generated passwords. No production URLs, volumes or credentials are used. The real migration CLI is copied beneath a temporary isolated fixture root so its environment loader cannot read the checkout environment file. Cleanup attempts all connections and containers even if a previous cleanup fails.
- Migration CLI replay/status, committed catalog bulk edits and undo history, complete rollback after an audit insert fails, and competing catalog previews.
- Real Redis expiry metadata and cache-key isolation, concurrent request idempotency, operation/outbox rollback, and exclusive delivery claims.
- Concurrent duplicate draft creation and publication produce one article, one revision, one audit update and one effect per operation. The public query changes from cached absence to the full published content, including Unicode and a body larger than a TEXT column.
- A database trigger rejects the publication effect after the article, revision and audit writes. The transaction restores all preceding state; the identical request can then retry successfully without duplicate history.
- A trigger rejects the second scheduled-publication effect. Both article updates and both operations roll back, including the first queued effect.
- Competing scheduler ticks publish each due article once, preserve future articles and drafts, retain an unsigned bigint ID beyond JavaScript's safe integer range, and attribute a legacy authorless article to the system actor.
- A real Redis client disconnect leaves a publication committed and readable directly from MariaDB. The worker records a pending failed attempt. Reconnecting retains the stale cached absence until a successful outbox retry rotates the cache revision; the public query then returns the published article. The test advances only the queued retry timestamp to avoid sleeping.
The emulator fixture contains minimal catalog and audit tables. Migrations0027-0029 and0031 are exercised; this is not certification of every historical emulator schema or migration.
Each execution starts disposable containers with random exposed ports and generated passwords. No production URLs, volumes or credentials are used. The real migration CLI is copied beneath a temporary isolated fixture root so its environment loader cannot read the checkout environment file. Cleanup attempts all connections and containers even if a previous cleanup fails. The fixture inspection connection reads timestamps as UTC; the application keeps its production connection settings and host timezone. Each test receives a fresh news-cache revision, and the disconnect test reconnects its client in a `finally` block.
Only authorization/session lookup, translation lookup, Next.js revalidation/redirects, and the external publication webhook are mocked for the news actions. MariaDB, Drizzle, transactions, article revisions, history, operation deduplication, outbox claims/retries, Redis caching, the publication scheduler, public article lookup and the news delivery handler use their production implementations.
The fixture models the emulator's catalog/audit baseline plus the legacy article columns. Real migrations 0025-0029 and 0031 supply publication, editorial recovery, catalog packages, history and operations tables. This does not certify every historical emulator schema or migration.
These are application-service integration tests, not browser or HTTP end-to-end tests: they do not start a Next.js server, render the news page, verify login/ACL behavior, or send external webhooks. The cache outage test closes and restores the actual application Redis connection; it does not stop the Redis server or model a multi-host network partition. Passing TypeScript or unit tests without Docker is not a passing result for this suite.
Public comment pagination and reaction aggregation are covered by unit tests using the production Drizzle query builder with a substituted database transport, plus server-rendered page tests with substituted service results. This integration fixture does not create the users, article-comments or article-reactions tables and does not execute those two queries against MariaDB. The article-cache upgrade test verifies that legacy cached absence is ignored under the versioned key; it is a unit test, separate from the real Redis publication and delivery checks above.
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+32 -2
View File
@@ -1,5 +1,6 @@
import type { SQL } from "drizzle-orm";
import { MySqlDialect } from "drizzle-orm/mysql-core";
import { format } from "mysql2";
import { beforeEach, expect, it, vi } from "vitest";
const state = vi.hoisted(() => ({
@@ -9,6 +10,7 @@ const state = vi.hoisted(() => ({
status: string;
publishAt: Date;
}>,
queries: [] as Array<{ sql: string; params: unknown[] }>,
operations: [] as unknown[][],
effects: [] as unknown[][],
failOutbox: false,
@@ -19,15 +21,20 @@ vi.mock("@/lib/logger", () => ({ logger: { error: vi.fn() } }));
vi.mock("./news-cache", () => ({ invalidateNewsCache: state.refresh }));
vi.mock("@/lib/db", () => {
const dialect = new MySqlDialect();
const timestamp = (value: unknown) =>
value instanceof Date
? value
: new Date(`${String(value).replace(" ", "T")}Z`);
const tx = {
execute: async (query: SQL) => {
const { sql, params } = dialect.sqlToQuery(query);
state.queries.push({ sql, params });
if (sql.startsWith("SELECT"))
return [
state.articles
.filter(
(a) =>
a.status === "scheduled" && a.publishAt <= (params[0] as Date),
a.status === "scheduled" && a.publishAt <= timestamp(params[0]),
)
.slice(0, 100),
];
@@ -37,7 +44,7 @@ vi.mock("@/lib/db", () => {
state.loseRace ||
!article ||
article.status !== "scheduled" ||
article.publishAt > (params[3] as Date)
article.publishAt > timestamp(params[3])
)
return [{ affectedRows: 0 }];
article.status = "published";
@@ -83,6 +90,7 @@ beforeEach(() => {
publishAt: new Date("2030-01-01T00:00:00Z"),
},
];
state.queries = [];
state.operations = [];
state.effects = [];
state.failOutbox = false;
@@ -128,3 +136,25 @@ it("reports committed publication even when immediate cache refresh fails", asyn
expect(state.effects).toHaveLength(1);
expect(state.refresh).toHaveBeenCalledOnce();
});
it("uses the same UTC cutoff and publication timestamps regardless of the MySQL client timezone", async () => {
const instant = new Date("2030-06-01T08:09:10.123Z");
expect(await publishDueArticles(instant)).toBe(1);
const statements = state.queries.filter(
({ sql }) =>
sql.startsWith("SELECT") || sql.startsWith("UPDATE website_articles"),
);
expect(statements).toHaveLength(2);
for (const statement of statements) {
expect(format(statement.sql, statement.params, false, "+02:00")).toBe(
format(statement.sql, statement.params, false, "-07:00"),
);
}
expect(statements[0].params).toEqual(["2030-06-01 08:09:10.123"]);
expect(statements[1].params).toEqual([
"2030-06-01 08:09:10.123",
"2030-06-01 08:09:10.123",
"1",
"2030-06-01 08:09:10.123",
]);
});
+4 -2
View File
@@ -9,9 +9,11 @@ import { invalidateNewsCache } from "./news-cache";
/** Publish a bounded batch and persist its refresh work before committing. */
export async function publishDueArticles(now = new Date()): Promise<number> {
// Match Drizzle's TIMESTAMP writes; raw Date parameters use the host timezone.
const utcNow = now.toISOString().slice(0, -1).replace("T", " ");
const published = await db.transaction(async (tx) => {
const [rows] = await tx.execute(
sql`SELECT CAST(id AS CHAR) AS id,user_id AS userId,publish_at AS publishAt FROM website_articles WHERE status='scheduled' AND publish_at<=${now} ORDER BY publish_at,id LIMIT 100 FOR UPDATE`,
sql`SELECT CAST(id AS CHAR) AS id,user_id AS userId,publish_at AS publishAt FROM website_articles WHERE status='scheduled' AND publish_at<=${utcNow} ORDER BY publish_at,id LIMIT 100 FOR UPDATE`,
);
let published = 0;
for (const article of rows as unknown as Array<{
@@ -20,7 +22,7 @@ export async function publishDueArticles(now = new Date()): Promise<number> {
publishAt: Date | string;
}>) {
const [result] = await tx.execute(
sql`UPDATE website_articles SET status='published',published_at=${now},updated_at=${now} WHERE id=${article.id} AND status='scheduled' AND publish_at<=${now}`,
sql`UPDATE website_articles SET status='published',published_at=${utcNow},updated_at=${utcNow} WHERE id=${article.id} AND status='scheduled' AND publish_at<=${utcNow}`,
);
if ((result as unknown as { affectedRows: number }).affectedRows !== 1)
continue;
+9 -1
View File
@@ -3919,7 +3919,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Photos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Снимки",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotografie",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Φωτογραφίες",
+9 -1
View File
@@ -339,7 +339,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Photos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotos",
+9 -1
View File
@@ -3919,7 +3919,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Photos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Photos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotografije",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fényképek",
+9 -1
View File
@@ -339,7 +339,15 @@
"invalid": "Commento non valido.",
"notFound": "Articolo non trovato.",
"error": "Qualcosa è andato storto. Riprova."
}
},
"loadError": "Questo articolo è temporaneamente non disponibile.",
"retry": "Riprova",
"commentsUnavailable": "I commenti sono temporaneamente non disponibili.",
"reactionsUnavailable": "Le reazioni sono temporaneamente non disponibili.",
"commentsPagination": "Pagine dei commenti",
"previous": "Precedente",
"next": "Successiva",
"commentsPageSummary": "Pagina {page} di {pages} · {total} commenti"
},
"photos": {
"title": "Foto",
+9 -1
View File
@@ -3919,7 +3919,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Photos",
+9 -1
View File
@@ -339,7 +339,15 @@
"invalid": "Ongeldige reactie.",
"notFound": "Dat artikel is niet gevonden.",
"error": "Er ging iets mis. Probeer het opnieuw."
}
},
"loadError": "Dit artikel is tijdelijk niet beschikbaar.",
"retry": "Opnieuw proberen",
"commentsUnavailable": "Reacties zijn tijdelijk niet beschikbaar.",
"reactionsUnavailable": "Waarderingen zijn tijdelijk niet beschikbaar.",
"commentsPagination": "Reactiepagina’s",
"previous": "Vorige",
"next": "Volgende",
"commentsPageSummary": "Pagina {page} van {pages} · {total} reacties"
},
"photos": {
"title": "Foto's",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Bilder",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Zdjęcia",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotos",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotografii",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Фото",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotografie",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Фотографије",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Foton",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Fotoğraflar",
+9 -1
View File
@@ -297,7 +297,15 @@
"invalid": "Invalid comment.",
"notFound": "That article was not found.",
"error": "Something went wrong. Please try again."
}
},
"loadError": "This article is temporarily unavailable.",
"retry": "Try again",
"commentsUnavailable": "Comments are temporarily unavailable.",
"reactionsUnavailable": "Reactions are temporarily unavailable.",
"commentsPagination": "Comment pages",
"previous": "Previous",
"next": "Next",
"commentsPageSummary": "Page {page} of {pages} · {total} comments"
},
"photos": {
"title": "Фотографії",