fix(catalog): read furnidata from one cache, purge the gamedata edge on write
Gitea Actions Runner Test / test-job (push) Successful in 1s
CI / check (push) Successful in 36s
CI / tests-integration (push) Successful in 1m52s
CI / tests-unit (push) Successful in 1m56s
CI / tests-ui (push) Successful in 2m48s
CI / preflight (push) Skipped
CI / deploy (push) Successful in 2m27s
Gitea Actions Runner Test / test-job (push) Successful in 1s
CI / check (push) Successful in 36s
CI / tests-integration (push) Successful in 1m52s
CI / tests-unit (push) Successful in 1m56s
CI / tests-ui (push) Successful in 2m48s
CI / preflight (push) Skipped
CI / deploy (push) Successful in 2m27s
Furniture was not always loading completely because the same file was cached twice and nobody could reach the client. The catalog items loader kept its own 30s TTL copy of FurnitureData.json next to the mtime-validated cache in `furni-data.ts`. An import cleared only the second one, so the catalog table kept serving pre-import furnidata — empty descriptions and revisions — until the TTL ran out. The loader now reads through `readFurniData`, which revalidates on mtime+size and is reset by every write, so there is exactly one cache and it cannot go stale on its own. `invalidateFurniDataCache` and its single call site are gone with it. The client was worse: nginx served all of /gamedata/ with `max-age=604800`, and the `cms-gamedata` purge that would have fixed it hung off the catalog Git export, which is disabled in production. A freshly imported item was invisible in the client for up to seven days no matter how often you imported. - `writeFurniData` now purges the gamedata edge tag itself. One place covers import, batch, resync, regen, nitro-editor, translate and dedupe. It is fire-and-forget and swallowed at every level: a stale edge copy is bounded by the edge TTL, so a failed purge must never fail an import. - nginx splits /gamedata/ by how mutable the content is: config/ gets `max-age=300, must-revalidate`, bundled/ `max-age=3600, must-revalidate`, and the content-addressed trees (c_images, album*, clothes) keep the long TTL. `must-revalidate` is the point — the client now revalidates instead of replaying the old body. All three keep `Cache-Tag: cms-gamedata` so the purge still reaches them. - A 30-minute safety-net purge in the jobs worker covers the case where Cloudflare was unreachable at write time.
This commit is contained in:
1 parent
cede541813
commit
a6cc3cafa9
5 files changed
+125
-44
No files matched your search
@@ -413,9 +413,6 @@ export async function translateCatalogItems(input: {
|
||||
};
|
||||
}
|
||||
const { items } = parsed.data;
|
||||
const { invalidateFurniDataCache } = await import(
|
||||
"@/lib/services/catalog-items-loader"
|
||||
);
|
||||
const { patchFurniEntryNames } = await import("@/lib/services/furni-data");
|
||||
const { patchLocalizedFurniDataEntries } = await import(
|
||||
"@/lib/services/furni-data-i18n"
|
||||
@@ -510,9 +507,9 @@ export async function translateCatalogItems(input: {
|
||||
furniPatches.length > 0
|
||||
? await patchFurniEntryNames(furniPatches)
|
||||
: { updated: 0, inserted: 0 };
|
||||
if (furniResult.updated > 0 || furniResult.inserted > 0) {
|
||||
invalidateFurniDataCache();
|
||||
}
|
||||
// `furniResult` needs no follow-up: `patchFurniEntryNames` writes through
|
||||
// `writeFurniData`, which resets the shared in-process cache and purges
|
||||
// the gamedata edge tag itself.
|
||||
|
||||
if (localizedEntries.length > 0) {
|
||||
try {
|
||||
|
||||
@@ -1,52 +1,39 @@
|
||||
import { promises as fs } from "node:fs";
|
||||
import { asc, sql } from "drizzle-orm";
|
||||
import { numericValue } from "@/features/catalog/domain/offer-input";
|
||||
import { CatalogPages, CatalogPagesBc, db, queryRows } from "@/lib/db";
|
||||
import { getFurnitureDataPath } from "@/lib/services/furni-data";
|
||||
import { readFurniData } from "@/lib/services/furni-data";
|
||||
import { getHabboGamedataHotel } from "@/lib/services/habbo-gamedata-hotel";
|
||||
|
||||
// Cache FurnitureData.json in memory with TTL to avoid repeated disk I/O
|
||||
let furniDataCache: {
|
||||
roomitemtypes?: {
|
||||
furnitype?: Array<{
|
||||
id: number;
|
||||
description?: string;
|
||||
classname?: string;
|
||||
revision?: number;
|
||||
}>;
|
||||
};
|
||||
wallitemtypes?: {
|
||||
furnitype?: Array<{
|
||||
id: number;
|
||||
description?: string;
|
||||
classname?: string;
|
||||
revision?: number;
|
||||
}>;
|
||||
};
|
||||
} | null = null;
|
||||
let furniDataCacheTime = 0;
|
||||
const FURNI_CACHE_TTL = 30_000; // 30 seconds
|
||||
interface FurnitypeEntry {
|
||||
id: number;
|
||||
description?: string;
|
||||
classname?: string;
|
||||
revision?: number;
|
||||
}
|
||||
|
||||
async function getFurnitureData() {
|
||||
if (furniDataCache && Date.now() - furniDataCacheTime < FURNI_CACHE_TTL)
|
||||
return furniDataCache;
|
||||
const furniDataPath = await getFurnitureDataPath();
|
||||
interface FurnitureData {
|
||||
roomitemtypes?: { furnitype?: FurnitypeEntry[] };
|
||||
wallitemtypes?: { furnitype?: FurnitypeEntry[] };
|
||||
}
|
||||
|
||||
/**
|
||||
* FurnitureData.json via the shared reader.
|
||||
*
|
||||
* This module used to keep its own 30s TTL copy of the file next to the
|
||||
* mtime-validated cache in `furni-data.ts`. Two caches over one file meant an
|
||||
* import cleared only the first one, so this loader kept serving the
|
||||
* pre-import furnidata (empty descriptions and revisions) until the TTL ran
|
||||
* out. `readFurniData` revalidates on `mtimeMs`+`size` and is reset by every
|
||||
* write, so there is now exactly one cache and it cannot go stale on its own.
|
||||
*/
|
||||
async function getFurnitureData(): Promise<FurnitureData | null> {
|
||||
try {
|
||||
const raw = await fs.readFile(furniDataPath, "utf-8");
|
||||
furniDataCache = JSON.parse(raw);
|
||||
furniDataCacheTime = Date.now();
|
||||
return furniDataCache;
|
||||
return (await readFurniData()) as FurnitureData;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Invalidate the FurnitureData cache after writes */
|
||||
export function invalidateFurniDataCache() {
|
||||
furniDataCache = null;
|
||||
furniDataCacheTime = 0;
|
||||
}
|
||||
|
||||
interface RawItem {
|
||||
id: number;
|
||||
itemIds: string;
|
||||
@@ -292,7 +279,7 @@ export async function loadCatalogItemsData(
|
||||
spriteToBaseId.set(b.spriteId, b.id);
|
||||
}
|
||||
|
||||
// Read FurnitureData.json (cached in memory)
|
||||
// Read FurnitureData.json through the shared mtime-validated reader.
|
||||
const foundBaseIds = new Set<number>();
|
||||
const furniDescriptionMap: Record<number, string> = {};
|
||||
const furniRevisionMap: Record<
|
||||
|
||||
@@ -277,6 +277,35 @@ export async function writeFurniData(
|
||||
// The file just changed: force the next read to reparse instead of serving
|
||||
// a possibly-mutated cached object.
|
||||
cachedFurniData = null;
|
||||
void purgeFurniGamedataEdge(
|
||||
`furnidata written (${targets.length} target(s))`,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop the edge copy of every gamedata file after a furniture write.
|
||||
*
|
||||
* The Habbo client reads FurnitureData.json from /gamedata/, which nginx
|
||||
* serves with its own TTL and Cloudflare with an edge TTL. Neither knows
|
||||
* about a write that happened on disk, so without this purge a freshly
|
||||
* imported item stays invisible in the client until both expire. The purge
|
||||
* used to hang off the catalog Git export, which is disabled in production,
|
||||
* so in practice it never ran.
|
||||
*
|
||||
* Fire-and-forget and swallowed on every level: a stale edge copy is bounded
|
||||
* by the edge TTL, so failing to purge must never fail the import itself.
|
||||
* Standalone scripts reach this through `tsx` without the react-server
|
||||
* condition, where `server-only` throws on import, hence the try/catch.
|
||||
*/
|
||||
async function purgeFurniGamedataEdge(reason: string): Promise<void> {
|
||||
try {
|
||||
const { EDGE_CACHE_TAGS, purgeEdgeCache } = await import(
|
||||
"@/lib/edge-cache"
|
||||
);
|
||||
await purgeEdgeCache([EDGE_CACHE_TAGS.gamedata], reason);
|
||||
} catch {
|
||||
/* no Cloudflare credentials, or a non-server runtime: nothing to purge */
|
||||
}
|
||||
}
|
||||
|
||||
export function buildFurniEntry(params: {
|
||||
|
||||
Reference in new issue
Block a user