feat(catalog): add scripts/catalog-sync.ts CLI + interactive menu
CI / check (push) Successful in 32s
CI / tests-integration (push) Successful in 1m55s
CI / tests-unit (push) Successful in 2m1s
CI / tests-imaging-browsers (push) Successful in 2m7s
CI / tests-ui (push) Successful in 2m44s
CI / preflight (push) Skipped
CI / tests-quality (push) Successful in 1m46s
CI / deploy (push) Successful in 3m28s
CI / check (push) Successful in 32s
CI / tests-integration (push) Successful in 1m55s
CI / tests-unit (push) Successful in 2m1s
CI / tests-imaging-browsers (push) Successful in 2m7s
CI / tests-ui (push) Successful in 2m44s
CI / preflight (push) Skipped
CI / tests-quality (push) Successful in 1m46s
CI / deploy (push) Successful in 3m28s
Updating the catalog was only possible from the admin "Catalog audit" page, which streams `runCatalogAudit` over SSE and always compares against every enabled clone source. This adds the same engine to the terminal so it can be run by hand or from cron: - `scripts/catalog-sync.ts` — CLI driver around `runCatalogAudit`. Prints the live progress, the per-source "on the source but not in this hotel" counts and a summary block; generated catalog SQL is saved to `storage/catalog-sync/*.sql` (gitignored). Supports `--list-sources`, `--sources=<a,b>`, `--icons`, `--nitros`, `--furnidata`, `--structure`, `--check-ids`, `--sql`, `--apply-sql`, `--organize-sql`, `--all`, `--json` and `--fail-on-errors`. - `scripts/catalog-sync.sh` — interactive menu (`pnpm catalog:sync`). Lists the configured sources (so the menu always matches `clone_sources`), lets Habbo (COM) be taken with Enter, and offers audit-only through full repair + applying the catalog SQL, each with a distinct confirmation. With no arguments it needs a terminal; with any flag it passes straight through to the CLI. - `runCatalogAudit` gains `sourceIds` to restrict which sites are compared; omitting it keeps the previous "compare against everything enabled" behaviour. An audit writes nothing, and `--apply-sql` is the only flag that changes the catalog. Verified against the live DB: 16370 items_base rows, 16479 catalog items, 2871 Habbo (COM) entries missing from this hotel.
This commit is contained in:
1 parent
61107848f8
commit
4b575d3a1d
7 files changed
+739
-1
No files matched your search
@@ -1161,6 +1161,7 @@ the headroom left for everything else on the machine; the effective ceiling is
|
|||||||
| `pnpm db:studio` | Drizzle Studio (dev) |
|
| `pnpm db:studio` | Drizzle Studio (dev) |
|
||||||
| `pnpm db:introspect` | drizzle-kit introspect (draft) |
|
| `pnpm db:introspect` | drizzle-kit introspect (draft) |
|
||||||
| `pnpm db:bulk` | Batch-import >50 MB JSON via `scripts/bulk-import-json.ts` |
|
| `pnpm db:bulk` | Batch-import >50 MB JSON via `scripts/bulk-import-json.ts` |
|
||||||
|
| `pnpm catalog:sync` | Sync the catalog against a clone source (interactive) |
|
||||||
| `pnpm db:up` / `pnpm db:down` | Start / stop the `mariadb-turbo` container |
|
| `pnpm db:up` / `pnpm db:down` | Start / stop the `mariadb-turbo` container |
|
||||||
| `pnpm gamedata:compress` | Pre-compress large gamedata JSON to `.gz` (gzip_static) |
|
| `pnpm gamedata:compress` | Pre-compress large gamedata JSON to `.gz` (gzip_static) |
|
||||||
| `pnpm analyze` | Build + report per-route bundle sizes |
|
| `pnpm analyze` | Build + report per-route bundle sizes |
|
||||||
|
|||||||
@@ -29,6 +29,8 @@
|
|||||||
"db:schema:generate": "node scripts/generate-drizzle-schema.mjs",
|
"db:schema:generate": "node scripts/generate-drizzle-schema.mjs",
|
||||||
"db:migrate": "tsx scripts/apply-migrations.ts",
|
"db:migrate": "tsx scripts/apply-migrations.ts",
|
||||||
"db:migrate:status": "tsx scripts/apply-migrations.ts --status",
|
"db:migrate:status": "tsx scripts/apply-migrations.ts --status",
|
||||||
|
"catalog:sync": "bash scripts/catalog-sync.sh",
|
||||||
|
"catalog:sync:cli": "node --conditions=react-server --import tsx scripts/catalog-sync.ts",
|
||||||
"db:studio": "drizzle-kit studio",
|
"db:studio": "drizzle-kit studio",
|
||||||
"gamedata:compress": "node scripts/compress-gamedata.mjs",
|
"gamedata:compress": "node scripts/compress-gamedata.mjs",
|
||||||
"hk:matrix:check": "tsx scripts/verify-housekeeping-matrix.ts",
|
"hk:matrix:check": "tsx scripts/verify-housekeeping-matrix.ts",
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ This directory contains utility scripts for managing the EpicNext-Cms project.
|
|||||||
| `doctor.sh` | Runs a system diagnostic report. |
|
| `doctor.sh` | Runs a system diagnostic report. |
|
||||||
| `setup-cron.sh` | Configures automated cron jobs. |
|
| `setup-cron.sh` | Configures automated cron jobs. |
|
||||||
| `alert.sh` | Sends notifications to a configured webhook. |
|
| `alert.sh` | Sends notifications to a configured webhook. |
|
||||||
|
| `catalog-sync.sh` | Interactive catalog update against a clone source (see below). |
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
Run any script directly, or use the dashboard:
|
Run any script directly, or use the dashboard:
|
||||||
@@ -46,3 +47,32 @@ It manages the `pnpm` and `Node.js` rows of the System Requirements table and
|
|||||||
the "Core stack" line. The CI `check` job runs the `--check` variant, so a
|
the "Core stack" line. The CI `check` job runs the `--check` variant, so a
|
||||||
version change cannot reach `main` with a stale README — the pipeline fails
|
version change cannot reach `main` with a stale README — the pipeline fails
|
||||||
until you run `pnpm readme:versions`.
|
until you run `pnpm readme:versions`.
|
||||||
|
|
||||||
|
### Updating the catalog (`catalog-sync.sh`)
|
||||||
|
|
||||||
|
Compares this hotel's catalog against a clone source (Habbo and the retros) and
|
||||||
|
fixes what is broken: missing furniture icons, missing `.nitro` bundles,
|
||||||
|
`FurnitureData.json` drift, orphaned/duplicate catalog rows, and items that
|
||||||
|
exist in `items_base` but have no catalog offer.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pnpm catalog:sync # interactive menu
|
||||||
|
./scripts/catalog-sync.sh --list-sources # which sites can be compared
|
||||||
|
./scripts/catalog-sync.sh --all --sources=default-habbo-com
|
||||||
|
./scripts/catalog-sync.sh --all --sql --apply-sql # catalogs changes are written
|
||||||
|
```
|
||||||
|
|
||||||
|
The menu asks which sources to compare (Habbo (COM) is the preset) and what may
|
||||||
|
be written. With no arguments it needs a terminal; with any flag it skips the
|
||||||
|
menu, so it is usable from cron.
|
||||||
|
|
||||||
|
Safety notes:
|
||||||
|
|
||||||
|
- **An audit writes nothing** — no repair flag means read-only.
|
||||||
|
- `--apply-sql` is the only flag that changes the catalog, and the generated SQL
|
||||||
|
is always written to `storage/catalog-sync/*.sql` (gitignored) so it can be
|
||||||
|
reviewed or replayed by hand.
|
||||||
|
- The engine is the same one the admin *Catalog audit* page streams
|
||||||
|
(`runCatalogAudit`), so a terminal run and an in-admin run agree.
|
||||||
|
- `--json` prints one machine-readable report; `--fail-on-errors` exits `3` when
|
||||||
|
error-severity issues remain, which is what a scheduled check should use.
|
||||||
Executable
+157
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -141,6 +141,11 @@ export interface CatalogAuditOptions {
|
|||||||
repairFurniData?: boolean;
|
repairFurniData?: boolean;
|
||||||
repairStructure?: boolean;
|
repairStructure?: boolean;
|
||||||
checkFurniDataIds?: boolean;
|
checkFurniDataIds?: boolean;
|
||||||
|
/**
|
||||||
|
* Restrict the clone-source comparison to these sources (by id or name).
|
||||||
|
* Empty/omitted keeps every enabled source, which is the default.
|
||||||
|
*/
|
||||||
|
sourceIds?: string[];
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function runCatalogAudit(
|
export async function runCatalogAudit(
|
||||||
@@ -838,7 +843,13 @@ export async function runCatalogAudit(
|
|||||||
|
|
||||||
onEvent?.({ type: "progress", message: "Comparing with clone sources…" });
|
onEvent?.({ type: "progress", message: "Comparing with clone sources…" });
|
||||||
|
|
||||||
const sources = await listSources();
|
const allSources = await listSources();
|
||||||
|
// An explicit selection compares against just those sites; leaving it empty
|
||||||
|
// keeps the historical "compare against everything enabled" behaviour.
|
||||||
|
const wantedIds = new Set(options?.sourceIds ?? []);
|
||||||
|
const sources = wantedIds.size
|
||||||
|
? allSources.filter((s) => wantedIds.has(s.id) || wantedIds.has(s.name))
|
||||||
|
: allSources;
|
||||||
const sourceResults = await Promise.all(
|
const sourceResults = await Promise.all(
|
||||||
sources
|
sources
|
||||||
.filter((s) => s.furnidataUrl)
|
.filter((s) => s.furnidataUrl)
|
||||||
|
|||||||
Reference in new issue
Block a user