feat(catalog): add scripts/catalog-sync.ts CLI + interactive menu
CI / check (push) Successful in 32s
CI / tests-integration (push) Successful in 1m55s
CI / tests-unit (push) Successful in 2m1s
CI / tests-imaging-browsers (push) Successful in 2m7s
CI / tests-ui (push) Successful in 2m44s
CI / preflight (push) Skipped
CI / tests-quality (push) Successful in 1m46s
CI / deploy (push) Successful in 3m28s
CI / check (push) Successful in 32s
CI / tests-integration (push) Successful in 1m55s
CI / tests-unit (push) Successful in 2m1s
CI / tests-imaging-browsers (push) Successful in 2m7s
CI / tests-ui (push) Successful in 2m44s
CI / preflight (push) Skipped
CI / tests-quality (push) Successful in 1m46s
CI / deploy (push) Successful in 3m28s
Updating the catalog was only possible from the admin "Catalog audit" page, which streams `runCatalogAudit` over SSE and always compares against every enabled clone source. This adds the same engine to the terminal so it can be run by hand or from cron: - `scripts/catalog-sync.ts` — CLI driver around `runCatalogAudit`. Prints the live progress, the per-source "on the source but not in this hotel" counts and a summary block; generated catalog SQL is saved to `storage/catalog-sync/*.sql` (gitignored). Supports `--list-sources`, `--sources=<a,b>`, `--icons`, `--nitros`, `--furnidata`, `--structure`, `--check-ids`, `--sql`, `--apply-sql`, `--organize-sql`, `--all`, `--json` and `--fail-on-errors`. - `scripts/catalog-sync.sh` — interactive menu (`pnpm catalog:sync`). Lists the configured sources (so the menu always matches `clone_sources`), lets Habbo (COM) be taken with Enter, and offers audit-only through full repair + applying the catalog SQL, each with a distinct confirmation. With no arguments it needs a terminal; with any flag it passes straight through to the CLI. - `runCatalogAudit` gains `sourceIds` to restrict which sites are compared; omitting it keeps the previous "compare against everything enabled" behaviour. An audit writes nothing, and `--apply-sql` is the only flag that changes the catalog. Verified against the live DB: 16370 items_base rows, 16479 catalog items, 2871 Habbo (COM) entries missing from this hotel.
This commit is contained in:
1 parent
61107848f8
commit
4b575d3a1d
7 files changed
+739
-1
No files matched your search
@@ -1161,6 +1161,7 @@ the headroom left for everything else on the machine; the effective ceiling is
|
||||
| `pnpm db:studio` | Drizzle Studio (dev) |
|
||||
| `pnpm db:introspect` | drizzle-kit introspect (draft) |
|
||||
| `pnpm db:bulk` | Batch-import >50 MB JSON via `scripts/bulk-import-json.ts` |
|
||||
| `pnpm catalog:sync` | Sync the catalog against a clone source (interactive) |
|
||||
| `pnpm db:up` / `pnpm db:down` | Start / stop the `mariadb-turbo` container |
|
||||
| `pnpm gamedata:compress` | Pre-compress large gamedata JSON to `.gz` (gzip_static) |
|
||||
| `pnpm analyze` | Build + report per-route bundle sizes |
|
||||
|
||||
@@ -29,6 +29,8 @@
|
||||
"db:schema:generate": "node scripts/generate-drizzle-schema.mjs",
|
||||
"db:migrate": "tsx scripts/apply-migrations.ts",
|
||||
"db:migrate:status": "tsx scripts/apply-migrations.ts --status",
|
||||
"catalog:sync": "bash scripts/catalog-sync.sh",
|
||||
"catalog:sync:cli": "node --conditions=react-server --import tsx scripts/catalog-sync.ts",
|
||||
"db:studio": "drizzle-kit studio",
|
||||
"gamedata:compress": "node scripts/compress-gamedata.mjs",
|
||||
"hk:matrix:check": "tsx scripts/verify-housekeeping-matrix.ts",
|
||||
|
||||
@@ -25,6 +25,7 @@ This directory contains utility scripts for managing the EpicNext-Cms project.
|
||||
| `doctor.sh` | Runs a system diagnostic report. |
|
||||
| `setup-cron.sh` | Configures automated cron jobs. |
|
||||
| `alert.sh` | Sends notifications to a configured webhook. |
|
||||
| `catalog-sync.sh` | Interactive catalog update against a clone source (see below). |
|
||||
|
||||
## Usage
|
||||
Run any script directly, or use the dashboard:
|
||||
@@ -46,3 +47,32 @@ It manages the `pnpm` and `Node.js` rows of the System Requirements table and
|
||||
the "Core stack" line. The CI `check` job runs the `--check` variant, so a
|
||||
version change cannot reach `main` with a stale README — the pipeline fails
|
||||
until you run `pnpm readme:versions`.
|
||||
|
||||
### Updating the catalog (`catalog-sync.sh`)
|
||||
|
||||
Compares this hotel's catalog against a clone source (Habbo and the retros) and
|
||||
fixes what is broken: missing furniture icons, missing `.nitro` bundles,
|
||||
`FurnitureData.json` drift, orphaned/duplicate catalog rows, and items that
|
||||
exist in `items_base` but have no catalog offer.
|
||||
|
||||
```bash
|
||||
pnpm catalog:sync # interactive menu
|
||||
./scripts/catalog-sync.sh --list-sources # which sites can be compared
|
||||
./scripts/catalog-sync.sh --all --sources=default-habbo-com
|
||||
./scripts/catalog-sync.sh --all --sql --apply-sql # catalogs changes are written
|
||||
```
|
||||
|
||||
The menu asks which sources to compare (Habbo (COM) is the preset) and what may
|
||||
be written. With no arguments it needs a terminal; with any flag it skips the
|
||||
menu, so it is usable from cron.
|
||||
|
||||
Safety notes:
|
||||
|
||||
- **An audit writes nothing** — no repair flag means read-only.
|
||||
- `--apply-sql` is the only flag that changes the catalog, and the generated SQL
|
||||
is always written to `storage/catalog-sync/*.sql` (gitignored) so it can be
|
||||
reviewed or replayed by hand.
|
||||
- The engine is the same one the admin *Catalog audit* page streams
|
||||
(`runCatalogAudit`), so a terminal run and an in-admin run agree.
|
||||
- `--json` prints one machine-readable report; `--fail-on-errors` exits `3` when
|
||||
error-severity issues remain, which is what a scheduled check should use.
|
||||
Executable
+157
@@ -0,0 +1,157 @@
|
||||
#!/usr/bin/env bash
|
||||
# Update the hotel catalog from a clone source (Habbo, retros, …).
|
||||
#
|
||||
# Two ways to use it:
|
||||
# ./scripts/catalog-sync.sh interactive menu
|
||||
# ./scripts/catalog-sync.sh --all --sql same flags as the CLI, no menu
|
||||
#
|
||||
# The menu asks which sites to compare (Habbo is the preset) and what may be
|
||||
# written. An audit is read-only; only a repair option touches the catalog, and
|
||||
# applying the generated catalog SQL always needs its own choice.
|
||||
|
||||
cd "$(dirname "$0")/.." || exit 1
|
||||
CLI="scripts/catalog-sync.ts"
|
||||
# The react-server export condition resolves `server-only` to its no-op build,
|
||||
# exactly like `pnpm jobs:worker` — the audit engine imports server modules.
|
||||
RUN=(node --conditions=react-server --import tsx "$CLI")
|
||||
|
||||
# Any flag passed straight through means "no menu", so the same entry point
|
||||
# works for cron/CI where only the flag path is safe.
|
||||
if [ $# -gt 0 ]; then
|
||||
exec "${RUN[@]}" "$@"
|
||||
fi
|
||||
|
||||
if [ ! -t 0 ]; then
|
||||
echo "Not a terminal. Use the non-interactive form, e.g.:" >&2
|
||||
echo " $0 --all --sources=default-habbo-com" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=== Catalog sync ==="
|
||||
echo
|
||||
|
||||
# ── Step 1: which sites to compare ───────────────────────────────────────────
|
||||
# Ask the CLI for the configured sources so the menu always matches the live
|
||||
# `clone_sources` setting (and its defaults) instead of a hardcoded list.
|
||||
mapfile -t SOURCE_ROWS < <("${RUN[@]}" --list-sources --tsv)
|
||||
if [ "${#SOURCE_ROWS[@]}" -eq 0 ]; then
|
||||
echo "No clone sources are configured." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IDS=()
|
||||
NAMES=()
|
||||
echo "Compare this catalog against:"
|
||||
for i in "${!SOURCE_ROWS[@]}"; do
|
||||
IFS=$'\t' read -r SOURCE_ID SOURCE_NAME _SOURCE_URL <<<"${SOURCE_ROWS[$i]}"
|
||||
IDS+=("$SOURCE_ID")
|
||||
NAMES+=("$SOURCE_NAME")
|
||||
printf " %2d) %-24s %s\n" "$((i + 1))" "$SOURCE_NAME" "$SOURCE_ID"
|
||||
done
|
||||
echo
|
||||
echo " a) all sources above"
|
||||
echo " Enter) preset: ${NAMES[0]}"
|
||||
|
||||
read -r -p "Sources [a, 1-${#IDS[@]}, Enter]: " SOURCE_CHOICE
|
||||
|
||||
SELECTED_FLAG=""
|
||||
case "$SOURCE_CHOICE" in
|
||||
"" | "Habbo" | "habbo" | "habbo.com")
|
||||
# Habbo is the first default preset (Habbo (COM)).
|
||||
SELECTED_FLAG="--sources=${IDS[0]}"
|
||||
SOURCE_LABEL="${NAMES[0]}"
|
||||
;;
|
||||
a | A | all)
|
||||
SOURCE_LABEL="all sources"
|
||||
;;
|
||||
*)
|
||||
SELECTED_IDS=()
|
||||
IFS=', ' read -r -a CHOICES <<<"$SOURCE_CHOICE"
|
||||
for CHOICE in "${CHOICES[@]}"; do
|
||||
[ -z "$CHOICE" ] && continue
|
||||
case "$CHOICE" in
|
||||
*[!0-9]* | "")
|
||||
echo "Invalid choice: '$CHOICE'" >&2
|
||||
exit 1
|
||||
;;
|
||||
*)
|
||||
if [ "$CHOICE" -lt 1 ] || [ "$CHOICE" -gt "${#IDS[@]}" ]; then
|
||||
echo "Out of range: $CHOICE" >&2
|
||||
exit 1
|
||||
fi
|
||||
SELECTED_IDS+=("${IDS[$((CHOICE - 1))]}")
|
||||
;;
|
||||
esac
|
||||
done
|
||||
if [ "${#SELECTED_IDS[@]}" -eq 0 ]; then
|
||||
echo "No sources selected." >&2
|
||||
exit 1
|
||||
fi
|
||||
SELECTED_FLAG="--sources=$(IFS=,; echo "${SELECTED_IDS[*]}")"
|
||||
SOURCE_LABEL="${#SELECTED_IDS[@]} source(s)"
|
||||
;;
|
||||
esac
|
||||
|
||||
# ── Step 2: what may be changed ──────────────────────────────────────────────
|
||||
cat <<EOF
|
||||
|
||||
What should be done? (comparing with: $SOURCE_LABEL)
|
||||
1) Audit only — nothing is written
|
||||
2) Repair furniture icons + nitro bundles
|
||||
3) 2 + FurnitureData.json + catalog structure
|
||||
4) 3 + generate the catalog SQL (saved, not applied)
|
||||
5) 4 + APPLY the catalog SQL to the catalog
|
||||
q) Cancel
|
||||
EOF
|
||||
|
||||
read -r -p "Action [1]: " ACTION_CHOICE
|
||||
ACTION_CHOICE="${ACTION_CHOICE:-1}"
|
||||
|
||||
ACTIONS=(audit)
|
||||
CONFIRM=""
|
||||
# Empty for the audit-only branch on purpose: no repair flags are passed.
|
||||
ACTION_FLAGS=()
|
||||
case "$ACTION_CHOICE" in
|
||||
1)
|
||||
ACTIONS=(audit "[nothing written]")
|
||||
;;
|
||||
2)
|
||||
ACTIONS=(audit "icons + nitro bundles downloaded")
|
||||
ACTION_FLAGS=(--icons --nitros)
|
||||
;;
|
||||
3)
|
||||
ACTIONS=(audit "icons + nitro bundles + FurnitureData.json + structure")
|
||||
ACTION_FLAGS=(--all)
|
||||
;;
|
||||
4)
|
||||
ACTIONS=(audit "everything + catalog SQL generated")
|
||||
ACTION_FLAGS=(--all --sql)
|
||||
;;
|
||||
5)
|
||||
ACTIONS=(audit "everything + catalog SQL APPLIED")
|
||||
ACTION_FLAGS=(--all --sql --apply-sql)
|
||||
CONFIRM="This writes to the catalog. Continue? [y/N]: "
|
||||
;;
|
||||
q | Q)
|
||||
echo "Cancelled."
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
echo "Invalid action: '$ACTION_CHOICE'" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo
|
||||
printf 'Running: %s against %s\n' "${ACTIONS[1]}" "$SOURCE_LABEL"
|
||||
if [ -n "$CONFIRM" ]; then
|
||||
read -r -p "$CONFIRM" REPLY
|
||||
if [ "$REPLY" != "y" ] && [ "$REPLY" != "Y" ]; then
|
||||
echo "Cancelled."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
echo
|
||||
# shellcheck disable=SC2086
|
||||
"${RUN[@]}" $SELECTED_FLAG "${ACTION_FLAGS[@]}"
|
||||
@@ -0,0 +1,114 @@
|
||||
import { mkdtempSync, readFileSync, rmSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
DEFAULT_CATALOG_SYNC_FLAGS,
|
||||
describeActions,
|
||||
parseCatalogSyncArgs,
|
||||
writeGeneratedSql,
|
||||
} from "./catalog-sync";
|
||||
|
||||
describe("catalog-sync argument parsing", () => {
|
||||
it("defaults to a read-only audit with no filters", () => {
|
||||
expect(parseCatalogSyncArgs([])).toEqual({
|
||||
...DEFAULT_CATALOG_SYNC_FLAGS,
|
||||
sourceIds: [],
|
||||
});
|
||||
expect(describeActions(parseCatalogSyncArgs([]))).toBe("audit");
|
||||
});
|
||||
|
||||
it("accepts comma-separated source ids", () => {
|
||||
const flags = parseCatalogSyncArgs([
|
||||
"--sources=default-habbo-com,default-wibbo",
|
||||
]);
|
||||
expect(flags.sourceIds).toEqual(["default-habbo-com", "default-wibbo"]);
|
||||
});
|
||||
|
||||
it("accepts display names, including ones with spaces", () => {
|
||||
const flags = parseCatalogSyncArgs(["--sources=Habbo (COM), Wibbo"]);
|
||||
expect(flags.sourceIds).toEqual(["Habbo (COM)", "Wibbo"]);
|
||||
});
|
||||
|
||||
it("trims and drops empty source entries", () => {
|
||||
expect(parseCatalogSyncArgs(["--sources= a , ,b,"]).sourceIds).toEqual([
|
||||
"a",
|
||||
"b",
|
||||
]);
|
||||
});
|
||||
|
||||
it("turns --all into every repair step", () => {
|
||||
const flags = parseCatalogSyncArgs(["--all"]);
|
||||
expect(flags.repairIcons).toBe(true);
|
||||
expect(flags.repairNitros).toBe(true);
|
||||
expect(flags.repairFurniData).toBe(true);
|
||||
expect(flags.repairStructure).toBe(true);
|
||||
expect(flags.checkFurniDataIds).toBe(true);
|
||||
// Generating/applying SQL is never implied by a repair.
|
||||
expect(flags.generateSql).toBe(false);
|
||||
expect(flags.applySql).toBe(false);
|
||||
});
|
||||
|
||||
it("treats applying SQL as implying generation", () => {
|
||||
expect(parseCatalogSyncArgs(["--apply-sql"]).generateSql).toBe(true);
|
||||
expect(parseCatalogSyncArgs(["--organize-sql"]).generateSql).toBe(true);
|
||||
expect(describeActions(parseCatalogSyncArgs(["--apply-sql"]))).toBe(
|
||||
"audit + apply-sql",
|
||||
);
|
||||
});
|
||||
|
||||
it("accepts the alias spellings of the repair flags", () => {
|
||||
const flags = parseCatalogSyncArgs([
|
||||
"--repair-icons",
|
||||
"--repair-nitros",
|
||||
"--repair-furnidata",
|
||||
"--repair-structure",
|
||||
]);
|
||||
expect(flags.repairIcons).toBe(true);
|
||||
expect(flags.repairNitros).toBe(true);
|
||||
expect(flags.repairFurniData).toBe(true);
|
||||
expect(flags.repairStructure).toBe(true);
|
||||
});
|
||||
|
||||
it("ignores a bare -- separator", () => {
|
||||
expect(parseCatalogSyncArgs(["--", "--icons"]).repairIcons).toBe(true);
|
||||
});
|
||||
|
||||
it("rejects an unknown flag", () => {
|
||||
expect(() => parseCatalogSyncArgs(["--nope"])).toThrow(/Unknown flag/);
|
||||
});
|
||||
|
||||
it("rejects --sources without a value", () => {
|
||||
expect(() => parseCatalogSyncArgs(["--sources"])).toThrow(/needs a value/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("generated catalog SQL output", () => {
|
||||
let cwd = "";
|
||||
|
||||
beforeEach(() => {
|
||||
cwd = process.cwd();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
process.chdir(cwd);
|
||||
});
|
||||
|
||||
it("writes the SQL under storage/catalog-sync with a trailing newline", async () => {
|
||||
let target = "";
|
||||
try {
|
||||
const dir = mkdtempSync(join(tmpdir(), "catalog-sync-sql-"));
|
||||
target = dir;
|
||||
process.chdir(dir);
|
||||
const file = await writeGeneratedSql("INSERT INTO `catalog_items` (1);");
|
||||
expect(file.startsWith("storage/catalog-sync/")).toBe(true);
|
||||
expect(file.endsWith(".sql")).toBe(true);
|
||||
expect(readFileSync(join(dir, file), "utf8")).toBe(
|
||||
"INSERT INTO `catalog_items` (1);\n",
|
||||
);
|
||||
} finally {
|
||||
if (target) rmSync(target, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,423 @@
|
||||
// Command-line driver for the catalog audit/repair pipeline.
|
||||
//
|
||||
// The same engine the admin "Catalog audit" UI streams from (`runCatalogAudit`),
|
||||
// for the terminal: audit the hotel catalog against one or more clone sources
|
||||
// (Habbo and the retros), fix the furniture that is broken, and bring missing
|
||||
// items into the catalog — optionally writing and applying the catalog SQL.
|
||||
//
|
||||
// Usage:
|
||||
// pnpm catalog:sync # interactive menu (scripts/catalog-sync.sh)
|
||||
// pnpm exec tsx scripts/catalog-sync.ts --list-sources
|
||||
// pnpm exec tsx scripts/catalog-sync.ts --sources="Habbo (COM)" # audit only
|
||||
// pnpm exec tsx scripts/catalog-sync.ts --all --sources=default-habbo-com
|
||||
// pnpm exec tsx scripts/catalog-sync.ts --all --sql --apply-sql # + write/apply SQL
|
||||
//
|
||||
// Flags:
|
||||
// --list-sources print the configured clone sources and exit
|
||||
// --tsv with the above, print `id\tname\tfurnidataUrl` rows
|
||||
// --sources=<a,b> compare against these sources (id or name, comma-separated)
|
||||
// --icons re-download missing furniture icons from the sources
|
||||
// --nitros re-download missing .nitro / swf bundles
|
||||
// --furnidata repair FurnitureData.json (add missing, drop duplicates)
|
||||
// --structure repair orphaned / duplicate catalog rows
|
||||
// --check-ids check FurnitureData.json for spriteId conflicts
|
||||
// --sql generate the catalog SQL for items missing an offer
|
||||
// --apply-sql apply the generated SQL (implies --sql)
|
||||
// --organize-sql order the generated SQL by catalog page (implies --sql)
|
||||
// --all --icons --nitros --furnidata --structure --check-ids
|
||||
// --json print one machine-readable JSON report
|
||||
// --fail-on-errors exit 3 when error-severity issues remain
|
||||
//
|
||||
// Audit is read-only by default: nothing is written until one of the repair
|
||||
// flags is passed, and `--apply-sql` is the only flag that changes the catalog.
|
||||
|
||||
import "./load-env";
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import { join } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import type { AuditEvent, AuditSummary } from "@/lib/services/catalog-audit";
|
||||
|
||||
/** Where generated catalog SQL is written (`storage/` is gitignored). */
|
||||
function sqlOutputDir(): string {
|
||||
return join(process.cwd(), "storage", "catalog-sync");
|
||||
}
|
||||
|
||||
export type SourceRef = { id: string; name: string; furnidataUrl: string };
|
||||
|
||||
export type AuditIssue = NonNullable<AuditEvent["issues"]>[number];
|
||||
export type MissingFromSource = NonNullable<
|
||||
AuditEvent["missingFromSources"]
|
||||
>[number];
|
||||
|
||||
export interface AuditReport {
|
||||
sources: SourceRef[];
|
||||
summary?: AuditSummary;
|
||||
issues: AuditIssue[];
|
||||
missingFromSources: MissingFromSource[];
|
||||
sqlFile?: string;
|
||||
}
|
||||
|
||||
export interface CatalogSyncFlags {
|
||||
listSources: boolean;
|
||||
tsv: boolean;
|
||||
sourceIds: string[];
|
||||
repairIcons: boolean;
|
||||
repairNitros: boolean;
|
||||
repairFurniData: boolean;
|
||||
repairStructure: boolean;
|
||||
checkFurniDataIds: boolean;
|
||||
generateSql: boolean;
|
||||
applySql: boolean;
|
||||
organizeSql: boolean;
|
||||
json: boolean;
|
||||
failOnErrors: boolean;
|
||||
}
|
||||
|
||||
export const DEFAULT_CATALOG_SYNC_FLAGS: Omit<CatalogSyncFlags, "sourceIds"> = {
|
||||
listSources: false,
|
||||
tsv: false,
|
||||
repairIcons: false,
|
||||
repairNitros: false,
|
||||
repairFurniData: false,
|
||||
repairStructure: false,
|
||||
checkFurniDataIds: false,
|
||||
generateSql: false,
|
||||
applySql: false,
|
||||
organizeSql: false,
|
||||
json: false,
|
||||
failOnErrors: false,
|
||||
};
|
||||
|
||||
const BOOLEAN_FLAGS: Record<string, keyof CatalogSyncFlags> = {
|
||||
"--list-sources": "listSources",
|
||||
"--tsv": "tsv",
|
||||
"--icons": "repairIcons",
|
||||
"--repair-icons": "repairIcons",
|
||||
"--nitros": "repairNitros",
|
||||
"--repair-nitros": "repairNitros",
|
||||
"--furnidata": "repairFurniData",
|
||||
"--repair-furnidata": "repairFurniData",
|
||||
"--structure": "repairStructure",
|
||||
"--repair-structure": "repairStructure",
|
||||
"--check-ids": "checkFurniDataIds",
|
||||
"--sql": "generateSql",
|
||||
"--apply-sql": "applySql",
|
||||
"--organize-sql": "organizeSql",
|
||||
"--json": "json",
|
||||
"--fail-on-errors": "failOnErrors",
|
||||
};
|
||||
|
||||
export function parseCatalogSyncArgs(raw: string[]): CatalogSyncFlags {
|
||||
const flags: CatalogSyncFlags = {
|
||||
...DEFAULT_CATALOG_SYNC_FLAGS,
|
||||
sourceIds: [],
|
||||
};
|
||||
|
||||
for (const arg of raw) {
|
||||
if (arg === "--") continue;
|
||||
|
||||
if (arg === "--all") {
|
||||
flags.repairIcons = true;
|
||||
flags.repairNitros = true;
|
||||
flags.repairFurniData = true;
|
||||
flags.repairStructure = true;
|
||||
flags.checkFurniDataIds = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
const separator = arg.indexOf("=");
|
||||
const name = separator >= 0 ? arg.slice(0, separator) : arg;
|
||||
|
||||
const booleanFlag = BOOLEAN_FLAGS[name];
|
||||
if (booleanFlag) {
|
||||
flags[booleanFlag] = true as never;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (name === "--sources" || name === "--source") {
|
||||
if (separator < 0)
|
||||
throw new Error(
|
||||
`${name} needs a value, e.g. --sources=default-habbo-com`,
|
||||
);
|
||||
const value = arg.slice(separator + 1);
|
||||
for (const part of value
|
||||
.split(",")
|
||||
.map((entry) => entry.trim())
|
||||
.filter(Boolean))
|
||||
flags.sourceIds.push(part);
|
||||
continue;
|
||||
}
|
||||
|
||||
throw new Error(
|
||||
`Unknown flag: ${arg} (run with --help for the supported flags)`,
|
||||
);
|
||||
}
|
||||
|
||||
// Applying SQL implies generating it, and organising it implies generating.
|
||||
if (flags.applySql) flags.generateSql = true;
|
||||
if (flags.organizeSql) flags.generateSql = true;
|
||||
return flags;
|
||||
}
|
||||
|
||||
/** Human-readable list of what the run will do. */
|
||||
export function describeActions(flags: CatalogSyncFlags): string {
|
||||
const actions = ["audit"];
|
||||
if (flags.repairIcons) actions.push("icons");
|
||||
if (flags.repairNitros) actions.push("nitros");
|
||||
if (flags.repairFurniData) actions.push("furnidata");
|
||||
if (flags.repairStructure) actions.push("structure");
|
||||
if (flags.checkFurniDataIds) actions.push("id-check");
|
||||
if (flags.applySql) actions.push("apply-sql");
|
||||
else if (flags.generateSql) actions.push("generate-sql");
|
||||
if (flags.organizeSql) actions.push("organised");
|
||||
return actions.join(" + ");
|
||||
}
|
||||
|
||||
/** True when the file is being executed, not imported by a test. */
|
||||
function isEntryPoint(): boolean {
|
||||
const entry = process.argv[1];
|
||||
return (
|
||||
Boolean(entry) && import.meta.url === pathToFileURL(String(entry)).href
|
||||
);
|
||||
}
|
||||
|
||||
async function loadSourceCatalogue(): Promise<SourceRef[]> {
|
||||
const { listSources } = await import("@/lib/services/clone-sources");
|
||||
const sources = await listSources();
|
||||
return sources
|
||||
.filter((source) => source.furnidataUrl)
|
||||
.map((source) => ({
|
||||
id: source.id,
|
||||
name: source.name,
|
||||
furnidataUrl: source.furnidataUrl,
|
||||
}));
|
||||
}
|
||||
|
||||
async function commandListSources(flags: CatalogSyncFlags): Promise<number> {
|
||||
const sources = await loadSourceCatalogue();
|
||||
|
||||
if (flags.tsv) {
|
||||
for (const source of sources)
|
||||
console.log([source.id, source.name, source.furnidataUrl].join("\t"));
|
||||
return 0;
|
||||
}
|
||||
|
||||
console.log(`Configured clone sources (${sources.length}):`);
|
||||
for (const source of sources)
|
||||
console.log(` ${source.id}\t${source.name}\t${source.furnidataUrl}`);
|
||||
console.log(
|
||||
"\nCompare with: --sources=<id> (comma-separated; display names work too)",
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
|
||||
/** Persist the generated SQL so it can be reviewed or replayed by hand. */
|
||||
export async function writeGeneratedSql(sql: string): Promise<string> {
|
||||
await mkdir(sqlOutputDir(), { recursive: true });
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, "-");
|
||||
const file = join(sqlOutputDir(), `catalog-${stamp}.sql`);
|
||||
await writeFile(file, sql.endsWith("\n") ? sql : `${sql}\n`, "utf8");
|
||||
return file.replace(`${process.cwd()}/`, "");
|
||||
}
|
||||
|
||||
export async function commandRun(flags: CatalogSyncFlags): Promise<number> {
|
||||
const catalogue = await loadSourceCatalogue();
|
||||
|
||||
// Resolve the selection up-front so a typo fails before anything is written.
|
||||
const known = new Set<string>();
|
||||
for (const source of catalogue) {
|
||||
known.add(source.id);
|
||||
known.add(source.name);
|
||||
}
|
||||
const unknown = flags.sourceIds.filter((id) => !known.has(id));
|
||||
if (unknown.length > 0) {
|
||||
console.error(
|
||||
`Unknown source: ${unknown.join(", ")}\nRun --list-sources to see the configured sites.`,
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
|
||||
const selected = flags.sourceIds.length
|
||||
? catalogue.filter(
|
||||
(source) =>
|
||||
flags.sourceIds.includes(source.id) ||
|
||||
flags.sourceIds.includes(source.name),
|
||||
)
|
||||
: catalogue;
|
||||
|
||||
const { runCatalogAudit } = await import("@/lib/services/catalog-audit");
|
||||
|
||||
const report: AuditReport = {
|
||||
sources: selected,
|
||||
issues: [],
|
||||
missingFromSources: [],
|
||||
};
|
||||
const quiet = flags.json;
|
||||
const log = (line: string) => {
|
||||
if (!quiet) console.log(line);
|
||||
};
|
||||
const started = Date.now();
|
||||
let pendingSql: string | undefined;
|
||||
|
||||
log(
|
||||
`Catalog sync — ${selected.length} source(s), ${describeActions(flags)}` +
|
||||
(flags.applySql ? " [SQL WILL BE APPLIED]" : ""),
|
||||
);
|
||||
|
||||
await runCatalogAudit(
|
||||
(event) => {
|
||||
switch (event.type) {
|
||||
case "progress":
|
||||
log(`· ${event.message ?? ""}`);
|
||||
break;
|
||||
case "items_loaded":
|
||||
log(`· items_base: ${event.total} rows`);
|
||||
break;
|
||||
case "catalog_loaded":
|
||||
log(`· catalog_items: ${event.total} rows`);
|
||||
break;
|
||||
case "sql_ready":
|
||||
log(`· catalog SQL generated: ${event.sqlCount ?? 0} statement(s)`);
|
||||
pendingSql = event.sql;
|
||||
break;
|
||||
case "sql_applied":
|
||||
log(
|
||||
`· catalog SQL applied: ${event.applied ?? 0} (failed ${event.sqlFailed ?? 0})`,
|
||||
);
|
||||
break;
|
||||
case "furnidata_repair_complete":
|
||||
log(
|
||||
`· FurnitureData.json: +${event.furniData?.added ?? 0} entries, ${event.furniData?.removedDuplicates ?? 0} duplicate(s) removed`,
|
||||
);
|
||||
break;
|
||||
case "structure_repair_complete":
|
||||
log(
|
||||
`· structure: ${event.structure?.orphanedRemoved ?? 0} orphaned removed, ${event.structure?.duplicatesMerged ?? 0} duplicate(s) merged, ${event.structure?.remappedReferences ?? 0} reference(s) remapped`,
|
||||
);
|
||||
break;
|
||||
case "repair_complete":
|
||||
log(
|
||||
`· asset repair: ${event.repair?.succeeded ?? 0} ok, ${event.repair?.failed ?? 0} failed, ${event.repair?.skipped ?? 0} skipped`,
|
||||
);
|
||||
break;
|
||||
case "batch_complete":
|
||||
report.issues = event.issues ?? [];
|
||||
report.missingFromSources = event.missingFromSources ?? [];
|
||||
report.summary = event.summary;
|
||||
break;
|
||||
case "error":
|
||||
console.error(`! ${event.message ?? "audit error"}`);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
},
|
||||
{
|
||||
sourceIds: flags.sourceIds,
|
||||
repair: flags.repairIcons,
|
||||
repairNitros: flags.repairNitros,
|
||||
sql: flags.generateSql,
|
||||
applySql: flags.applySql,
|
||||
organizeSql: flags.organizeSql,
|
||||
repairFurniData: flags.repairFurniData,
|
||||
repairStructure: flags.repairStructure,
|
||||
checkFurniDataIds: flags.checkFurniDataIds,
|
||||
},
|
||||
);
|
||||
|
||||
if (pendingSql?.trim()) report.sqlFile = await writeGeneratedSql(pendingSql);
|
||||
|
||||
if (flags.json) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{ flags, elapsedMs: Date.now() - started, ...report },
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
printSummary(report, started);
|
||||
if (report.sqlFile) console.log(`Catalog SQL: ${report.sqlFile}`);
|
||||
}
|
||||
|
||||
const errors = report.issues.filter((issue) => issue.severity === "error");
|
||||
if (flags.failOnErrors && errors.length > 0) {
|
||||
if (!quiet) console.error(`${errors.length} error(s) remain unresolved.`);
|
||||
return 3;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
function printSummary(report: AuditReport, started: number): void {
|
||||
const summary = report.summary;
|
||||
const seconds = ((Date.now() - started) / 1000).toFixed(1);
|
||||
|
||||
console.log(`\nDone in ${seconds}s`);
|
||||
if (!summary) {
|
||||
console.log("No summary was produced (the audit stopped early).");
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(` items_base ${summary.totalItemsBase}`);
|
||||
console.log(` catalog_items ${summary.totalCatalogItems}`);
|
||||
console.log(` missing catalog entry ${summary.missingCatalog}`);
|
||||
console.log(` orphaned catalog rows ${summary.orphanedCatalog}`);
|
||||
console.log(` duplicate classnames ${summary.duplicates}`);
|
||||
console.log(
|
||||
` missing icons ${summary.missingIcon} (repaired ${summary.iconsRepaired}, still missing ${summary.iconsStillMissing})`,
|
||||
);
|
||||
console.log(
|
||||
` missing nitro bundles ${summary.missingNitro} (repaired ${summary.nitrosRepaired}, still missing ${summary.nitrosStillMissing})`,
|
||||
);
|
||||
console.log(
|
||||
` catalog SQL ${summary.catalogSqlCount} stmt, applied ${summary.catalogSqlApplied}`,
|
||||
);
|
||||
console.log(` furnidata added ${summary.furniDataAdded}`);
|
||||
console.log(
|
||||
` not restorable ${summary.unrepairableCount} (no icon/bundle on any compared source)`,
|
||||
);
|
||||
|
||||
if (report.missingFromSources.length > 0) {
|
||||
const bySource = new Map<string, number>();
|
||||
for (const entry of report.missingFromSources)
|
||||
bySource.set(entry.sourceName, (bySource.get(entry.sourceName) ?? 0) + 1);
|
||||
console.log(
|
||||
`\n on compared sources but not in this hotel: ${report.missingFromSources.length}`,
|
||||
);
|
||||
for (const [name, count] of bySource)
|
||||
console.log(` ${name.padEnd(24)} ${count}`);
|
||||
}
|
||||
|
||||
const errors = report.issues.filter((issue) => issue.severity === "error");
|
||||
if (errors.length > 0) {
|
||||
console.log(`\n unresolved errors: ${errors.length}`);
|
||||
for (const issue of errors.slice(0, 20))
|
||||
console.log(` ${issue.classname ?? issue.type}: ${issue.message}`);
|
||||
if (errors.length > 20) console.log(` … and ${errors.length - 20} more`);
|
||||
}
|
||||
}
|
||||
|
||||
async function main(): Promise<number> {
|
||||
const flags = parseCatalogSyncArgs(process.argv.slice(2));
|
||||
try {
|
||||
if (flags.listSources) return await commandListSources(flags);
|
||||
return await commandRun(flags);
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`Catalog sync failed: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (isEntryPoint()) {
|
||||
main().then(
|
||||
(code) => {
|
||||
process.exit(code);
|
||||
},
|
||||
(error) => {
|
||||
console.error(error);
|
||||
process.exit(1);
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -141,6 +141,11 @@ export interface CatalogAuditOptions {
|
||||
repairFurniData?: boolean;
|
||||
repairStructure?: boolean;
|
||||
checkFurniDataIds?: boolean;
|
||||
/**
|
||||
* Restrict the clone-source comparison to these sources (by id or name).
|
||||
* Empty/omitted keeps every enabled source, which is the default.
|
||||
*/
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
export async function runCatalogAudit(
|
||||
@@ -838,7 +843,13 @@ export async function runCatalogAudit(
|
||||
|
||||
onEvent?.({ type: "progress", message: "Comparing with clone sources…" });
|
||||
|
||||
const sources = await listSources();
|
||||
const allSources = await listSources();
|
||||
// An explicit selection compares against just those sites; leaving it empty
|
||||
// keeps the historical "compare against everything enabled" behaviour.
|
||||
const wantedIds = new Set(options?.sourceIds ?? []);
|
||||
const sources = wantedIds.size
|
||||
? allSources.filter((s) => wantedIds.has(s.id) || wantedIds.has(s.name))
|
||||
: allSources;
|
||||
const sourceResults = await Promise.all(
|
||||
sources
|
||||
.filter((s) => s.furnidataUrl)
|
||||
|
||||
Reference in new issue
Block a user