feat(catalog-sync): add import, reconcile, sprite-id and offer steps
CI / check (push) Successful in 37s
CI / tests-unit (push) Successful in 2m3s
CI / tests-ui (push) Successful in 2m52s
CI / tests-integration (push) Successful in 2m54s
CI / preflight (push) Skipped
CI / tests-imaging-browsers (push) Failing after 3m23s
CI / deploy (push) Successful in 3m3s
CI / tests-quality (push) Successful in 6m8s
CI / check (push) Successful in 37s
CI / tests-unit (push) Successful in 2m3s
CI / tests-ui (push) Successful in 2m52s
CI / tests-integration (push) Successful in 2m54s
CI / preflight (push) Skipped
CI / tests-imaging-browsers (push) Failing after 3m23s
CI / deploy (push) Successful in 3m3s
CI / tests-quality (push) Successful in 6m8s
The CLI could audit and repair what already existed, but adding furniture present at a source was still only possible from the admin UI item by item. The repo already ships every piece of that pipeline, so wire them together as further steps of the same run: - `--import-new` imports furniture the compared source has and this hotel lacks. It reuses `getClonableClassnames` for the diff and `cloneSingleFurni` per item, which downloads the icon and the bundle (sources serve `.nitro`, stored as `.hab`), appends the furnidata entry with `deferFurniData` and creates the catalog page plus the offer itself. Deferred entries are written in one `appendFurniEntriesBulk` call, so 26000 items cost one JSON write, not one per item. `--only`, `--limit` and `--concurrency` narrow/tune it, failures are isolated per item and never abort the run. - `--reconcile`, `--fix-sprite-ids` and `--rebuild-offers` run the existing `reconcileFurniDataWithItemsBase`, `verifyAndFixSpriteIds` and `rebuildCatalogOfferIds` passes — the post-import consistency work. - `--align-ids` and `--rename-bundles` delegate to the existing standalone scripts; both only report until `--apply` is given, matching their own contracts. - The menu grows options 6-8 plus a dry run, each with its own confirmation. `--everything` turns on every repair + import step but deliberately leaves the destructive ones off. A live dry run against Wibbo listed 26593 items on the source that this hotel does not have, and Habbo (COM) reported 2871.
This commit is contained in:
1 parent
a4bcd40e7b
commit
f56b345598
4 files changed
+563
-15
No files matched your search
+39
-3
@@ -66,13 +66,49 @@ The menu asks which sources to compare (Habbo (COM) is the preset) and what may
|
|||||||
be written. With no arguments it needs a terminal; with any flag it skips the
|
be written. With no arguments it needs a terminal; with any flag it skips the
|
||||||
menu, so it is usable from cron.
|
menu, so it is usable from cron.
|
||||||
|
|
||||||
|
Every step the engine exposes is available as a flag:
|
||||||
|
|
||||||
|
| Flag | What it does |
|
||||||
|
| :--- | :--- |
|
||||||
|
| `--sources=<a,b>` | Compare against these sources (id or display name) |
|
||||||
|
| `--icons` | Re-download missing furniture icons |
|
||||||
|
| `--nitros` | Re-download missing bundles (stored as `.hab`) |
|
||||||
|
| `--furnidata` | Repair `FurnitureData.json` (add missing, drop duplicates) |
|
||||||
|
| `--structure` | Repair orphaned / duplicate catalog rows |
|
||||||
|
| `--check-ids` | Check `FurnitureData.json` for spriteId conflicts |
|
||||||
|
| `--sql` / `--apply-sql` | Generate / apply the catalog SQL |
|
||||||
|
| `--organize-sql` | Also emit the `catalog_pages` INSERTs into the SQL file |
|
||||||
|
| `--import-new` | Import furniture the source has and this hotel lacks |
|
||||||
|
| `--only=<a,b>`, `--limit=N`, `--concurrency=N` | Narrow and tune the import |
|
||||||
|
| `--reconcile` | Align `FurnitureData.json` with `items_base` after importing |
|
||||||
|
| `--fix-sprite-ids` | Correct `items_base.sprite_id` drift |
|
||||||
|
| `--rebuild-offers` | Rebuild catalog `offer_id` mirrors |
|
||||||
|
| `--align-ids` | Move `items_base` ids onto their furnidata ids (report; `--apply` runs it) |
|
||||||
|
| `--rename-bundles` | Rename legacy `.nitro` bundles to `.hab` (report; `--apply` runs it) |
|
||||||
|
| `--everything` | Every repair + import step, but still none of the destructive ones |
|
||||||
|
|
||||||
|
`--import-new` is the step that actually adds furniture: `cloneSingleFurni`
|
||||||
|
downloads the icon and the bundle (sources serve `.nitro`, the local copy is
|
||||||
|
stored as `.hab`), appends the `FurnitureData.json` entry **and** creates the
|
||||||
|
catalog page plus the offer, exactly like the app's own import. It skips
|
||||||
|
anything already present, so it is safe to re-run.
|
||||||
|
|
||||||
Safety notes:
|
Safety notes:
|
||||||
|
|
||||||
- **An audit writes nothing** — no repair flag means read-only.
|
- **An audit writes nothing** — no repair flag means read-only.
|
||||||
- `--apply-sql` is the only flag that changes the catalog, and the generated SQL
|
- `--dry-run` together with `--import-new` lists what would be imported and
|
||||||
is always written to `storage/catalog-sync/*.sql` (gitignored) so it can be
|
imports nothing; use it first on a source before importing thousands of items.
|
||||||
reviewed or replayed by hand.
|
- Generating the catalog SQL creates the `catalog_pages` sub-pages it references
|
||||||
|
(same as the app's import), so a "generate only" run is not entirely
|
||||||
|
read-only — only the `catalog_items` rows wait for `--apply-sql`. The SQL also
|
||||||
|
lands in `storage/catalog-sync/*.sql` (gitignored) so it can be reviewed or
|
||||||
|
replayed by hand.
|
||||||
|
- `--apply-sql` is the only flag that changes the catalog; `--apply` authorises
|
||||||
|
the two heavyweight child scripts (`align-db-ids`, `migrate-nitro-to-hab`).
|
||||||
|
Both only report without it.
|
||||||
- The engine is the same one the admin *Catalog audit* page streams
|
- The engine is the same one the admin *Catalog audit* page streams
|
||||||
(`runCatalogAudit`), so a terminal run and an in-admin run agree.
|
(`runCatalogAudit`), so a terminal run and an in-admin run agree.
|
||||||
- `--json` prints one machine-readable report; `--fail-on-errors` exits `3` when
|
- `--json` prints one machine-readable report; `--fail-on-errors` exits `3` when
|
||||||
error-severity issues remain, which is what a scheduled check should use.
|
error-severity issues remain, which is what a scheduled check should use.
|
||||||
|
- Items that no compared source can provide stay missing: the run reports them
|
||||||
|
instead of inventing an icon or a bundle.
|
||||||
+38
-3
@@ -99,9 +99,15 @@ What should be done? (comparing with: $SOURCE_LABEL)
|
|||||||
1) Audit only — nothing is written
|
1) Audit only — nothing is written
|
||||||
2) Repair furniture icons + nitro bundles
|
2) Repair furniture icons + nitro bundles
|
||||||
3) 2 + FurnitureData.json + catalog structure
|
3) 2 + FurnitureData.json + catalog structure
|
||||||
4) 3 + generate the catalog SQL (saved, not applied)
|
4) 3 + generate the catalog SQL (saved, not applied)
|
||||||
5) 4 + APPLY the catalog SQL to the catalog
|
5) 4 + APPLY the catalog SQL to the catalog
|
||||||
q) Cancel
|
6) 5 + import the furniture the source has and this hotel lacks
|
||||||
|
(downloads icon + bundle, adds FurnitureData.json, creates the catalog
|
||||||
|
page and the offer for every new item)
|
||||||
|
7) 6 + reconcile FurnitureData.json / sprite ids / offer ids
|
||||||
|
8) 7 + align items_base ids and rename legacy .nitro bundles (heaviest)
|
||||||
|
d) Dry run — only report what 6 would import
|
||||||
|
q) Cancel
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
read -r -p "Action [1]: " ACTION_CHOICE
|
read -r -p "Action [1]: " ACTION_CHOICE
|
||||||
@@ -132,6 +138,35 @@ case "$ACTION_CHOICE" in
|
|||||||
ACTION_FLAGS=(--all --sql --apply-sql)
|
ACTION_FLAGS=(--all --sql --apply-sql)
|
||||||
CONFIRM="This writes to the catalog. Continue? [y/N]: "
|
CONFIRM="This writes to the catalog. Continue? [y/N]: "
|
||||||
;;
|
;;
|
||||||
|
6)
|
||||||
|
ACTIONS=(audit "everything + catalog SQL + import new furniture")
|
||||||
|
ACTION_FLAGS=(--all --sql --apply-sql --import-new)
|
||||||
|
CONFIRM="This writes to the catalog AND imports every missing item. Continue? [y/N]: "
|
||||||
|
;;
|
||||||
|
7)
|
||||||
|
ACTIONS=(audit "everything + import + reconcile")
|
||||||
|
ACTION_FLAGS=(
|
||||||
|
--all --sql --apply-sql --import-new --reconcile --fix-sprite-ids
|
||||||
|
--rebuild-offers
|
||||||
|
)
|
||||||
|
CONFIRM="This writes to the catalog AND imports every missing item. Continue? [y/N]: "
|
||||||
|
;;
|
||||||
|
8)
|
||||||
|
ACTIONS=(audit "everything + import + reconcile + id alignment + rename")
|
||||||
|
ACTION_FLAGS=(
|
||||||
|
--all --sql --apply-sql --import-new --reconcile --fix-sprite-ids
|
||||||
|
--rebuild-offers --align-ids --rename-bundles --apply
|
||||||
|
)
|
||||||
|
read -r -p " This rewrites items_base ids and renames bundle files. Type RUN to confirm: " REPLY
|
||||||
|
if [ "$REPLY" != "RUN" ]; then
|
||||||
|
echo "Cancelled."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
d | D)
|
||||||
|
ACTIONS=(audit "report only — nothing is imported")
|
||||||
|
ACTION_FLAGS=(--import-new --dry-run)
|
||||||
|
;;
|
||||||
q | Q)
|
q | Q)
|
||||||
echo "Cancelled."
|
echo "Cancelled."
|
||||||
exit 0
|
exit 0
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
+372
-9
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user