The previous commit made imports update the Studio without a reload, but the
guarantee only held inside the tab that started the import and only as long as
every read succeeded. Four holes were left, and this closes them.
A session that mounted the tree before an import kept the pre-import tree for
the rest of its life, because ensureCatalogTreeLoaded() was a once-per-session
no-op. It now asks the server whether what it holds is still current. The answer
is a revision: sendCatalogUpdate() already runs after every catalog write, so it
bumps one, and clients read it on mount, on focus, on a 20s poll and from other
tabs over a BroadcastChannel. An import that finishes in another tab, another
browser or the job worker now lands here too.
A failed read used to be swallowed, which is the worst outcome available: the
rail kept showing pre-import counts as if they were current and nothing said so.
The snapshot now carries the error, the rail shows it with a retry, and the
previous tree stays on screen because stale beats empty.
Every settled import pulled the entire flat tree, which is the one payload that
grows with the size of the catalog. The revision doubles as the ETag on
mode=full, so an unchanged catalog answers 304 and the poll costs a file read.
An import could also report success for an offer the hotel will never sell: a
hidden or disabled page, an item_ids that misses the furni id, a zero amount.
importSingleFurni reads its own row back and reports each of those as a warning,
where the import report already is, instead of leaving it to surface as "the
import did not work" in the client.
Finally, the catalog items table no longer falls back to router.refresh() —
onRefresh is now required, so every mutation ends in a refresh of the caller's
own data instead of a route re-render that threw away editor state and scroll
position. useServerAction keeps its default, because 47 callers across the app
depend on it. The 750-line CatalogTree in catalog-tree.tsx was dead code that
kept its own stale tree and three more router.refresh() calls; only CatalogIcon
and LAYOUT_COLORS are still imported, so the rest is gone.
Tests: the store now covers revisions, 304s, probe failures and error recovery;
a jsdom test mounts a consumer and asserts the tree updates in place with no
navigation; the old organize-imports e2e asserted nothing about the endpoints
the code actually calls, and is replaced by one that asserts a cross-tab write
lands in the mounted categories without a reload.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The live catalog store only covered part of the import surface. A durable
job settled, a sync queue drained, a .nitro upload or a clone run left the
Studio rail and the stats bar showing pre-import numbers until the page was
reloaded, and the Catalog Manager kept a second tree that never saw writes
made elsewhere in the session.
Every one of those paths now pulls the tree again, and the refresh carries
the totals with it: importing writes catalog rows server-side, so the counts
the store holds were stale for the rest of the session.
- refreshCatalogTree shares one request between concurrent callers and queues
a single follow-up read when a write lands mid-flight, so a burst of edits
costs at most one extra read.
- useFurnitureJobs treats its first payload as a baseline, so a page load no
longer replays every past import as "just settled", and hands the settled
jobs to the callback.
- The Catalog Manager pushes its own mutations into the store and re-reads its
active tab when the store changes.
- The 30s unstable_cache on the admin totals is now tagged and invalidated from
every catalog write, including the import worker, so it no longer survives an
import even across a hard reload.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
"Missing furnidata" was only a filter in the Studio status dropdown, so
imported items whose classname was absent from FurnitureData.json could
be found but not fixed from that screen. Only the Catalog Audit page could
repair them, and only globally.
Adds the same shape of quick action that "no nitro" already had:
- studio-client.tsx: a "N no furnidata" shortcut next to the "N no nitro"
button that sets the missingFurnidata status filter, and a bulk "Add
missing furnidata (N)" button for the selected rows. Both only appear
when there is something to act on. Rows that come back repaired flip
to hasFurnidata: true so the badges and counts update in place; rows
the server reported in errors keep their state.
- resync/route.ts: accepts an optional { classnames: string[] } body to
target exactly the selected rows. classnames are resolved through the
same normalized local index the listing uses to decide hasFurnidata, so
the rows written are the rows flagged as missing. The upsert is already
idempotent, and RCON updateCatalog + updateItems run afterwards so the
emulator picks the new entries up.
Also clears the Studio furnidata cache after a write, which this route
never did: without it the listing kept serving a stale hasFurnidata for
up to the 30s cache TTL, so a repair looked like it had done nothing.
PERMS is now imported from permission-slugs (identical re-export) so the
route no longer pulls next-auth into tests.
- studio-filters.test.ts: pins the missingFurnidata branch, in particular
that an unchecked item (hasFurnidata undefined) is not treated as missing.
The existing ?days / ?missing / ?broken / ?all modes are unchanged; the
body is only consulted when it carries a classnames array.
Organising imports, the Studio furni batch, the catalog totals and the
"import from a source" stats all used to need a full page reload, or at
best a router.refresh() that re-rendered the whole admin route, before
anything on screen reflected what the import had just written.
- live-catalog-merge.ts (new): pure tree and total arithmetic. Applies a
delta of created pages, added offers and moved offers, recomputes depth
for the touched subtree, bumps parent child counts and the item totals.
Returns the input untouched when a delta is empty, so subscribers can
bail out instead of re-rendering. Depth resolution tolerates a parent
cycle in a dirty DB and still terminates, matching getTreeFlat.
- use-live-catalog.ts (new): one module-level store exposed through
useSyncExternalStore, so every consumer shares a single instance without
threading a provider through the admin layout. Deltas only apply to the
"normal" catalog, so public and public_handlers trees stay separate.
seedCatalogTotals() takes the first server value per mode and never
overwrites it afterwards, so a later hard render cannot make the header
totals jump backwards.
- actions/catalog.ts: organizeImportFurni now reports each group through
the new OrganizedPageChange, carrying parentId, pageLayout, the icon,
isNew and the per-source movedFrom counts, so the client can fold the
result into the tree without reading the page back.
- organize-imports-dialog.tsx: drops useRouter and router.refresh(); the
response is applied as a delta the moment the run finishes.
- studio-client.tsx: reads the tree from the store instead of freezing it
with useState(initialTree), loads it on mount when empty, and refreshes
it once a batch import settles. The batch is server-side and derives its
import pages from furnidata, so that one path re-reads the tree via
GET /api/admin/catalog/tree?mode=full rather than trusting the delta.
- studio/furni/page.tsx: stops calling getTreeFlat() and no longer passes
initialTree; the store is the single source of truth for the rail.
- import-clone-client.tsx: tracks which items are already present, so
present and clonable update per cloned row instead of only at the end.
- catalog-manager-dialog.tsx: seeds the totals once and renders the live
values, so the header reflects an import that just ran.
- e2e/ui/fixtures/entry.tsx: drops the removed initialTree prop.
Add a gamedata cleanup to the Nitro Cleanup panel: a read-only preview
plus an apply run that dedupes FurnitureData classnames and removes
rows and figure entries that reference nothing.
Three passes run in a fixed order, because cleanFigureMap has to precede
cleanFigureData: dropping the part that points at a set is what makes
that set unreferenced.
A pass refuses to write when it would delete more than maxRemovals rows
(default 500) and reports the reason, a wrong asset directory otherwise
turns every row into an orphan and one call would empty the file. Passes
that would act on empty input (no libraries, no sets) treat that as a
missing file rather than as a reason to delete everything. Every write
copies the file to a timestamped backup first, so a pass that turns out
to be wrong can be undone by hand.
The plan reads FurnitureData once and hands the parsed copy to both
furniture passes; the file is tens of megabytes in a real deployment.
Newly converted .nitro bundles now store their spritesheet as WebP VP8L
instead of PNG, so imports land much smaller without changing a single
pixel. The texture member and spritesheet.meta.image are both labelled
from the actual bytes, never from a caller's assumption.
- encode through sharp with lossless and exact, so colour hidden under
alpha 0 survives; this mirrors ImageSharp's TransparentColorMode.Preserve
- detect PNG/WebP by magic bytes and reject anything the client cannot
render, on create, download and upload paths
- keep the source format when deriving size-32 sheets, scaling composites
and editing metadata, so existing bundles are never silently rewritten
- report fidelity in the studio: the compression panel re-encodes with the
same options the importer uses, so it cannot drift and invent false
warnings, and shows PNG/WebP size estimates
convertSwfToNitro and buildSpritesheet are now async, so the worker, the
main-thread fallback and every import call site await them. PNG stays
supported for existing bundles and icon sidecars are untouched.
Extract FurniThumb, LayoutPreview, and GroupItemList into a dedicated
mall-helpers module alongside OrganizeImportsDialog. Preserves all
virtualization, drag-and-drop, and preview behavior while reducing
the main dialog component size.
The cleanup scan used to read every .nitro bundle in full and decompress
the large PNG texture just to confirm the file is structurally valid. On
directories with hundreds of thousands of bundles this took minutes, the
reverse proxy cut the request at its 30s timeoutable with an HTML 504, and
the panel then crashed with "Unexpected token '<'".
Validate bundles with a cheap header-only read (a few KB, no decompression)
that mirrors parseNitroBundle's byte layout; only files whose header looks
suspicious get the expensive full parse. Robust against downloads that
landed as an HTML error page, truncated or zero-filled files. The scan
drops from minutes to seconds on large nitro directories.
Also guard the panel against non-JSON (proxy error page / HTML) responses
so it reports a clear error message instead of a JSON parse failure.
Scan distinguishes fake, broken, and orphaned SWF/icon assets with age
metadata, deletes per asset kind, re-downloads broken nitro bundles from
configured sources, auto-cleans old fake leftovers, and exports a JSON
manifest. Adds rebuild and auto-clean API endpoints with audit coverage
and a housekeeping preview route under the hotel domain.
Verified: full vitest suite (2213 tests), typecheck, and biome all pass.
- Native HTML5 drag & drop between categories with drop-target highlight
- Virtualized item lists via @tanstack/react-virtual (fixed 34px rows)
- Auto-batching of large groups (500 items / 50 groups per run)
- Duplicate detection against the destination page with badge + summary
- Per-category layout preview grid
- Undo history for item moves (single + batch, tracks source groups)
- Days-range selector to load older imports (30/60/90/180)
- LocalStorage persistence of user settings (mode, destination, price, days)
- Added translation keys across all 25 locales
Mirror interactive batch runs into the import-job store so interrupted imports (restart, time-out, disconnect) can be resumed from Import History. Items are checkpointed as they settle (coalesced, serialized saves) and the mirror starts 'running' so the boot-time worker marks it 'interrupted' instead of double-importing; done items are never re-imported. Add bounded backoff retry for transient download/connection failures before marking an item failed, and point the client's time-out/network toasts at Import History.
Highlight the active search term in names/classnames (grid + table), add zebra striping and a left accent bar on selected table rows, fade the results list when switching grid/table or on first load, and swap the broken-icon fallback for a cleaner placeholder.
Raise batch concurrency (furni 3->12, clone 10->12) with a Speed control next to Translate. Skip the SWF download when a .nitro bundle already exists on disk (color variants share the base nitro), and stop flagging that as a failed download.
- Render the table view through a virtualizer too, using a shared grid
template so the sticky header and rows keep perfect column alignment
- Keep semantic table/row/cell elements while virtualizing
- Cap the batch item-details list to the latest 60 rows (newest first)
- Coalesce per-item progress events server-side (120ms throttle) in both
the exact-import and clone SSE batch runners
- Add TTL-based caching for local index lookup, furnidata classnames,
catalog id set, nitro file presence and import stats
- Invalidate caches after furnace single/batch/clone imports
- Rewrite batch progress with elapsed time, rate and verification chips
- Virtualize the grid with @tanstack/react-virtual and replace the
Load more button with infinite scroll via an IntersectionObserver
The "Create" CTA used totalSelected (the sum of furni items across
approved groups) as its plural count, so with many imported items it
claimed to create thousands of pages. One approved group creates exactly
one page, so the label now counts approved groups instead.
resolveNitroFrame now matches spritesheet frame keys that carry a .png
suffix or namespaced naming, and isScale/scaleName preserve that suffix.
Broken source sprites (missing frames or references to icon artwork) are
skipped and reported instead of aborting the whole generation, and the
studio UI surfaces the skipped count.
Adds a second mode to the organize-imports dialog: instead of creating
one new page per approved group (which could produce dozens of tiny
pages), the user can pick an existing destination page and have every
approved item moved into it. No catalog page is created in this mode.
- organizeImportFurni: groups accept destinationPageId; when set, the
existing page is reused, new offers append after its current highest
order, moved offers keep their original name, and the real page
caption is used for logging and results
- OrganizeImportsDialog: mode toggle (create pages / move into page),
searchable destination picker via /api/admin/catalog/tree?search=,
name/icon/layout editors hidden in move mode, button shows a move
count, and the success toast reports moved/added instead of pages
- en + nl translations for the new mode, destination, and move keys
The original GET route used a correlated NOT EXISTS / FIND_IN_SET
subquery over the entire catalog_items table for every recent import
audit entry, causing server timeouts when the audit log or catalog
grew large. The per-item host-page validation inside the create
action also issued one SELECT + one UPDATE per moved offer.
Changes:
- GET /api/admin/import/organize: replace the correlated subquery
with a bounded candidate list and a JS-side placed-set check, then
resolve all needed base items in a single indexed SELECT. This
bounds the query cost regardless of catalog or audit log size.
- organizeImportFurni action: validate mover ids in one SELECT, then
batch every move per group into a single UPDATE with a CASE
expression instead of one UPDATE per item.
- OrganizeImportsDialog: add a 45-second abort timeout on the fetch
and a distinct load-error state so the UI never silently hangs.
- Add 'loadError' translation key (en + nl).
Adds a Studio 'Organize imports' dialog that groups recently imported
furniture and furniture already sitting in the auto-created import
pages into suggested catalog categories. Each group is presented with
its suggested name, icon, and layout which can be overridden before
approval; approved groups are turned into real catalog pages in a
single atomic export run. Offers already inside the import subtree are
moved to the new pages; brand-new furniture gets a fresh offer.
- groupSuggestedCategories: generic pure helper reusing the same
per-item label heuristic that drives suggestCategoryName; items with
no label land in a 'Other Furni' remainder bucket
- GET /api/admin/import/organize: returns items from the imported
furniture tree (catalog_items JOIN items_base via page id set from
the imported-furniture root) union audit-logged but not-yet-placed
recent imports; marks alreadyPlaced vs new
- organizeImportFurni server action: validates import-page membership
before moving any offer, creates pages + inserts/moves items in one
withCatalogExport snapshot, logs activity
- OrganizeImportsDialog: full-featured Studio dialog with price panel
(applies to new offers only), parent select, per-group approval,
editable name/icon/layout with suggestion reset chips, source tags
- import-pages.ts server helper: locates the imported-furniture tree
- Studio nav: 'Organize imports' button with FolderTree icon,
gated on CATALOG_EDIT, wired next to the catalog manager
- en + nl translations for organizeImports.* keys with ICU plurals
- groupSuggestedCategories unit tests (deterministic grouping,
remainder handling, size+alpha ordering, label consistency)
openhandsandClaude Opus 4.8
Simo