Per single furni import, patchLocalizedFurniDataEntries ran for every
language and did a full 100k-entry JSON.parse + scan + JSON.stringify on the
main thread. That CPU spike is now offloaded to the translation worker
(init/prepare/finalize reuses the same pure core helpers), so importing
never blocks the event loop. The main thread only does async file I/O and the
network LibreTranslate calls. Falls back to the same helpers on the main
thread if no worker is available.
Also cache readFurniData() by file mtime+size so the 100k-entry JSON is
parsed once per change instead of on every import/fix/reconcile read
(invalidated on write).
The translation build's CPU-heavy work (deep clone of the 100k+ entry
master, the two full scans and JSON.stringify per language) now runs in a
worker thread, so "Alles vertalen" no longer blocks the main event loop.
Network/DB parts (official Habbo fetch, LibreTranslate, file writes) stay
on the main thread. The pure helpers were extracted to furni-data-i18n-core
(no server-only deps) so the worker can import them. Falls back to the same
helpers on the main thread when no worker is available or in tests.
The synchronous convertSwfToNitro / extractIconFromSwf calls (pure CPU,
no DB/network) now run in a small worker pool so the main Node event
loop stays free during imports — the dominant import-time CPU spike is
moved off the request thread. The pool degrades gracefully to a
synchronous fallback if workers can't be created, and is skipped in the
test environment.
- patchLocalizedFurniDataEntries now only touches the wanted classnames
instead of scanning the whole 100k file, reads the master once (was
twice), and uses the batched translator instead of one HTTP call per
text.
- Remove the now-unused serial translateWithLibre.
- buildLocalizedFurniDataFiles yields between the two section scans and
before stringify so a single language no longer blocks the event loop
for the whole pass at once.
Lower LibreTranslate concurrency to 2 and yield the event loop between
languages so the 13x deep-clone + stringify loop no longer pins a core
continuously. The build is also already opt-in (off by default on import).
The maintenance "Fix alles" now has a checkbox to include or skip the
duplicate catalog_pages merge, so admins can choose whether to run it.
removeDuplicates takes an includePages flag and fixEverything threads it
through to the action.
Furni imports can now choose whether to translate (per import) and which
languages to build, instead of always rebuilding all 13 FurnitureData_<lang>
files. The studio header also has a global switch that persists the
furnidata_translate_enabled setting, seeding the per-import default.
The maintenance "Fix alles" now collapses catalog_pages that share the same
caption_save + parent_id into a single survivor (moving catalog_items and
reparenting child pages before deleting duplicates), so the catalog tree
never renders double links. The duplicate-page count is also surfaced in the
health panel.
- Move /admin/import/* tools under /admin/studio/* and add studio nav, layout and tabs
- Add shared catalog maintenance panel plus a /admin/studio/maintenance tab
- Make furnidata reconciliation overwrite conflicting entries with the
DB-authoritative items_base classname/id (surfaced via fixedConflicts)
- Add furnidata_translate_enabled setting to skip the heavy localized
furnidata build during import (manual build-languages still forces it)
- Update staff smoke contract test for the studio hub
Translate deduplicated texts in batches of 48 with 4 parallel workers
instead of one request per string, serve cached hits locally, and
persist results so rebuilds never re-call the API for the same text.
Every write path now sets offer_id = catalog_items.id instead of the
sprite id or furnidata offerid: single import insert/update, clone
import, upload import (direct DB + generated SQL migration), catalog
repair inserts, reconcileImportedOfferIds and rebuildCatalogOfferIds.
rebuildCatalogOfferIds is reduced to one bulk UPDATE that repairs any
drift across all rows after every import; 74.838 legacy rows were
repaired on the live database with this change.
- Change skipFurniDataWrite from true to false in both import routes
- Previously: FurnitureData.json entries were deferred/skipped during import
- Now: Entries are written immediately after each furniture item import
- Result: FurnitureData.json is updated directly, translations work again, offer_id fix preserves correctly set values
- Change condition from to
- Prevents overwriting offer_id that was correctly set to sprite_id during furniture import
- Fixes catalog showing wrong furniture when offer_ids and sprite_ids don't match
Simo
openhands