Per single furni import, patchLocalizedFurniDataEntries ran for every
language and did a full 100k-entry JSON.parse + scan + JSON.stringify on the
main thread. That CPU spike is now offloaded to the translation worker
(init/prepare/finalize reuses the same pure core helpers), so importing
never blocks the event loop. The main thread only does async file I/O and the
network LibreTranslate calls. Falls back to the same helpers on the main
thread if no worker is available.
Also cache readFurniData() by file mtime+size so the 100k-entry JSON is
parsed once per change instead of on every import/fix/reconcile read
(invalidated on write).
The translation build's CPU-heavy work (deep clone of the 100k+ entry
master, the two full scans and JSON.stringify per language) now runs in a
worker thread, so "Alles vertalen" no longer blocks the main event loop.
Network/DB parts (official Habbo fetch, LibreTranslate, file writes) stay
on the main thread. The pure helpers were extracted to furni-data-i18n-core
(no server-only deps) so the worker can import them. Falls back to the same
helpers on the main thread when no worker is available or in tests.
The synchronous convertSwfToNitro / extractIconFromSwf calls (pure CPU,
no DB/network) now run in a small worker pool so the main Node event
loop stays free during imports — the dominant import-time CPU spike is
moved off the request thread. The pool degrades gracefully to a
synchronous fallback if workers can't be created, and is skipped in the
test environment.
- patchLocalizedFurniDataEntries now only touches the wanted classnames
instead of scanning the whole 100k file, reads the master once (was
twice), and uses the batched translator instead of one HTTP call per
text.
- Remove the now-unused serial translateWithLibre.
- buildLocalizedFurniDataFiles yields between the two section scans and
before stringify so a single language no longer blocks the event loop
for the whole pass at once.
Lower LibreTranslate concurrency to 2 and yield the event loop between
languages so the 13x deep-clone + stringify loop no longer pins a core
continuously. The build is also already opt-in (off by default on import).
The maintenance "Fix alles" now has a checkbox to include or skip the
duplicate catalog_pages merge, so admins can choose whether to run it.
removeDuplicates takes an includePages flag and fixEverything threads it
through to the action.
Furni imports can now choose whether to translate (per import) and which
languages to build, instead of always rebuilding all 13 FurnitureData_<lang>
files. The studio header also has a global switch that persists the
furnidata_translate_enabled setting, seeding the per-import default.
The maintenance "Fix alles" now collapses catalog_pages that share the same
caption_save + parent_id into a single survivor (moving catalog_items and
reparenting child pages before deleting duplicates), so the catalog tree
never renders double links. The duplicate-page count is also surfaced in the
health panel.
- Move /admin/import/* tools under /admin/studio/* and add studio nav, layout and tabs
- Add shared catalog maintenance panel plus a /admin/studio/maintenance tab
- Make furnidata reconciliation overwrite conflicting entries with the
DB-authoritative items_base classname/id (surfaced via fixedConflicts)
- Add furnidata_translate_enabled setting to skip the heavy localized
furnidata build during import (manual build-languages still forces it)
- Update staff smoke contract test for the studio hub
Translate deduplicated texts in batches of 48 with 4 parallel workers
instead of one request per string, serve cached hits locally, and
persist results so rebuilds never re-call the API for the same text.
Every write path now sets offer_id = catalog_items.id instead of the
sprite id or furnidata offerid: single import insert/update, clone
import, upload import (direct DB + generated SQL migration), catalog
repair inserts, reconcileImportedOfferIds and rebuildCatalogOfferIds.
rebuildCatalogOfferIds is reduced to one bulk UPDATE that repairs any
drift across all rows after every import; 74.838 legacy rows were
repaired on the live database with this change.
- Change skipFurniDataWrite from true to false in both import routes
- Previously: FurnitureData.json entries were deferred/skipped during import
- Now: Entries are written immediately after each furniture item import
- Result: FurnitureData.json is updated directly, translations work again, offer_id fix preserves correctly set values
- Change condition from to
- Prevents overwriting offer_id that was correctly set to sprite_id during furniture import
- Fixes catalog showing wrong furniture when offer_ids and sprite_ids don't match
- Switch default EMULATOR_MODE from 'rcon' to 'api'
- Remove RCON_HOST/PORT/timeout/maxRetries env vars
- Add EMULATOR_API_URL to .env and .env.example
- Update createEmulatorTransport() to use HTTP API only
- Remove rconHost/port from offline alert context
- All 827 tests pass, TypeScript compiles cleanly
- Add EMULATOR_MODE=api option with HTTP transport mirroring RCON payload
- buildLocalizedFurniDataFiles now falls back to LibreTranslate for customs (incl. pl/ru/ar/ja)
- Remove Organiseer import button from studio and Reorganize Catalog from import page
- Delete organize API route and PUT handler; remove organizeCatalogAll from catalog-repair.ts
- Increase imported-furni list perPage from 50 to 500
tryDownloadCandidates now fires every candidate URL concurrently and keeps
the first valid response, removing the old 15s×retry sequential waits on
slow/unreachable sources that made import speed uneven. Batch concurrency
raised 8 -> 12 to absorb the now-faster downloads.
AddSize32ToBundle downscales the largest existing sprite (usually size 64)
to generate a 32 visualization, so every bundle — including the
source-provided gamedata furniture — carries a 32 frame without needing the
original SWF. AddSize32AllNitros rewrites every existing .nitro across all
nitro directories (incl. /var/www/Gamedata/bundled/furniture) and is
idempotent (bundles that already have 32 are skipped). Expose it via a Studio
button and the background admin endpoint POST /api/admin/import/furni/add-size32.
The SWF-to-Nitro converter previously dropped the size 32 visualization
and any sprite image whose name contained '_32_', so bundles lacked the
small/catalog render size. Keep them so each furniture bundle renders
correctly at every scale the client requests.
- xml-processor: stop skipping visualization size === 32
- index.ts: stop excluding '_32_' sprite assets/images from the sheet
Run the independent icon and swf downloads in parallel via Promise.all
instead of sequentially. The nitro fallback still runs after the swf check
since it depends on the swf result.
Combined with batch concurrency and the single FurnitureData write, this
further cuts per-item import time (roughly halves the download phase).
- Run batch imports with bounded concurrency (IMPORT_BATCH_CONCURRENCY = 8)
instead of fully sequentially, parallelizing network downloads and DB writes.
- Collect FurnitureData.json entries and write the whole file once per batch
(appendFurniEntriesBulk) instead of rewriting it on every single item.
Catalog id allocation stays serialized and FurnitureData writes remain
lock-guarded, so concurrent imports are safe.
organizeCatalogAll no longer pulls every items_base row into the catalog.
It now touches only items whose catalog entry lives on a page under the
Imported Furniture parent (re-homing/re-pricing onto line/category
sub-pages). Items are never newly added. Button renamed to 'Organiseer
import'.
Line pages used caption_save 'imp_line_<slug>' which could exceed the
25-char column limit (e.g. imp_line_pixel_collectible = 26 chars),
causing the INSERT to fail. Cap the slug at 16 chars.
Add a fully automatic "Organize all" action in the Studio that organizes
the entire database into catalog pages:
- Create a dedicated sub-page per furniture line (e.g. weebz, habbox) from
FurnitureData.json, falling back to the auto-detected category.
- Add missing purchasable items to the catalog on their line/category page.
- Re-home and re-price existing catalog entries onto their correct page.
- Pages are created automatically when missing; no manual selection needed.
Includes the organizeCatalogAll service, a POST /api/admin/import/furni/organize
endpoint, and the Studio button.
- Extended fixDatabaseConsistencyAfterImport() to also set cost_credits = 3 where <= 0
- Integrated into both single and batch import routes
- API responses now include costCreditsFixed count
Ensures 100% of catalog_items have cost_credits > 0 after import
- Typecheck, lint, tests all pass
- Pushed to GitLab
- Added fixDatabaseConsistencyAfterImport() function in furni-import.ts
* Fills empty catalog_name from items_base.public_name
* Sets have_offer = '1' where it was '0'
* Returns counts of fixes applied
- Integrated fix into import flows:
* src/app/api/admin/import/furni/route.ts (single & batch POST)
* src/app/api/admin/import/furni/batch/route.ts (SSE batch POST)
- API responses now include:
* catalogNameFixed: number of catalog names filled
* haveOfferFixed: number of have_offer values fixed
- Ensures 100% catalog_name coverage and have_offer=1 after every import
- Typecheck, lint, and all 827 tests pass
Extend FURNIDATA_LANGUAGES from 8 to 13 (nl/en/de/fr/es/tr/it/pt plus
fi/pl/ru/ar/ja). Official Habbo translations for fi (habbo.fi),
pt (habbo.com.br) use hotel gamedata; pl/ru/ar/ja fall back to
LibreTranslate en->xx. Docker LibreTranslate now loads 13 language
models (en,nl,de,fr,es,tr,it,pt,fi,pl,ru,ar,ja).
Use official Habbo translations where available; fall back to
self-hosted LibreTranslate (http://127.0.0.1:5000, host-network, 7
languages) for custom furni with no official translation. Patch path
(single imports) translates name/description via LibreTranslate
(en -> nl/de/fr/es/tr/it) with persistent cache
(.translations_libre_cache.json) and caches 6h for official maps.
Full rebuilds stay official-only to avoid 500k+ API calls; customs are
translated incrementally per import. LibreTranslate URL configurable via
site setting libretranslate_url. Docker: host-network fix for DNS.
Generate localized FurnitureData_<lang>.json files from the master
FurnitureData.json + official Habbo hotel translations. Fetches per-hotel
furnidata with in-memory cache (6h TTL), maps by classname (* variant
aware), and overrides name/description where a translation exists.
Full rebuild after every batch/batch-regen/clone import and as admin
GET ?action=build-languages; single imports patch incrementally via
patchLocalizedFurniDataEntries. Failures are best-effort and never fail
the import itself. Files are written to every configured gamedata write
path (primary + mirrors) so /gamedata/config/FurnitureData_<lang>.json
is available for per-language Nitro clients.
New service: src/lib/services/furni-data-i18n.ts with
buildLocalizedFurniDataFiles, patchLocalizedFurniDataEntries,
FURNIDATA_LANGUAGES (nl/en/de/fr/es/tr/it) and applyTranslationsForTest.
The emulator sends items_base.sprite_id to the client and Nitro resolves
the furniture by looking up that id in FurnitureData.json. Legacy rows
where sprite_id drifted from the id made the client render a completely
different item (e.g. a wired or custom asset instead of the purchased
furniture).
Add verifyAndFixSpriteIds() which repairs any row whose sprite_id no
longer matches its id while the local furnidata carries the item under
that id, and run it after every import path (single, batch, batch-regen,
clone) next to the offer_id rebuild.
Interaction detection:
- Trust SWF-derived sit/lay/stand flags only when the logic XML actually
contains action data (new hasActions metadata); otherwise fall back to
keyword detection so custom furni without <action> nodes are still
classified correctly
- Add French/Dutch/German/Spanish/Italian keywords (chaise, banquette,
stoel, silla, sedia, stuhl, tafel, mesa, ...) to sit/lay/stand detection
with token-boundary matching to avoid false positives like bedside_table
- Unify interaction_modes_count priority: mechanic fixed modes, then raw
animation state count, then sit/lay fallback, then keyword default
- Detect mechanic type even when real flags are not used
Catalog integrity:
- Skip duplicate catalog_items inserts in clone and upload imports
- Re-check live catalog rows before applying generated repair SQL
- Add rebuildCatalogOfferIds() which rebuilds every catalog_items.offer_id
from the local FurnitureData.json (matched by entry id, then classname)
and run it after every import path (single, batch, batch-regen, clone)
- Add /api/admin/import/furni?action=live-reconcile POST endpoint that reconciles FurnitureData.json with items_base without triggering RCON hotel reload
- Add 'notImported' status filter in studio to show all non-imported furniture items
- Useful for verifying imports and seeing what's missing after furniture import
The getOrCreateImportedParentPage() function was skipping database checks due to an early return cache check. This caused the 'imported_furni' catalog page to not be created when missing, preventing imported furniture from appearing in the catalog.
Removed the cache early return so the function always queries the DB and creates the page if needed.
Add verified retro-hotel sources to the studio furni clone list. All
endpoints (furnidata JSON, .nitro bundles, icons, SWF) were probed and
verified reachable (HTTP 200), and Leet/Hubbly nitro bundles parse with
parseNitroBundle.
Also allow source-specific SWF downloads without a revision path segment
(/hof_furni/{classname}.swf), which Leet and Habboon use.
Newly imported items with a generic .nitro logicType stayed on interaction_type
'default' until the post-import verification sweep ran. Fall back to the same
auto-detected type used by the sweep (classname mechanic + keyword + real flags)
so imports carry the correct interaction immediately; updates keep their
existing emulator handler types.
Also count cross-section id alignments under a separate normalizedIds field in
reconcileFurniDataWithItemsBase so fixedIds keeps its original meaning.
Classnames listed in both roomitemtypes and wallitemtypes keep a canonical copy
with the correct id/offerid while the stray wall copy can carry a legacy wrong
id. reconcileFurniDataWithItemsBase now normalizes those stray copies to the
offerid whenever the offerid maps to an items_base row whose classname matches
exactly, so id === offerid === sprite id holds for every uniquely-mapped
classname. The sandbox rehearsal asserts this invariant and the live database
has been reconciled (76 ids fixed, 0 remaining).
runCatalogAudit computed missingCatalogEntries from the pre-repair snapshot,
so generated catalog SQL could reference sprite ids deleted by the
duplicate-classname merge and create new orphaned catalog_items rows. Re-derive
entries after repairStructure and add a defensive guard in generateCatalogSql
that skips entries whose item id no longer exists in items_base.
Also fix the catalog_pages INSERT template (18 columns vs 17 values) which made
page creation always fail with 'Column count doesn't match value count'.
Add a sandbox-guarded live repair rehearsal test covering the full pipeline.
RENOVATE_CONFIG_FILE must point to a config file on disk; the inline
JSON string made Renovate abort with 'Custom config file ... must exist'.
The repo-root renovate.json (which already extends config:recommended)
is picked up automatically, so the env var is not needed.
- importSingleFurni: actually update an existing item instead of failing on a primary key INSERT collision; refuse to reassign a spriteId owned by a different classname
- reconcileFurniDataWithItemsBase: also sync each entry's offerid to the DB sprite id, not just the id
- Move the full-table interaction_modes_count verification out of the per-item import hot path and run it once per batch/single import
- clone-import: apply the auto-detected interaction_type (was hardcoded 'default'), set FurnitureData offerid and catalog_items.offer_id to the new local sprite id (was the source hotel's id / -1)
- clone batch route: run the same post-import reconcile/verify/ownership/RCON consolidation as the furni batch route
- upload-import: set offer_id to the allocated id in both the direct insert and the generated SQL migration (was -1)
- generateCatalogSql: use the classname as catalog_name and the item id as offer_id so rows match the app-managed import convention
- stats + missing-nitro endpoints: match catalog membership via item_ids instead of the catalog_name join, so translated display names no longer break counts
- deleteImportedItem/rollback: delete catalog rows by the canonical item_ids link only
- classifyFurni: wall items always classify as 'walls' before prefix rules (rare_/val_/xmas_) can misfile them
New POST /api/admin/import/furni/verify re-runs the full interaction
verification over items_base at any time, plus a Tools dropdown entry
in the furni import admin. It corrects interaction_type,
interaction_modes_count and allow_sit/lay/walk against real furniture
data (furnidata flags + .nitro animation states); items without real
data are skipped and existing emulator handler types are preserved.
verifyAndFixInteractionModesCount now corrects the full interaction
profile against real furniture data, not just the modes count:
- allow_sit/allow_lay/allow_walk set from real furnidata flags
- interaction_type filled in when the DB still has the generic
'default' (existing emulator handler types are never overwritten)
- allow_sit/lay/walk of 2 (emulator-special: tents, walk-through)
is preserved
Verified against the live DB: 0 sit/lay/stand mismatches remain
across all 82,737 items_base rows.
Replace classname/name keyword guessing with real data so sit/lay/stand
has zero false positives:
- autoDetectInteraction() now accepts real flags (cansiton/canlayon/
canstandon), logicType, and animation-state count; real data wins and
keywords are only a fallback when no real data exists.
- New furni-real-interaction helper reads the local FurnitureData.json
(highest-revision entries win; a true flag in any duplicate is kept)
and falls back to the official Habbo furnidata.
- interaction_modes_count now uses emulator conventions: dice=6,
gate/teleport=2, roller=1, generic multistate = animation-state count,
chairs/beds=1, plain items=0.
- verifyAndFixInteractionModesCount only touches items with real data
and skips everything else (no keyword overwrites).
- parseNitroBundle falls back to gunzip for gzip-compressed .nitro files.
- Wired through furni-import, upload-import, clone-import, and the nitro
editor auto-detect button.
- fetchSourceFurnidata now parses JSON regardless of content-type,
so GitHub raw mirrors (Kyzegs, sphynxkitten) that serve JSON as
text/plain work again instead of forcing a FlareSolverr round-trip.
- Add curl subprocess fallback (curl-fetch.ts) for CDNs that block
Node's fetch by TLS fingerprint (Leet.city) — used for furnidata,
nitro bundle and icon downloads before giving up.
- Merge verified default clone source presets with stored user sources
so the admin always offers many working sources.
- New auto-interaction.ts: detects canSit/canLay/canStand from classname/name keywords
- Maps keywords to interaction types (bench, chair, bed, table, etc.)
- Returns clickLimit and interactionModesCount for proper click/interaction limits
- Nitro editor: added auto-detect button with preview of detected settings
- furni-import.ts: auto-sets interaction_modes_count during import
- upload-import.ts: auto-detects for direct insert and SQL migrations
- clone-import.ts: auto-detects when cloning from other hotels
After a batch import the progress panel now shows a verification
summary (offer_id fixed, furnidata ids fixed / missing / conflicts,
nitros and icons synced, folders chowned). Single imports include the
key counts in the success toast.
After every single import, batch import and batch-regen, cross-check all
items_base rows against the local FurnitureData.json: entries whose
spriteId drifted are corrected to the DB id, and missing entries plus
real id conflicts are reported in the API/SSE response. Entries that are
missing entirely are counted (rebuilding them needs SWF/nitro metadata).
Extend the post-import backfill to also copy missing _icon.png files
into /var/www/Gamedata/icons (in addition to nitros into
bundled/furniture) so all imported furniture shows an icon in-game.
Report copied nitros and icons separately in the API/SSE responses.
After each single import, batch import and batch-regen, copy every
.nitro file that is missing from /var/www/Gamedata/bundled/furniture so
the emulator can render all imported furniture even when the mirror write
was skipped. Reports the copied files in the API/SSE response.
The Studio's batch import uses /api/admin/import/furni/batch, which never
ran the offer_id reconciliation or the www-data ownership fix (only the
single/legacy-batch route did). Run both after the batch finishes and
report the counts in the batch_complete SSE event.
After each single or batch import, chown the swf/icon/nitro asset
directories and the FurnitureData.json folders to www-data:www-data so
nginx and the emulator can read newly written files. Best-effort and
non-blocking; the API response reports which folders were fixed.
After each single or batch import, walk all catalog items under the
Imported Furniture tree and set offer_id to the furnidata sprite id,
repairing any stale values (e.g. legacy -1 rows). Reports how many
rows were fixed in the API response.
Import used a hardcoded offer_id of -1, while the matching
FurnitureData.json entry already carries offerid = spriteId.
Set catalog_items.offer_id to the sprite id on insert and update so
purchases resolve to the correct furni offer.
A concurrent import could create the same catalog sub-page twice
(getOrCreateCategoryPage / getOrCreateImportedParentPage do a
SELECT-then-INSERT with no unique constraint). Deduplicate right after
insert by keeping the lowest-id page and removing the duplicate, while
never deleting a page that already received catalog items.
- Card-shaped loading skeletons with shimmer text lines
- Hover lift and accent shadow on furni cards
- Subtle dotted floor pattern behind previews and detail image
- Sticky action footer in the detail drawer
- Theme-aware slim scrollbars inside the admin panel
- Nicer empty state with icon tile
- Add debounced live search with keyboard shortcuts (/ focus,
Ctrl/Cmd+A select all, Esc clear selection)
- Add category filter and sort controls (name, classname, category,
imported-first)
- Add grid/list view toggle with a compact list table
- Add regenerate .nitro action for missing-nitro items
- Show result count in the header and a missing-nitro shortcut chip
- Fix invalid nested button elements in furni grid cards that broke
card borders, hover styles and selection rendering
- Fix stale-closure bug where switching furni source fetched from the
previously selected source
- Add progress bar plus Cancel/Dismiss controls to batch imports
- Make the detail drawer an overlay on small screens
- Add clear-search and clear-selection actions in the toolbar
- Use pixelated image rendering and keyboard focus rings on cards
- Cache read-heavy public API routes via redisCache (leaderboard, values,
shop, articles, photos, guilds, teams, staff, users, home, radio, badges)
- Add single-flight and bounded-memory cache layer with unit tests
- Parallelize independent DB queries on search, rares, shop, staff, polls
and profile pages
- Push radio points leaderboard aggregation to SQL with a LIMIT
- Split studio-client and import-furni-client into focused modules
- Clean up next.config.ts
Allow the admin Studio to browse and import furniture from custom retro
hotel sources, not just official Habbo furnidata.
- GET /api/admin/import/furni?source=<id> lists a clone source's
furnidata (via getCloneList) and returns a per-item iconUrl from the
source's iconBaseUrl so previews render correctly
- Studio client gets a source selector dropdown (official Habbo plus all
configured clone_sources) that resets the selection and reloads the
list when switched
- Single and batch imports now send sourceId so SWF/nitro/icon assets
are fetched from the selected hotel's CDN
- Preview images prefer the source iconUrl with the existing fallback
chain; header shows the active source name
Add a full-viewport /admin/studio workspace that unifies the import and
catalog systems into a single automated flow:
- Furni library browser against official Habbo furnidata with search,
type/status filters, live previews and pagination
- Catalog structure rail showing where imported furniture lands
- Detail drawer with automatic placement preview (category + price via
classifyFurni/autoPriceFurni) and one-click import that downloads
assets, converts SWF to nitro, writes items_base, updates
FurnitureData.json and creates an auto-priced catalog entry
- Batch auto-import with SSE progress and RCON cache refresh
- Nitro editor integration and imported-item deletion
Extract pure auto-catalog helpers (CATEGORY_PAGE, CLASSNAME_RULES,
classifyFurni, autoPriceFurni) into a client-safe module shared by the
server pipeline and the Studio UI so the preview always matches what the
emulator import applies.
- Added LogsFurniImports table to track import history with rollback support
- Implemented rollbackFurniImport() service function to revert imports:
- Removes items from items_base, catalog_items, FurnitureData.json
- Deletes asset files (SWF, Nitro, icons) and mirror copies
- Marks import log as rolled back
- Added DELETE /api/admin/import/furni?importId=X endpoint
- Added dry-run support for import validation (?dryrun=1)
- Added updateExisting option to update existing catalog entries
- All TypeScript/Biome checks pass, tests pass (806 passed, 1 pre-existing skip)
- Added updateExisting parameter to importSingleFurni and API routes
- When updateExisting=true, existing items are updated (catalog price/page) instead of failing
- Added catalogUpdated flag to ImportSingleResult
- Updates existing catalog entries (price, page) instead of skipping duplicates
- Added spriteId conflict resolution to repairFurniData(): keeps first classname per id, removes conflicting entries
- Returns new removedIdConflicts count in repair result
- Updated audit event type and client UI to display removedIdConflicts
- Updated catalog-audit.ts event type and client state type
- When 'Repair FurnitureData.json' is checked, it now fixes id conflicts automatically
- Added checkFurniDataIds to POST body so the 'Check FurnitureData.json for spriteId conflicts' checkbox actually works
- Added checkFurniDataIds to useCallback dependency array for correct React hook behavior
- Prevents deploy-workflow-contract.test.ts timeout in full-suite runs
- Test runs in ~2s individually but hits 5s default under 30s+ import time
- 10s provides sufficient headroom under resource contention
openhands