Three separate things that were each costing more than they needed to on the
hot path.
- Single-flight avatar renders. The disk cache was checked first and a miss
went straight to the upstream, with nothing shared between callers, so a page
requesting dozens of avatars at once turned N concurrent requests for one
figure into N renders. A render is the most expensive operation this app
does, and the duplication happened exactly when the cache had nothing to
offer. Eight concurrent requests now cause one render instead of eight. The
map lives on globalThis because Next can evaluate the module more than once
per process, and two copies would each start their own render.
- Let public read-only routes be cached by a shared cache. Every JSON response
was `cache-control: no-store`, so a CDN in front of the app could not answer
any of it and every request reached the origin. publicCacheControl() opts a
route in with s-maxage and stale-while-revalidate, using the same TTL as the
server-side cache so the two layers cannot disagree. The default stays
no-store: most routes here are personalised, admin-only or auth-dependent.
/api/badges/leaderboard is deliberately left alone because it returns
per-viewer rank entries to signed-in callers.
Note this only takes effect once a cache rule exists for /api/* at the CDN, or
the explicit `cache: "no-store"` is dropped from the client fetches (24 files
do that today, including the /api/online poll). The headers alone are inert
until one of those happens.
- Take the homepage row counts from the storage engine estimate instead of
COUNT(*), which walks an index and gets slower as the tables grow. A missing
or zero estimate falls back to the exact count rather than ever showing a
wrong zero. The online count stays exact: it is an indexed read over a small
subset and a few seconds of drift reads as broken rather than approximate.
The counters move into one module because the homepage and the boot warm-up
populate the same cache keys, so two implementations would race to write
different values into the same entry.
3223 tests pass.
The in-process cache was a FIFO of 500 entries that was never touched on a
read, so a key polled on every request could be evicted by an unrelated burst
of dynamic keys. That looked exactly like the cache being cleared at random,
and it is what made the site fall back to the database unpredictably.
- Evict least-recently-used instead, and raise the default budget to 2000
(CACHE_MEMORY_MAX_ENTRIES). Reading a key now marks it as used, so a hot key
only leaves when a hotter one takes its place.
- Add opt-in stale-while-revalidate (CachedOptions.staleMs). The grace window
lives on the entry, so one call site opting in protects every reader of that
key. A failed background refresh keeps serving the last good value instead of
falling through to the origin, and is reported once rather than per read.
- Invalidate across processes. invalidateKey() now clears memory, deletes the
Redis key and publishes a signal, so a value written by one process is no
longer served stale by the others for the rest of its TTL. A failed Redis
delete no longer skips the broadcast.
- Guard against a refresh that started before an invalidation writing its
outdated result back into the cache.
- Read the news revision at most once a second per process instead of on every
call, with a pub/sub signal to drop the local copy when it rotates. A Redis
outage now degrades to the in-process cache rather than to no cache at all.
- Warm the hot public keys on boot, so the first visitors after a deploy do not
each pay for a miss.
- Count hits, misses, stale serves, errors and evictions per key, exposed at
GET /api/admin/devops/cache. Without it a wrong REDIS_URL, a full budget and
a dead origin all look identical from the outside.
- Enforce the imaging cache budget for real: records are .img/.json pairs, so
the old cap counted files and never removed anything while entries were
fresh. Sweeps are throttled per directory and prune to a low-water mark.
- Cap the JWT version map, and stop per-test scratch roots from littering the
runtime imaging cache.
Public read-only endpoints get grace windows; admin, account and auth data
deliberately stays fresh. Redis TTLs get a little jitter so keys written
together no longer expire together.
3209 tests pass. next build could not be verified on this host: the optimized
build is OOM-killed before prerender, so this has not run in a real Next
runtime yet.
- Track referral attribution at registration via ?ref code with
same-IP and duplicate-pair guards
- Add daily login rewards with streak tracking, claim flow and
sendCurrency payout backed by RCON with DB fallback
- Add admin pages for referral settings and the daily reward schedule
- Add migration 0033 with tables, seed schedule, settings and ACL grants
- Add admin.referrals.* and admin.dailyrewards.* permission slugs
- Localize new copy in en, nl and it
Replace raw db.execute tuple casts with queryRows/rowsFrom/execResult/
affectedRows helpers from lib/db, drop redundant mysql2 casts on typed
query builders, and centralize per-test fakeForm into test/fake-form.
Update db mocks in tests so helpers resolve against mocked execute.
The cleanup scan used to read every .nitro bundle in full and decompress
the large PNG texture just to confirm the file is structurally valid. On
directories with hundreds of thousands of bundles this took minutes, the
reverse proxy cut the request at its 30s timeoutable with an HTML 504, and
the panel then crashed with "Unexpected token '<'".
Validate bundles with a cheap header-only read (a few KB, no decompression)
that mirrors parseNitroBundle's byte layout; only files whose header looks
suspicious get the expensive full parse. Robust against downloads that
landed as an HTML error page, truncated or zero-filled files. The scan
drops from minutes to seconds on large nitro directories.
Also guard the panel against non-JSON (proxy error page / HTML) responses
so it reports a clear error message instead of a JSON parse failure.
Scan distinguishes fake, broken, and orphaned SWF/icon assets with age
metadata, deletes per asset kind, re-downloads broken nitro bundles from
configured sources, auto-cleans old fake leftovers, and exports a JSON
manifest. Adds rebuild and auto-clean API endpoints with audit coverage
and a housekeeping preview route under the hotel domain.
Verified: full vitest suite (2213 tests), typecheck, and biome all pass.
Introduce getCachedAdminCount to cache un-filtered table count(*) queries in Redis for admin lists (starting with UsersPage), avoiding heavy full table scans on every request while keeping exact counts for search/filtered queries.
The 5-minute disk probe now reclaims storage automatically: from 85% it runs the gentle age-windowed Docker prune, from 90% it drops the age windows (docker-prune.sh --force: all unused build cache and unreferenced images, all stopped containers) so a mount can never silently max out. Alerts still fire at 85/90/95% and their hint now points at non-Docker growth when reclaiming is not enough. Force mode is reserved for the worker; deploys keep the gentle mode. Volumes are off-limits in every path.
Add a pure df parser (disk-usage.ts) with 85/90/95% threshold classification, a diskPressure() alert (Discord/email/alert_logs, severity escalates with fill), and a 5-minute host-side probe in jobs-worker.ts that raises one alert per crossing mount, cooldown-gated per mount+level. Real mounts only: overlay/tmpfs pseudo filesystems are ignored.
Mirror interactive batch runs into the import-job store so interrupted imports (restart, time-out, disconnect) can be resumed from Import History. Items are checkpointed as they settle (coalesced, serialized saves) and the mirror starts 'running' so the boot-time worker marks it 'interrupted' instead of double-importing; done items are never re-imported. Add bounded backoff retry for transient download/connection failures before marking an item failed, and point the client's time-out/network toasts at Import History.
Raise batch concurrency (furni 3->12, clone 10->12) with a Speed control next to Translate. Skip the SWF download when a .nitro bundle already exists on disk (color variants share the base nitro), and stop flagging that as a failed download.
- Render the table view through a virtualizer too, using a shared grid
template so the sticky header and rows keep perfect column alignment
- Keep semantic table/row/cell elements while virtualizing
- Cap the batch item-details list to the latest 60 rows (newest first)
- Coalesce per-item progress events server-side (120ms throttle) in both
the exact-import and clone SSE batch runners
- Add TTL-based caching for local index lookup, furnidata classnames,
catalog id set, nitro file presence and import stats
- Invalidate caches after furnace single/batch/clone imports
- Rewrite batch progress with elapsed time, rate and verification chips
- Virtualize the grid with @tanstack/react-virtual and replace the
Load more button with infinite scroll via an IntersectionObserver
The Arcturus errors "page hierarchy contains a cycle page 354 and 357" and
"sibling order 1 is used more than once (111 problems)" come from
catalog_pages, not catalog_items: pages 354/357 point at themselves, and
many parents have child pages sharing the same order_num. Extend the
emulator catalog scan + fix to detect both: pages whose parent chain loops
back get detached (parent_id = 0 on the highest cycle member) and every
affected parent's children are renumbered sequentially, preserving their
current relative order.
Add scripts/diag-emulator.ts to inspect the live catalog state.
Block invalid catalog_items writes at the API level (points currency
allowlist, non-negative prices, positive amount, limited stack >= sold
count, unique sibling order numbers) and auto-assign unique order numbers
on bulk create. Add a catalog-maintenance scan + transactional repair that
fixes pre-existing rows: resets unsupported points_type, clamps negative
costs, sets amount to 1, raises limited_stack, renumbers duplicate orders
and deletes offers with missing page/item references. Surface the issue
count and a fix button in the admin maintenance panel.
Also: add enabled/retired flag to clone sources, classify poster and
currency furniture in item-kind, and remove the obsolete update-Nitrov3.sh.
openhands
Simo