Three separate things that were each costing more than they needed to on the
hot path.
- Single-flight avatar renders. The disk cache was checked first and a miss
went straight to the upstream, with nothing shared between callers, so a page
requesting dozens of avatars at once turned N concurrent requests for one
figure into N renders. A render is the most expensive operation this app
does, and the duplication happened exactly when the cache had nothing to
offer. Eight concurrent requests now cause one render instead of eight. The
map lives on globalThis because Next can evaluate the module more than once
per process, and two copies would each start their own render.
- Let public read-only routes be cached by a shared cache. Every JSON response
was `cache-control: no-store`, so a CDN in front of the app could not answer
any of it and every request reached the origin. publicCacheControl() opts a
route in with s-maxage and stale-while-revalidate, using the same TTL as the
server-side cache so the two layers cannot disagree. The default stays
no-store: most routes here are personalised, admin-only or auth-dependent.
/api/badges/leaderboard is deliberately left alone because it returns
per-viewer rank entries to signed-in callers.
Note this only takes effect once a cache rule exists for /api/* at the CDN, or
the explicit `cache: "no-store"` is dropped from the client fetches (24 files
do that today, including the /api/online poll). The headers alone are inert
until one of those happens.
- Take the homepage row counts from the storage engine estimate instead of
COUNT(*), which walks an index and gets slower as the tables grow. A missing
or zero estimate falls back to the exact count rather than ever showing a
wrong zero. The online count stays exact: it is an indexed read over a small
subset and a few seconds of drift reads as broken rather than approximate.
The counters move into one module because the homepage and the boot warm-up
populate the same cache keys, so two implementations would race to write
different values into the same entry.
3223 tests pass.
The in-process cache was a FIFO of 500 entries that was never touched on a
read, so a key polled on every request could be evicted by an unrelated burst
of dynamic keys. That looked exactly like the cache being cleared at random,
and it is what made the site fall back to the database unpredictably.
- Evict least-recently-used instead, and raise the default budget to 2000
(CACHE_MEMORY_MAX_ENTRIES). Reading a key now marks it as used, so a hot key
only leaves when a hotter one takes its place.
- Add opt-in stale-while-revalidate (CachedOptions.staleMs). The grace window
lives on the entry, so one call site opting in protects every reader of that
key. A failed background refresh keeps serving the last good value instead of
falling through to the origin, and is reported once rather than per read.
- Invalidate across processes. invalidateKey() now clears memory, deletes the
Redis key and publishes a signal, so a value written by one process is no
longer served stale by the others for the rest of its TTL. A failed Redis
delete no longer skips the broadcast.
- Guard against a refresh that started before an invalidation writing its
outdated result back into the cache.
- Read the news revision at most once a second per process instead of on every
call, with a pub/sub signal to drop the local copy when it rotates. A Redis
outage now degrades to the in-process cache rather than to no cache at all.
- Warm the hot public keys on boot, so the first visitors after a deploy do not
each pay for a miss.
- Count hits, misses, stale serves, errors and evictions per key, exposed at
GET /api/admin/devops/cache. Without it a wrong REDIS_URL, a full budget and
a dead origin all look identical from the outside.
- Enforce the imaging cache budget for real: records are .img/.json pairs, so
the old cap counted files and never removed anything while entries were
fresh. Sweeps are throttled per directory and prune to a low-water mark.
- Cap the JWT version map, and stop per-test scratch roots from littering the
runtime imaging cache.
Public read-only endpoints get grace windows; admin, account and auth data
deliberately stays fresh. Redis TTLs get a little jitter so keys written
together no longer expire together.
3209 tests pass. next build could not be verified on this host: the optimized
build is OOM-killed before prerender, so this has not run in a real Next
runtime yet.
Add rateLimit protection to /api/paypal/create, /api/paypal/capture, /api/tokens, /api/radio/shouts, and /api/articles/[slug]/comment to prevent abuse and spamming.
- Cache read-heavy public API routes via redisCache (leaderboard, values,
shop, articles, photos, guilds, teams, staff, users, home, radio, badges)
- Add single-flight and bounded-memory cache layer with unit tests
- Parallelize independent DB queries on search, rares, shop, staff, polls
and profile pages
- Push radio points leaderboard aggregation to SQL with a LIMIT
- Split studio-client and import-furni-client into focused modules
- Clean up next.config.ts
Database:
- Add missing indexes (users.credits, users_currency(type,amount),
users_settings.respects_received, camera_web.timestamp,
messenger_offline.user_id) via migrations 0020/0021
- Use partial .select() everywhere instead of SELECT * (tickets, users,
rooms, audit logs, catalog tree, polls, radio, password reset)
- Add queryPrepared/queryPreparedOne (server-side prepared statements)
and switch the login check to a prepared statement; drop dead
cache options from the pool config
- Raise total_users/total_rooms COUNT(*) cache TTL to 5m
Caching:
- Consolidate the three cache helpers (cached, redisCache, cachedQuery)
into a single memory-first implementation backed by Redis
- invalidateKey now clears the in-process cache as well as Redis
- Cache homepage sections, news list, and leaderboard tabs; share one
news_list cache key between homepage and news archive
- siteSettings: in-process cache with TTL so repeated getters no longer
pay a Redis round-trip per call
- Share a 10s poll cache across all radio SSE connections
- Normalize timestamps after cache reads (Redis JSON round-trip)
Assets:
- Enable AVIF/WebP via images.formats and remove unoptimized from news
covers and the homepage hero (149KB jpg) with proper sizes/priority
- Support ?format=webp|avif|png in the /imaging proxy via sharp
Other:
- Fix pnpm supply-chain minimumReleaseAge failures by excluding the
freshly-published packages (next 16.3.1, hookform resolvers 5.8.0,
resend 6.20.0)
- Remove unused before/after fields from housekeeping AuditEntry
- Clone import: defer FurnitureData.json writes and append all entries in a
single batched write instead of one read-modify-write per item, removing
the main serialization bottleneck for large batches.
- Clone import: raise SSE batch concurrency cap from 5 to 10 and bump the
clone client/route default from 2 to 6.
- Add a flush hook to runSseBatch so callers can batch deferred work before
batch_complete is emitted, and surface flush errors as an error event.
- Enable Next.js Cache Components (instant: false opt-out) and silence the
related build warnings in next.config.ts.
- Switch isomorphic-dompurify to dompurify and refresh dependencies.
- Remove production DB dump (db_backup_*.sql) and update.log from git tracking
- Add DB backups to .gitignore
- Replace all console.log/console.error with structured logger module
- Translate Dutch error messages to English (link-discord.ts)
- Remove dead code blocks (register-form.tsx false && pattern)
- Add ESLint flat config with TypeScript, React, Next.js, jsx-a11y, and security plugins
- Add Prettier config
- Add eslint-plugin-security for security-aware linting
- Fix all 119+ ESLint warnings across the codebase:
- Resolve security/detect-object-injection with safe access patterns
- Resolve security/detect-non-literal-fs-filename with path traversal validation
- Replace <img> with next/image <Image> component
- Remove unused variables and imports
- Replace non-null assertions with proper type guards
- Replace <a> with <Link> for internal navigation
- Use next/script Script component for external scripts
- Fix setState-in-useEffect anti-patterns (navbar-color-picker, logo-generator, theme-switcher)
- Add lint and format scripts to package.json
All checks: typecheck ✓, tests 58/58 ✓, lint 0 errors 0 warnings ✓
- Rich profile (/u/[username]): wallet (credits/duckets/diamonds), friends
grid (messenger_friendships), and owned rooms sections.
- Login history: new website_login_logs table (model + migration 0007),
recorded on every successful sign-in (ip + user-agent), surfaced on a new
/settings/sessions page (with failed-attempt list from failed_logins).
- Photos lightbox + home article slider (client components, no Swiper dep).
- /client/flash launcher (SSO ticket like the Nitro page).
- Admin: private chatlogs section in /admin/logs, /admin/radio/moderation
(shout moderation), a "users by rank" inline bar chart on the dashboard,
and a TinyMCE rich-text editor on the article admin forms.
- Niche API: /api/values/[id], /api/guilds(+/[id]), /api/radio/auto-play.
Verified live (prod, amx_test): login recorded → /settings/sessions shows
it with device; profile renders wallet/friends/rooms; dashboard chart +
private-chat logs + /client/flash + /api/guilds all OK. Reverted test data.
tsc 0, vitest 49/49, next build 0.
Final parity push (web-tier only):
- REST API write + token auth: POST /api/tokens (issue a personal_access_token
for the session user), Bearer auth via src/lib/api-auth.ts, POST
/api/articles/[slug]/comment, GET/DELETE /api/me/tokens, full tickets API
(/api/tickets +[id] +[id]/reply), radio current-dj/points/points-leaderboard/
embed-config + POST shouts, and a real-time /api/radio/stream (SSE). 31 public
API routes total.
- Pages: /draw-badge (buy a custom profile badge → credits + RCON), /me
dashboard (stats + online friends + referral claim). Wired into the nav.
- HTML sanitisation (sanitize-html) — the HTMLPurifier equivalent — applied to
writeable boxes + article bodies before dangerouslySetInnerHTML.
- "Dusk" dark theme preset + a default-dark site option honoured by the
no-flash boot script.
Verified live (prod, amx_test): token issue → Bearer endpoint 200, no-token
401; /api/me/tokens lists it; current-dj/leaderboard JSON; /me + /draw-badge
200; reverted the test user + tokens. tsc 0, vitest 49/49, next build 0.
openhands
Simo
