feat: replace Redis with DragonflyDB
- Replace Redis server with DragonflyDB v1.40.1 (Redis protocol compatible) - Stop redis-server service, enable dragonfly service on 127.0.0.1:6379 - Configure dragonfly in /etc/dragonfly/dragonfly.conf (bind 127.0.0.1, maxmemory 2gb) - Update .env: remove REDIS_URL reference Improve database reliability: - Fix catalog-tree.ts: remove CAST(page_id AS CHAR) to enable index usage (122 rows vs 78k full scan) - Fix catalog-repair.ts: replace sql.raw() string interpolation with parameterized sql queries using quoteIdentifier() - Improve redis retry resilience: change retryStrategy to not give up after 3 attempts, enabling automatic reconnect after server restart Update documentation: - Update README: replace Redis references with DragonflyDB, add DragonflyDB setup section, update performance features list, update architecture diagram - biome and typecheck pass clean
This commit is contained in:
1 parent
9463c8d4da
commit
65e3915a5f
4 files changed
+89
-28
No files matched your search
@@ -13,7 +13,7 @@ Features a premium animated homepage (typewriter hero, floating orbs, scroll cou
|
|||||||
| Node.js | >= 22 | Required by Next.js 16 |
|
| Node.js | >= 22 | Required by Next.js 16 |
|
||||||
| pnpm | >= 10.33.4 | Package manager (npm/yarn not supported) |
|
| pnpm | >= 10.33.4 | Package manager (npm/yarn not supported) |
|
||||||
| MySQL / MariaDB | 8.0+ / 10.6+ | Shared with the emulator |
|
| MySQL / MariaDB | 8.0+ / 10.6+ | Shared with the emulator |
|
||||||
| Redis | 7.x+ | Optional — caching, rate limiting, SSE |
|
| DragonflyDB | 1.x+ | Optional — caching, rate limiting, SSE (Redis-protocol compatible) |
|
||||||
| Java | 17+ | Required only if building the emulator |
|
| Java | 17+ | Required only if building the emulator |
|
||||||
| Maven | 3.9+ | Required only if building the emulator |
|
| Maven | 3.9+ | Required only if building the emulator |
|
||||||
|
|
||||||
@@ -56,7 +56,7 @@ HOTEL_NAME=YourHotel
|
|||||||
APP_URL=http://localhost:3000
|
APP_URL=http://localhost:3000
|
||||||
```
|
```
|
||||||
|
|
||||||
See `.env.example` for all optional variables (RCON, email, Redis, OAuth, PayPal, etc.).
|
See `.env.example` for all optional variables (RCON, email, DragonflyDB, OAuth, PayPal, etc.).
|
||||||
|
|
||||||
### 4. ORM Setup & Type Generation
|
### 4. ORM Setup & Type Generation
|
||||||
|
|
||||||
@@ -121,6 +121,60 @@ Open `http://localhost:3000` in your browser.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## DragonflyDB (caching, rate limiting, SSE)
|
||||||
|
|
||||||
|
DragonflyDB is a drop-in, Redis-compatible in-memory datastore. The CMS connects to it
|
||||||
|
via `REDIS_URL` using the `ioredis` client, so no application code changes are needed —
|
||||||
|
every Redis command (`PING`, `GET`, `SETEX`, `DEL`, `INCR`, `PEXPIRE`, `PTTL`) works
|
||||||
|
unchanged. It is optional: without it the CMS falls back to in-process memory.
|
||||||
|
|
||||||
|
### Install (Ubuntu/Debian)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsSL -o /tmp/dragonfly_amd64.deb \
|
||||||
|
https://github.com/dragonflydb/dragonfly/releases/download/v1.40.1/dragonfly_amd64.deb
|
||||||
|
apt-get install -y /tmp/dragonfly_amd64.deb
|
||||||
|
systemctl enable --now dragonfly
|
||||||
|
```
|
||||||
|
|
||||||
|
This installs a `dragonfly` systemd service and a config file at `/etc/dragonfly/dragonfly.conf`.
|
||||||
|
|
||||||
|
### Configure
|
||||||
|
|
||||||
|
```ini
|
||||||
|
--bind=127.0.0.1
|
||||||
|
--port=6379
|
||||||
|
--maxmemory=2gb
|
||||||
|
--version_check=false
|
||||||
|
```
|
||||||
|
|
||||||
|
- `--bind=127.0.0.1` keeps it private on the machine (matches `REDIS_URL=redis://127.0.0.1:6379`).
|
||||||
|
- `--port=6379` is the default Redis port, so `.env` stays unchanged.
|
||||||
|
- `--maxmemory` must be at least `0.25GiB` per CPU thread (e.g. `2gb` on a 6-thread server).
|
||||||
|
- `--version_check=false` disables the periodic outbound update check.
|
||||||
|
- Snapshots are written to `--dir` (`/var/lib/dragonfly/dump-*.dfs`).
|
||||||
|
|
||||||
|
### Point the CMS at it
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
REDIS_URL=redis://127.0.0.1:6379?connect_timeout=2
|
||||||
|
```
|
||||||
|
|
||||||
|
### Useful commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
redis-cli PING # → PONG
|
||||||
|
redis-cli FLUSHALL # clear the cache
|
||||||
|
systemctl status dragonfly # service health
|
||||||
|
```
|
||||||
|
|
||||||
|
Verify everything is wired up via the health endpoint:
|
||||||
|
`/api/health` should report `"redis": true`. The ioredis client automatically reconnects
|
||||||
|
after a DragonflyDB restart.
|
||||||
|
|
||||||
|
> **Note:** if an old Redis install still occupies port 6379, stop it first:
|
||||||
|
> `systemctl disable --now redis-server`.
|
||||||
|
|
||||||
## Nginx Configuration
|
## Nginx Configuration
|
||||||
|
|
||||||
The CMS is designed to run behind an nginx reverse proxy. Below is a reference configuration covering SSL termination, WebSocket upgrade, proxy caching, and the Habbo imager integration.
|
The CMS is designed to run behind an nginx reverse proxy. Below is a reference configuration covering SSL termination, WebSocket upgrade, proxy caching, and the Habbo imager integration.
|
||||||
@@ -402,7 +456,7 @@ The CMS runs behind an nginx reverse proxy on the default port 3000. Static asse
|
|||||||
| **View Transitions API** | Native browser transitions between page navigations |
|
| **View Transitions API** | Native browser transitions between page navigations |
|
||||||
| **Lenis Smooth Scroll** | Fluid, customizable scrolling (respects `prefers-reduced-motion`) |
|
| **Lenis Smooth Scroll** | Fluid, customizable scrolling (respects `prefers-reduced-motion`) |
|
||||||
| **Server-Sent Events** | Real-time radio now-playing & listeners via SSE (no polling) |
|
| **Server-Sent Events** | Real-time radio now-playing & listeners via SSE (no polling) |
|
||||||
| **Redis Caching** | Caches API responses (home, radio config) up to 30s in Redis |
|
| **DragonflyDB Caching** | Caches API responses (home, radio config) up to 30s in DragonflyDB |
|
||||||
| **Bundle Analyzer** | Run `pnpm analyze` to visualize and optimize bundle sizes |
|
| **Bundle Analyzer** | Run `pnpm analyze` to visualize and optimize bundle sizes |
|
||||||
| **RCON (TCP Socket)** | Live commands to the emulator (credits, badges, kick, ban) |
|
| **RCON (TCP Socket)** | Live commands to the emulator (credits, badges, kick, ban) |
|
||||||
| **Streaming & Suspense** | Next.js App Router streaming for fast page loads |
|
| **Streaming & Suspense** | Next.js App Router streaming for fast page loads |
|
||||||
@@ -437,9 +491,9 @@ The CMS runs behind an nginx reverse proxy on the default port 3000. Static asse
|
|||||||
│ │ ├── auth/ # NextAuth, password hashing, 2FA, SSO tickets
|
│ │ ├── auth/ # NextAuth, password hashing, 2FA, SSO tickets
|
||||||
│ │ ├── services/ # RCON, email, currency, PayPal, alerts
|
│ │ ├── services/ # RCON, email, currency, PayPal, alerts
|
||||||
│ │ ├── db.ts # Drizzle connection singleton (runtime)
|
│ │ ├── db.ts # Drizzle connection singleton (runtime)
|
||||||
│ │ ├── cached-db.ts # Redis-backed query cache helpers
|
│ │ ├── cached-db.ts # DragonflyDB-backed query cache helpers
|
||||||
│ │ ├── redis.ts # Redis client (ioredis)
|
│ │ ├── redis.ts # Cache client (ioredis → DragonflyDB)
|
||||||
│ │ ├── redis-cache.ts # Redis caching utility for API routes
|
│ │ ├── redis-cache.ts # Caching utility for API routes (uses DragonflyDB)
|
||||||
│ │ ├── cache.ts # In-memory cache fallback
|
│ │ ├── cache.ts # In-memory cache fallback
|
||||||
│ │ ├── motion.ts # Framer Motion animation variants
|
│ │ ├── motion.ts # Framer Motion animation variants
|
||||||
│ │ └── use-event-source.ts # React hook for SSE subscriptions
|
│ │ └── use-event-source.ts # React hook for SSE subscriptions
|
||||||
|
|||||||
+1
-2
@@ -24,8 +24,7 @@ function createRedis(): Redis | null {
|
|||||||
const client = new Redis(url, {
|
const client = new Redis(url, {
|
||||||
maxRetriesPerRequest: 3,
|
maxRetriesPerRequest: 3,
|
||||||
retryStrategy(times) {
|
retryStrategy(times) {
|
||||||
if (times > 3) return null;
|
return Math.min(times * 200, 5000);
|
||||||
return Math.min(times * 200, 2000);
|
|
||||||
},
|
},
|
||||||
lazyConnect: true,
|
lazyConnect: true,
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -30,6 +30,13 @@ function escSqlLiteral(value: string): string {
|
|||||||
return value.replace(/'/g, "''");
|
return value.replace(/'/g, "''");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const VALID_IDENTIFIER = /^[a-zA-Z0-9_]+$/;
|
||||||
|
|
||||||
|
function quoteIdentifier(identifier: string): string {
|
||||||
|
if (!VALID_IDENTIFIER.test(identifier)) throw new Error("Invalid identifier");
|
||||||
|
return `\`${identifier}\``;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Mapping of event prefix substrings (lowercase, no leading underscore) to
|
* Mapping of event prefix substrings (lowercase, no leading underscore) to
|
||||||
* their human-readable English page labels. Covers every seasonal/holiday/event
|
* their human-readable English page labels. Covers every seasonal/holiday/event
|
||||||
@@ -524,7 +531,7 @@ export async function repairDuplicateClassnames(): Promise<DuplicateClassnameRes
|
|||||||
];
|
];
|
||||||
for (const { table, idCol } of listTables) {
|
for (const { table, idCol } of listTables) {
|
||||||
const [rows] = (await db.execute(
|
const [rows] = (await db.execute(
|
||||||
sql.raw(`SELECT ${idCol} AS pk, item_ids AS v FROM ${table}`),
|
sql`SELECT ${sql.raw(quoteIdentifier(idCol))} AS pk, item_ids AS v FROM ${sql.raw(quoteIdentifier(table))}`,
|
||||||
)) as unknown as [Array<{ pk: number; v: string | null }>, unknown];
|
)) as unknown as [Array<{ pk: number; v: string | null }>, unknown];
|
||||||
for (const row of rows) {
|
for (const row of rows) {
|
||||||
if (!row.v) continue;
|
if (!row.v) continue;
|
||||||
@@ -545,9 +552,7 @@ export async function repairDuplicateClassnames(): Promise<DuplicateClassnameRes
|
|||||||
}
|
}
|
||||||
if (!changed) continue;
|
if (!changed) continue;
|
||||||
await db.execute(
|
await db.execute(
|
||||||
sql.raw(
|
sql`UPDATE ${sql.raw(quoteIdentifier(table))} SET item_ids = ${out.join(";")} WHERE ${sql.raw(quoteIdentifier(idCol))} = ${row.pk}`,
|
||||||
`UPDATE ${table} SET item_ids = '${out.join(";")}' WHERE ${idCol} = ${row.pk}`,
|
|
||||||
),
|
|
||||||
);
|
);
|
||||||
remapped++;
|
remapped++;
|
||||||
}
|
}
|
||||||
@@ -587,16 +592,16 @@ export async function repairDuplicateClassnames(): Promise<DuplicateClassnameRes
|
|||||||
for (const table of singleTables) {
|
for (const table of singleTables) {
|
||||||
for (let i = 0; i < entries.length; i += CHUNK) {
|
for (let i = 0; i < entries.length; i += CHUNK) {
|
||||||
const chunk = entries.slice(i, i + CHUNK);
|
const chunk = entries.slice(i, i + CHUNK);
|
||||||
const cases = chunk
|
const cases = chunk.map(
|
||||||
.map(([dup, canonical]) => `WHEN ${dup} THEN ${canonical}`)
|
([dup, canonical]) => sql`WHEN ${dup} THEN ${canonical}`,
|
||||||
.join(" ");
|
);
|
||||||
const dupList = chunk.map(([dup]) => `${dup}`).join(",");
|
const dupList = chunk.map(([dup]) => dup);
|
||||||
try {
|
try {
|
||||||
const [result] = (await db.execute(
|
const [result] = (await db.execute(sql`
|
||||||
sql.raw(
|
UPDATE ${sql.raw(quoteIdentifier(table))}
|
||||||
`UPDATE ${table} SET item_id = CASE item_id ${cases} ELSE item_id END WHERE item_id IN (${dupList})`,
|
SET item_id = CASE item_id ${sql.join(cases, sql` `)} ELSE item_id END
|
||||||
),
|
WHERE item_id IN (${sql.join(dupList, sql`, `)})
|
||||||
)) as unknown as [Record<string, unknown>, unknown];
|
`)) as unknown as [Record<string, unknown>, unknown];
|
||||||
remapped += Number(result.affectedRows ?? 0);
|
remapped += Number(result.affectedRows ?? 0);
|
||||||
} catch {
|
} catch {
|
||||||
// Table may not exist on some hotel schemas — skip it.
|
// Table may not exist on some hotel schemas — skip it.
|
||||||
@@ -605,9 +610,9 @@ export async function repairDuplicateClassnames(): Promise<DuplicateClassnameRes
|
|||||||
}
|
}
|
||||||
|
|
||||||
const dupIds = entries.map(([dup]) => dup);
|
const dupIds = entries.map(([dup]) => dup);
|
||||||
await db.execute(
|
await db.execute(sql`
|
||||||
sql.raw(`DELETE FROM items_base WHERE id IN (${dupIds.join(",")})`),
|
DELETE FROM items_base WHERE id IN (${sql.join(dupIds, sql`, `)})
|
||||||
);
|
`);
|
||||||
|
|
||||||
return { merged, rowsRemoved: dupIds.length, remapped };
|
return { merged, rowsRemoved: dupIds.length, remapped };
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,7 +12,9 @@ function toInt(value: unknown, fallback = 0): number {
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Count catalog_items per page via raw SQL.
|
* Count catalog_items per page via raw SQL.
|
||||||
* Real Habbo DBs often store page_id as VARCHAR; numeric groupBy fails or returns 0.
|
* Real Habbo DBs often store page_id as VARCHAR. String literals match both
|
||||||
|
* INT and VARCHAR columns while keeping the page_id index usable (CAST(... AS CHAR)
|
||||||
|
* would force a full index scan).
|
||||||
*/
|
*/
|
||||||
export async function getCatalogItemCounts(
|
export async function getCatalogItemCounts(
|
||||||
pageIds?: number[],
|
pageIds?: number[],
|
||||||
@@ -26,7 +28,7 @@ export async function getCatalogItemCounts(
|
|||||||
const [rows] = (await db.execute(sql`
|
const [rows] = (await db.execute(sql`
|
||||||
SELECT page_id, COUNT(*) as cnt
|
SELECT page_id, COUNT(*) as cnt
|
||||||
FROM catalog_items
|
FROM catalog_items
|
||||||
WHERE CAST(page_id AS CHAR) IN (${sql.join(idStrs, sql`, `)})
|
WHERE page_id IN (${sql.join(idStrs, sql`, `)})
|
||||||
GROUP BY page_id
|
GROUP BY page_id
|
||||||
`)) as unknown as [
|
`)) as unknown as [
|
||||||
{ page_id: string | number; cnt: number | bigint }[],
|
{ page_id: string | number; cnt: number | bigint }[],
|
||||||
@@ -216,14 +218,15 @@ export async function movePage(
|
|||||||
|
|
||||||
/**
|
/**
|
||||||
* Delete catalog_items for the given page ids.
|
* Delete catalog_items for the given page ids.
|
||||||
* Habbo DBs often store page_id as VARCHAR; typed Int deletes can miss rows.
|
* Habbo DBs often store page_id as VARCHAR; text delete on INT or VARCHAR
|
||||||
|
* columns keeps the page_id index usable (CAST would force a full scan).
|
||||||
*/
|
*/
|
||||||
async function deleteCatalogItemsByPageIds(pageIds: number[]): Promise<void> {
|
async function deleteCatalogItemsByPageIds(pageIds: number[]): Promise<void> {
|
||||||
if (pageIds.length === 0) return;
|
if (pageIds.length === 0) return;
|
||||||
const idStrs = pageIds.map(String);
|
const idStrs = pageIds.map(String);
|
||||||
await db.execute(sql`
|
await db.execute(sql`
|
||||||
DELETE FROM catalog_items
|
DELETE FROM catalog_items
|
||||||
WHERE CAST(page_id AS CHAR) IN (${sql.join(idStrs, sql`, `)})
|
WHERE page_id IN (${sql.join(idStrs, sql`, `)})
|
||||||
`);
|
`);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user