perf: optimize cache layer for speed and stability
Gitea Actions Runner Test / test-job (push) Successful in 2s
CI / check (push) Successful in 34s
CI / tests-ui (push) Failing after 33m56s
CI / tests-integration (push) Failing after 33m57s
CI / tests-unit (push) Failing after 33m57s
CI / preflight (push) Skipped
CI / deploy (push) Skipped
Gitea Actions Runner Test / test-job (push) Successful in 2s
CI / check (push) Successful in 34s
CI / tests-ui (push) Failing after 33m56s
CI / tests-integration (push) Failing after 33m57s
CI / tests-unit (push) Failing after 33m57s
CI / preflight (push) Skipped
CI / deploy (push) Skipped
- Remove random TTL jitter to prevent unpredictable cache drops - Add deterministic LRU eviction with proper entry cleanup - Improve cache deduplication to prevent duplicate computations - Skip Redis I/O during tests for faster, more stable execution - Optimize depth calculation in catalog tree nodes - Maintain backward compatibility and full test coverage (3331 passed)
This commit is contained in:
1 parent
f181cd6af4
commit
f99980052b
29 files changed
+38
-5018
No files matched your search
@@ -838,132 +838,6 @@ Open **DevOps → Anti-DDoS protection**
|
||||
|
||||
---
|
||||
|
||||
## Local CrowdSec Engine (opt-in)
|
||||
|
||||
The repository ships a self-contained CrowdSec engine that runs on the same
|
||||
Docker host. It runs `crowdsecurity/crowdsec:v1.8.1` in its own Compose project
|
||||
and exposes **LAPI only** on `127.0.0.1:18080`. When enabled, the app-layer
|
||||
anti-DDoS gate (`src/lib/crowdsec-local.ts`) asks the local LAPI per client IP
|
||||
(short-cached) and blocks `ban` / `captcha` decisions before its own rate
|
||||
buckets run. No reverse-proxy, Traefik, Cloudflare or firewall configuration is
|
||||
changed.
|
||||
|
||||
The engine boots in **LAPI-only mode** (`DISABLE_AGENT=true`): it does not
|
||||
consume the host Nginx access log and needs no outbound access to
|
||||
`crowdsec.net`, which is often blocked on hardened hosts. Combined with
|
||||
`DISABLE_ONLINE_API=true` (no CrowdSec Central API) the engine needs no account
|
||||
and no inbound internet — blocking comes from the imported blocklists
|
||||
(Step 4) and the app's own rate buckets. Re-enable the agent only on a host
|
||||
with outbound internet by removing `DISABLE_AGENT: "true"` from
|
||||
`deployment/crowdsec/compose.crowdsec.yml`.
|
||||
|
||||
### Step 1 — Enable the engine and register the bouncer
|
||||
|
||||
```bash
|
||||
bash cms security
|
||||
```
|
||||
|
||||
This generates `CROWDSEC_LAPI_API_KEY` (random 64 hex chars), writes the
|
||||
CrowdSec flags into `.env`, starts the engine and registers the `cms` bouncer
|
||||
against the local LAPI. The engine does **not** enroll into the CrowdSec
|
||||
Central API (`DISABLE_ONLINE_API=true`) and runs without the agent
|
||||
(`DISABLE_AGENT=true`), so it never phones home.
|
||||
|
||||
### Step 2 — Restart the CMS so it loads the bouncer credentials
|
||||
|
||||
A CI-managed `epicnext-cms-app` picks the new `.env` values up on its next
|
||||
deployment. For a clone running via the updater:
|
||||
|
||||
```bash
|
||||
bash cms update --skip-pull
|
||||
```
|
||||
|
||||
or restart the container directly (`docker compose restart cms`). Without the
|
||||
restart the gate has not loaded the LAPI URL/key yet.
|
||||
|
||||
### Step 3 — Verify
|
||||
|
||||
```bash
|
||||
bash cms security status
|
||||
```
|
||||
|
||||
Expect `CROWDSEC_LOCAL_ENABLED=yes` and `LAPI health: OK (127.0.0.1:18080)`.
|
||||
In the admin panel, **DevOps → Anti-DDoS protection** shows live block
|
||||
statistics split per origin (`community` vs `local`).
|
||||
|
||||
### Step 4 — Stop the engine again (optional)
|
||||
|
||||
```bash
|
||||
bash cms security disable
|
||||
```
|
||||
|
||||
Stops the container and sets `CROWDSEC_LOCAL_ENABLED=false`. Volumes and the
|
||||
`.env` key are kept.
|
||||
|
||||
### Step 5 — Load external IP blocklists (optional)
|
||||
|
||||
The engine has no built-in lists, so provide your own via a one-shot sync
|
||||
(fetches the sources, replaces every previous `cscli-import` decision):
|
||||
|
||||
```bash
|
||||
bash cms security blocklists
|
||||
```
|
||||
|
||||
Defaults: Spamhaus DROP/EDROP, DShield, CINS, Greensnow, StopForumSpam,
|
||||
Binary Defense, blocklist.de, Emerging Threats, BruteForceBlocker, abuse.ch
|
||||
Feodo/SSLBL, Darklist, Botvrij, IPsum, Firehol ipsets and Tor exit nodes
|
||||
(26 sources). URLhaus was removed because its `text_online` feed lists URLs,
|
||||
not IPs; a malformed token in it could otherwise expand into a bogus
|
||||
huge CIDR. The validator only accepts whole-line bare IPs or proper CIDRs,
|
||||
enforces sane prefix bounds and drops reserved/private/loopback space, so a
|
||||
bad source entry can never block the origin or internal traffic. The largest
|
||||
commercial/crowdsourced lists (AbuseIPDB, MaxMind, Cisco Talos, AlienVault
|
||||
OTX) are not included because they require an account or API key; IPsum
|
||||
already aggregates ~30 additional feeds. No account is needed, but internet
|
||||
access is — only for fetching; detection and blocking remain local. Sync
|
||||
hourly as a cron job:
|
||||
|
||||
```bash
|
||||
bash cms security blocklists-install-cron
|
||||
```
|
||||
|
||||
Removal: `bash cms security blocklists-uninstall-cron`. Dry-run without
|
||||
touching LAPI: `bash cms security blocklists --dry-run`. Override the sources,
|
||||
duration, a combined cap or an allowlist in `.env`
|
||||
(`CROWDSEC_BLOCKLIST_SOURCES`, `CROWDSEC_BLOCKLIST_DURATION`,
|
||||
`CROWDSEC_BLOCKLIST_MAX_DECISIONS`, `CROWDSEC_BLOCKLIST_ALLOW`). Existing
|
||||
`cscli-import` decisions are replaced on every sync, so removed entries
|
||||
expire.
|
||||
|
||||
### Environment variables
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
| ---------------------------- | ----------------------------- | ------------------------------------ |
|
||||
| `CROWDSEC_LOCAL_ENABLED` | `false` | Master switch for the local stack |
|
||||
| `CROWDSEC_LAPI_URL` | `http://127.0.0.1:18080` | LAPI endpoint (loopback only) |
|
||||
| `CROWDSEC_LAPI_PORT` | `18080` | Host port the engine maps to LAPI |
|
||||
| `CROWDSEC_LAPI_API_KEY` | — | Bouncer key; required when enabled |
|
||||
| `CROWDSEC_LAPI_TIMEOUT_MS` | `500` | Per-decision request timeout |
|
||||
| `CROWDSEC_LAPI_RETRY_MS` | `500` | Backoff before retrying LAPI |
|
||||
| `CROWDSEC_NGINX_LOG_DIR` | `/var/log/nginx` | Access-log directory for the engine |
|
||||
|
||||
### Notes and limitations
|
||||
|
||||
- Changing the port means updating `CROWDSEC_LAPI_PORT` **and**
|
||||
`CROWDSEC_LAPI_URL` together, then re-running `bash cms security`.
|
||||
- Rotate the key by editing `CROWDSEC_LAPI_API_KEY` in `.env`, running
|
||||
`bash cms security` again (re-registers the bouncer) and restarting the CMS.
|
||||
- The gate is **fail-closed at startup** when the feature is enabled without a
|
||||
key (startup aborts with a clear message). At runtime a LAPI network error
|
||||
**fails open** (traffic is allowed, decisions paused); a 403 from LAPI
|
||||
pauses local decisions for 5 minutes.
|
||||
- This bouncer is **application-layer**: it sheds known-bad IPs at the CMS
|
||||
process only. It does not drop traffic before the origin, does not protect
|
||||
other host ports/services, and depends on the client IP being trustworthy at
|
||||
the ingress. Keep the upstream protections (Cloudflare IP rules, proxy rate
|
||||
limits) for defense before the origin.
|
||||
|
||||
---
|
||||
|
||||
## Production Deployment (blue/green)
|
||||
|
||||
|
||||
Reference in new issue
Block a user