fix(ops): run heavy commands under a hard memory cap to stop host OOM kills
Gitea Actions Runner Test / test-job (push) Successful in 1s
CI / check (push) Successful in 35s
CI / tests-integration (push) Successful in 2m5s
CI / tests-unit (push) Successful in 2m30s
CI / tests-ui (push) Successful in 3m3s
CI / preflight (push) Skipped
CI / deploy (push) Failing after 44s

The host runs with vm.overcommit_memory=0 and no swap, so a process that
grows past free memory makes the kernel OOM-kill across the whole machine
-- the Turbopack build (commit 3d828a61) could take out the database,
nginx or the live release.

Add scripts/with-memory-cap.sh: it moves a command into its own systemd
scope with MemoryMax, so only that cgroup gets OOM-killed (verified: a
Turbopack build died at its 6GB cap, host untouched). Build/analyze/dev/
test*/typecheck now run under explicit caps; ulimit -v is only an explicit
opt-in because it bounds virtual address space per process and 10g/20g both
break V8-based builds. Docker and GitLab builds run in their own isolated
containers with a read-only cgroupfs and opt out explicitly (webpack +
--max-old-space-size stay their bound).

Measured: webpack build peaks ~6.5GB RSS, so 10GB leaves headroom within
the 23.5GB host.
This commit is contained in:
openhands committed 2026-10-07 20:17:00 +02:00
1 parent 3265c149da
commit 0845f80768
8 files changed
+563 -393

No files matched your search

+7
View File
@@ -1,5 +1,12 @@
image: node:26 image: node:26
# Runner containers have no systemd, so scripts/with-memory-cap.sh cannot
# enforce an RSS cap there and correctly refuses to run unbounded. These
# builds are already isolated inside their own runner container (not the
# host) and use the webpack builder; opt out explicitly on purpose.
variables:
CMS_MEMORY_CAP_BACKEND: "none"
stages: stages:
- test - test
- build - build
+10 -1
View File
@@ -6,7 +6,7 @@ ENV NEXT_TELEMETRY_DISABLED=1
# Installeer git en pnpm v12 # Installeer git en pnpm v12
RUN --mount=type=cache,target=/var/cache/apk \ RUN --mount=type=cache,target=/var/cache/apk \
apk add --no-cache git \ apk add --no-cache git \
&& npm install -g pnpm@12.8.1 && npm install -g pnpm@12.10.1
# Stel het PATH zo in dat Alpine pnpm gegarandeerd overal herkent # Stel het PATH zo in dat Alpine pnpm gegarandeerd overal herkent
ENV PNPM_HOME="/usr/local/share/pnpm" ENV PNPM_HOME="/usr/local/share/pnpm"
@@ -31,7 +31,16 @@ ENV NEXT_DEPLOYMENT_ID="$NEXT_DEPLOYMENT_ID"
# this 329-route app and gets OOM-killed; webpack peaks around 5GB. A # this 329-route app and gets OOM-killed; webpack peaks around 5GB. A
# --max-old-space-size cap does NOT help, because that memory is native # --max-old-space-size cap does NOT help, because that memory is native
# Turbopack memory rather than the V8 heap. # Turbopack memory rather than the V8 heap.
#
# `pnpm run build` goes through scripts/with-memory-cap.sh. BuildKit's build
# container has /sys/fs/cgroup mounted read-only (no cgroup MemoryMax) and
# `ulimit -v` breaks V8-based builds (see the script header), so this stage
# explicitly opts out of the cap. The real bound here is the webpack builder
# + the V8 heap cap above, and the build runs isolated in its own container,
# not on the host; the host itself is protected by the same wrapper through
# systemd.
ENV NODE_OPTIONS="--max-old-space-size=4096" ENV NODE_OPTIONS="--max-old-space-size=4096"
ENV CMS_MEMORY_CAP_BACKEND=none
# Bouw de Next.js applicatie met caching # Bouw de Next.js applicatie met caching
RUN --mount=type=cache,target=/app/.next/cache \ RUN --mount=type=cache,target=/app/.next/cache \
+2 -2
View File
@@ -969,7 +969,7 @@ branch guard inside `ci-deploy.sh` only accepts `main`/`master`.
| `pnpm db:bulk` | Batch-import >50 MB JSON via `scripts/bulk-import-json.ts` | | `pnpm db:bulk` | Batch-import >50 MB JSON via `scripts/bulk-import-json.ts` |
| `pnpm db:up` / `pnpm db:down` | Start / stop the `mariadb-turbo` container | | `pnpm db:up` / `pnpm db:down` | Start / stop the `mariadb-turbo` container |
| `pnpm gamedata:compress` | Pre-compress large gamedata JSON to `.gz` (gzip_static) | | `pnpm gamedata:compress` | Pre-compress large gamedata JSON to `.gz` (gzip_static) |
| `pnpm analyze` | Build + open bundle analyzer | | `pnpm analyze` | Build + report per-route bundle sizes |
| `pnpm jobs:worker` | Start background task worker | | `pnpm jobs:worker` | Start background task worker |
| `pnpm biome:check` | Lint and format code | | `pnpm biome:check` | Lint and format code |
@@ -1098,7 +1098,7 @@ The CMS automatically translates furniture names and descriptions to **13 langua
pnpm dev # Start with hot reload pnpm dev # Start with hot reload
pnpm typecheck # Type check all files pnpm typecheck # Type check all files
pnpm test # Run test suite pnpm test # Run test suite
pnpm analyze # Build and analyze bundle sizes pnpm analyze # Build and report per-route bundle sizes
pnpm biome:check # Lint and format pnpm biome:check # Lint and format
``` ```
+25 -25
View File
File diff suppressed because it is too large. Load diff
+351 -363
View File
File diff suppressed because it is too large. Load diff
+166
View File
File diff suppressed because it is too large. Load diff
@@ -266,7 +266,7 @@ function scanModuleAccesses(source: string): ModuleAccessScan {
} else if (value.type === "ImportExpression") { } else if (value.type === "ImportExpression") {
recordArgument(value.source, "import"); recordArgument(value.source, "import");
} else if (value.type === "TSImportType") { } else if (value.type === "TSImportType") {
recordArgument(value.argument, "import"); recordArgument(value.source ?? value.argument, "import");
} else if ( } else if (
value.type === "CallExpression" || value.type === "CallExpression" ||
value.type === "OptionalCallExpression" value.type === "OptionalCallExpression"
@@ -255,7 +255,7 @@ function scanModuleAccesses(source: string): ModuleAccessScan {
} else if (value.type === "ImportExpression") { } else if (value.type === "ImportExpression") {
recordArgument(value.source, "import"); recordArgument(value.source, "import");
} else if (value.type === "TSImportType") { } else if (value.type === "TSImportType") {
recordArgument(value.argument, "import"); recordArgument(value.source ?? value.argument, "import");
} else if ( } else if (
value.type === "CallExpression" || value.type === "CallExpression" ||
value.type === "OptionalCallExpression" value.type === "OptionalCallExpression"