fix(ops): stop a compose replica from blocking the blue/green release
Gitea Actions Runner Test / test-job (push) Successful in 1s
CI / check (push) Successful in 30s
CI / tests-integration (push) Successful in 1m57s
CI / tests-unit (push) Successful in 2m3s
CI / tests-ui (push) Successful in 2m49s
CI / preflight (push) Skipped
CI / deploy (push) Successful in 2m48s

The deploy failed after the build, the migrations and the browser gate:
"Port 3002 is already in use". The holder was `epicnext-cms`, a compose
replica of release 6bffc537 that the daily scripts/docker-update.sh cron
had recreated at 03:30 with restart=unless-stopped. nginx serves the green
slot on 3003, so that replica was squatting the blue slot the next
candidate needed, and live traffic never noticed.

It got there because the updater's CI-ownership guard only tested
epicnext-cms-app. After a cutover to the green slot that container is
stopped, renamed and deleted, so the guard stopped firing while the host
stayed CI-managed.

- scripts/docker-update.sh: refuse a compose deployment on a CI host by
  checking both slot containers and the nginx upstream, which is the only
  thing that still marks the host as blue/green while a slot is idle.
- scripts/ci-deploy.sh: retire a compose replica of this checkout from
  the candidate port before starting the candidate, so a stray replica
  can never block a release again. Never a slot container, never the port
  nginx serves; anything else still fails loudly in assert_port_free.
- Tests cover both directions: a squatting replica is removed and the
  release lands, a replica on the live port is left alone.
This commit is contained in:
openhands committed 2026-10-05 21:13:49 +02:00
1 parent 11ad6d4376
commit 5b2eb91c5c
7 files changed
+186 -9

No files matched your search

+18
View File
@@ -875,6 +875,24 @@ that the CI deploy path deliberately uses `docker run` rather than compose, so
the resource limits are declared in **both** places — limits that only existed
in compose would never apply to a real release.
### Compose on a CI host
Compose and CI both want port 3002, so only one of them can own a host. A stray
compose replica (`docker compose up`, or the daily `scripts/docker-update.sh`
cron) parked an `epicnext-cms` container on the blue slot while nginx served the
green slot, and every later release stopped on "Port 3002 is already in use" —
after the build, the migrations and the browser gate. Two guards now prevent
that:
- `scripts/docker-update.sh` refuses to run on a CI host. It used to test only
`epicnext-cms-app`, but after a cutover to the green slot that container is
stopped and deleted, so the guard stopped firing while the host stayed
CI-managed. It now checks both slot containers and the nginx upstream.
- `ci-deploy.sh` retires a compose replica of *this* checkout
(`com.docker.compose.project.config_files`) from the candidate port before
starting the candidate — but never a slot container, and never the port nginx
currently serves. Anything else still fails loudly in `assert_port_free`.
### The nginx upstream is the switch
nginx does not know about container names; it reads a plain list of backends from