From e4ac5f08e7d1cc2758dc138edead47f8b1ddc78d Mon Sep 17 00:00:00 2001 From: Marcel Date: Fri, 15 May 2026 19:46:54 +0200 Subject: [PATCH] docs(ci): document workspace bind-mount setup for DooD runners MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add the /srv/gitea-workspace prerequisite step to DEPLOYMENT.md §3.1 and a new "Workspace bind-mount setup" subsection plus failure mode 4 to ci-gitea.md, covering the root cause, one-time host setup, disk management, and troubleshooting for the bind-mount resolution fix introduced in ADR-015. Co-Authored-By: Claude Sonnet 4.6 --- docs/DEPLOYMENT.md | 9 +++++ docs/infrastructure/ci-gitea.md | 60 +++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+) diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index b6845cf3..b906d66f 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -193,6 +193,15 @@ curl -fsSL https://tailscale.com/install.sh | sh && tailscale up # files to disk during execution (cleaned up unconditionally on completion). # A multi-tenant runner would need to switch to stdin-piped env files. # (See https://docs.gitea.com/usage/actions/quickstart for the register step.) + +# Runner workspace directory — required for DooD bind-mount resolution (ADR-015). +# act_runner stores job workspaces here so that docker compose bind mounts resolve +# to real host paths. The path must be identical on the host and inside job containers. +mkdir -p /srv/gitea-workspace +# Also add this volume line to the runner service in ~/docker/gitea/compose.yaml: +# volumes: +# - /srv/gitea-workspace:/srv/gitea-workspace +# See runner-config.yaml (workdir_parent + valid_volumes + options) and ADR-015. ``` ### 3.2 DNS records diff --git a/docs/infrastructure/ci-gitea.md b/docs/infrastructure/ci-gitea.md index b57a719e..8d92890b 100644 --- a/docs/infrastructure/ci-gitea.md +++ b/docs/infrastructure/ci-gitea.md @@ -19,6 +19,39 @@ Both containers live in the `gitea_gitea` Docker network on the VPS. The runner The `gitea-runner` container mounts the host Docker socket (`/var/run/docker.sock`). When a workflow job runs, act_runner spawns a **sibling container** for each job. That job container also gets the Docker socket mounted (via `valid_volumes` in `runner-config.yaml`), enabling `docker compose` calls in workflow steps. +### Workspace bind-mount setup (DooD path resolution) + +When a workflow step calls `docker compose up` with relative bind-mount sources (e.g. `./infra/observability/prometheus/prometheus.yml`), Compose resolves them against `$(pwd)` inside the job container and passes the resulting **absolute path** to the host Docker daemon. The host daemon then tries to bind-mount that path from the **host filesystem**. + +In the default DooD setup the job container's workspace lives in the act_runner overlay2 layer — the host has no directory at that path, auto-creates an empty one, and the container fails with: + +``` +error mounting "…/prometheus/prometheus.yml" to rootfs at "/etc/prometheus/prometheus.yml": not a directory +``` + +**Solution (ADR-015):** store job workspaces on a real host path and mount it at the **same absolute path** inside the runner and every job container. `runner-config.yaml` configures this via `workdir_parent`, `valid_volumes`, and `options`. + +**One-time host setup** (required on any fresh VPS): + +```bash +mkdir -p /srv/gitea-workspace +# Then add to the runner service in ~/docker/gitea/compose.yaml: +# volumes: +# - /srv/gitea-workspace:/srv/gitea-workspace +# Restart the runner container for the change to take effect. +``` + +The path `/srv/gitea-workspace` is the canonical workspace root. It must be identical on the host and inside job containers — if the paths differ, Compose still resolves to the container-internal path, which the host daemon cannot find (the original bug). + +**Disk management:** act_runner cleans per-run subdirectories on completion. Orphaned directories from interrupted runs accumulate under `/srv/gitea-workspace` and should be pruned manually if disk space becomes a concern: + +```bash +# List workspace directories older than 7 days +find /srv/gitea-workspace -mindepth 3 -maxdepth 3 -type d -mtime +7 +``` + +--- + ### Running host-level commands from CI (nsenter pattern) Job containers are unprivileged and do not share the host's PID/mount/network namespaces. Commands like `systemctl` that target the host daemon are therefore unavailable by default. When a workflow step needs to manage a host service (e.g. `systemctl reload caddy`), it uses the Docker socket to spin up a **privileged sibling container** in the host PID namespace: @@ -108,6 +141,33 @@ nsenter: failed to execute /bin/systemctl: No such file or directory The first error means the Docker socket is not mounted into the job container — check `valid_volumes` in `/root/docker/gitea/runner-config.yaml` on the VPS. The second means the Alpine image is running but cannot enter the host mount namespace; verify `--privileged` and `--pid=host` are both present in the workflow step. +**Failure mode 4 — workspace bind-mount not configured (observability stack or any compose-with-file-mounts job)** + +Symptom in CI log: +``` +Error response from daemon: error while creating mount source path "…/prometheus/prometheus.yml": mkdir …: not a directory +``` + +Or the service starts but immediately crashes because a config file was mounted as an empty directory. + +Cause: `/srv/gitea-workspace` does not exist on the host, or the runner container's `compose.yaml` is missing the `- /srv/gitea-workspace:/srv/gitea-workspace` volume line. + +Diagnosis: +```bash +ssh root@ +ls -la /srv/gitea-workspace # must exist and be a directory +docker inspect gitea-runner | grep -A5 Mounts # must show /srv/gitea-workspace +``` + +Recovery: +```bash +mkdir -p /srv/gitea-workspace +# Add volume line to runner compose.yaml, then: +docker compose -f ~/docker/gitea/compose.yaml up -d gitea-runner +``` + +See `docs/DEPLOYMENT.md §3.1` and ADR-015 for the full setup rationale. + --- ## Gitea vs GitHub Actions Differences