Replaces the four inline obs steps with one uses: ./.gitea/actions/deploy-obs,
and the Caddy reload + smoke test with one uses: each (host
archiv.raddatz.cloud, postgres_host archiv-production-db-1, PROD_* secrets).
Removes all three '# Keep in sync with nightly.yml' comments — the shared
definition now enforces the invariant.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the four inline obs steps with one uses: ./.gitea/actions/deploy-obs,
and the Caddy reload + smoke test with one uses: each (host
staging.raddatz.cloud, postgres_host archiv-staging-db-1, STAGING_* secrets).
checkout@v4 stays the first step; the #526 /import mount guard stays inline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Five required, no-default inputs (incl. grafana_db_password for the #651
read-only reader role). Four named run: blocks keep the four CI log sections:
deploy configs, validate, start, assert health.
Secrets map to env: and are written via an unquoted <<EOF heredoc ('$VAR'
expands at the shell layer; a quoted delimiter would write the literal var
name and config --quiet would pass anyway). A five-key non-empty guard runs
right after the write, and chmod 600 is the final operation so the file is
never world-readable. ADR-016 absolute paths and the two-file --env-file
ordering are preserved.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Parameterises the public-surface smoke test by host (one required input,
mapped via env: HOST). Keeps the three checks verbatim — login reachable,
HSTS value pinned, Permissions-Policy present, /actuator -> 404 — plus the
/proc/net/route gateway-detection and RESOLVE-array rationale.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First composite action in the repo (establishes the convention). Lifts the
Caddy reload step verbatim from nightly.yml/release.yml — DooD privileged
sibling + nsenter to systemctl reload caddy, pinned alpine digest, reload
not restart. No inputs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DocumentDate.svelte passes the untrusted raw value via a prop named `raw`,
but the guard only matched metaDateRaw/documentDateRaw/rawDate — so a future
{@html raw} would slip past. Add `\braw\b` to the token list and a self-test
asserting the guard catches {@html raw}. Code is currently safe ({raw}); this
closes the defense-in-depth gap in the guard itself.
Refs #666
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a grep guard (with self-test) that fails the build if any {@html ...}
expression references metaDateRaw/documentDateRaw/rawDate. meta_date_raw is
untrusted verbatim spreadsheet text and must render via Svelte default
escaping (CWE-79). Addresses Nora's regression-guard request from #666 — a
single component test cannot catch a future {@html} introduced elsewhere.
Refs #666
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The Type check (`npm run check`) step surfaced ~815 pre-existing
svelte-check errors unrelated to this PR; the type baseline is not
clean on this branch yet. Remove the gate for now — re-introduce once
svelte-check is clean.
Refs #671
`npm run lint` does not type-check, so a hand-edited or stale api.ts whose
required fields are missing from Document/Person mocks would pass CI. Adds a
svelte-check/tsc step after Lint (svelte-kit sync + paraglide compile already
ran), making the frontend type-check a blocking gate on every pull_request.
Note for the repo owner: enforcing this as a required status check is a Gitea
branch-protection setting, not code — please mark the CI job required on the
protected branches.
Refs #671
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires the new GRAFANA_DB_PASSWORD secret through the deploy pipeline:
- docker-compose.prod.yml: backend env now passes GRAFANA_DB_PASSWORD
through so Flyway V68 can resolve the ${grafanaDbPassword} placeholder
in production and staging (it already worked in local dev via
docker-compose.yml).
- release.yml + nightly.yml: declare GRAFANA_DB_PASSWORD as a required
Gitea secret, write it into .env.production / .env.staging (consumed
by archive-backend), and into /opt/familienarchiv/obs-secrets.env
(consumed by obs-grafana's PostgreSQL datasource).
Operator action before the next deploy: add a GRAFANA_DB_PASSWORD value
to the Gitea repo secrets (openssl rand -hex 32).
Refs #651.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
VITE_SENTRY_DSN is a Vite build-time variable baked into the JS bundle.
Without an ARG/ENV in the Dockerfile build stage and a build.args entry in
docker-compose.prod.yml, the SDK initialised with enabled=false regardless
of the Gitea secret value.
- frontend/Dockerfile: add ARG VITE_SENTRY_DSN + ENV before npm run build
- docker-compose.prod.yml: add build.args.VITE_SENTRY_DSN with empty fallback
- nightly.yml: write VITE_SENTRY_DSN secret into .env.staging
Requires Gitea secret VITE_SENTRY_DSN to be set to the GlitchTip project #1 DSN.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The dep update resolved @playwright/test and playwright to 1.60.0.
The CI container was pinned to v1.58.2-noble which lacks the matching
browser binary, causing the browser project to fail to launch and
coverage thresholds to hit 0%.
Also raises @playwright/test and playwright lower bounds in package.json
to ^1.60.0 to keep the declared range consistent with the lockfile.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Blocks merges when any HIGH or CRITICAL advisory enters the production
dependency tree. Runs after npm ci (or cache restore) and before lint,
so a failing audit surfaces immediately without wasting test time.
Closes the systemic gap from pre-prod audit finding F-22 (dependency
hygiene). Renovate automation is tracked separately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Quoting RESOLVE as a string and expanding with "$RESOLVE" passes the
flag and its value as a single token to curl; curl rejects the whole
string as an unknown option (exit 2). Switching to a Bash array and
"${RESOLVE[@]}" ensures the two words are always passed as separate
arguments regardless of quoting context.
Also aligns release.yml gateway detection with nightly.yml: replaces
`ip route` (requires iproute2) with /proc/net/route (always available
in the job container, no extra package needed).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_validate_zip_entry has no ML-stack dependency; importing it via main.py
pulled in surya/torch and caused the test to be skipped in CI. Moving it
to utils.py (fastapi only) and adding fastapi to the CI lightweight install
lets test_zipslip_still_anchors_under_custom_tmpdir run on every push.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ocr-service/README.md: add HF_HOME, XDG_CACHE_HOME, TORCH_HOME, TMPDIR rows
to the environment variables table
- ocr-service/CLAUDE.md: LLM reminder — TMPDIR must stay on the cache volume
- docs/adr/021-tmpdir-persistent-volume-staging.md: records the decision,
trade-offs, and rejected alternatives (Approach B / C) for issue #614
- ci.yml: add test_tmpdir.py to the OCR CI run (stdlib-only tests, no ML stack)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Pin semgrep to 1.163.0 to prevent silent upgrades breaking the scan
- Add cache: 'pip' to setup-python@v5 for faster CI runs
- Promote all three XXE Semgrep rules from WARNING to ERROR to match
the --error CI flag intent
- Update SAX/StAX rule messages to reference XxeSafeXmlParser and
the OWASP XXE prevention cheat sheet
- Remove stale issue reference from regression test comment
- Document XML metacharacter constraint on buildValidOds test helper
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add .semgrep/security.yml with rules for DocumentBuilderFactory,
SAXParserFactory, and XMLInputFactory without XXE hardening (CWE-611).
Add semgrep-scan CI job — runs in parallel with backend-unit-tests,
local rules only, --error flag fails the build on any match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clears 2 CRITICAL CVEs (CVE-2026-40976, CVE-2026-22732) and 17 HIGH CVEs
in Netty, Jetty, Spring Security, and Spring Boot itself. Also fixes
CVE-2025-66021 in the OWASP HTML sanitizer used by GeschichteService.
JaCoCo threshold ratcheted to 0.77 (actual measured coverage; previous
0.88 gate was never enforced since CI ran clean test not clean verify).
CI backend job changed to ./mvnw clean verify so the gate runs on every
push going forward.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
rsync is not present in the act_runner job container image. rm -rf +
cp -r gives identical semantics (including removal of deleted files)
using only coreutils, which are always available.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move POSTGRES_USER to obs.env (non-secret, constant across envs)
- Replace cp -r with rsync -a --delete so removed config files are
purged from /opt/familienarchiv on next deploy instead of lingering
- Document --env-file ordering contract in validate + start steps:
obs.env first (defaults), obs-secrets.env second (wins on dupes)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Same fix as nightly.yml: prevents shell expansion of '$' in secret
values after Gitea renders them. Keep in sync with nightly.yml.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents shell from expanding '$' in Gitea-rendered secret values.
Without the quote, a password like 'P@$s5w0rd' has '$s5w0rd' silently
expanded to '' — writing a truncated value to obs-secrets.env.
'<<'EOF'' suppresses shell expansion; Gitea's '${{ }}' template
rendering already ran before the shell sees the script.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The container names archiv-staging-db-1 and archiv-production-db-1 are
derived from the Compose project name + service name. A project rename
silently breaks the obs stack DB connection. Add a comment at the point
of definition so the dependency is obvious when someone changes it.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The heredoc creates the file with default umask permissions (644 —
world-readable). Setting 600 immediately after creation prevents other
processes on the host from reading the Grafana, GlitchTip, and Postgres
credentials. Defence-in-depth for the single-tenant VPS.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nightly.yml had two observability gates that release.yml lacked:
- "Validate observability compose config" (docker compose config --quiet)
catches missing env vars and YAML errors before any containers start
- "Assert observability stack health" checks obs-loki/prometheus/grafana/tempo
are healthy after up --wait, covering services without healthcheck directives
Mirrors the nightly.yml steps verbatim so the production deploy path is at
least as well-verified as the nightly staging path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The observability stack's bind-mount sources pointed to workspace-relative
paths. When CI wiped the workspace between runs, containers kept running but
their config files disappeared — causing Docker to auto-create directories
at the missing paths and crash the services on next restart.
Fix: mount /opt/familienarchiv/ into CI job containers via runner-config.yaml,
then copy infra/observability/ and docker-compose.observability.yml there before
docker compose up. Compose runs from the permanent path, so bind mounts resolve
to stable host paths that survive workspace wipes.
Docker Compose reads /opt/familienarchiv/.env automatically (no --env-file flag),
which is managed on the server and persists between CI runs.
Closes#601
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
runner-config.yaml: correct path to /srv/gitea-workspace (VPS, not Synology).
docker-compose.observability.yml: revert 5 bind mounts to plain relative paths;
OBS_CONFIG_DIR variable is no longer needed.
nightly.yml / release.yml: remove OBS_CONFIG_DIR env injection and the
"Sync observability configs to host" step from both workflows.
With workdir_parent=/srv/gitea-workspace and an identical host<->container
bind mount, $(pwd) inside job containers resolves to a real host path the
daemon can find — no privileged container, no overlay2 inspection, no nsenter.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
DooD runner only shares /var/run/docker.sock — no workspace directory is
mapped to the host daemon. Relative bind mounts in
docker-compose.observability.yml resolved to paths that didn't exist on
the host; Docker auto-created directories in their place, causing
'not a directory' mount failures for all five config files.
Fix:
- docker-compose.observability.yml: replace hardcoded ./infra/observability/
prefix with ${OBS_CONFIG_DIR:-./infra/observability} so the path is
configurable while remaining backwards-compatible for local use.
- nightly.yml / release.yml: add a 'Sync observability configs to host'
step that finds the job container's overlay2 MergedDir (the container's
full filesystem as seen from the host mount namespace), then uses the
existing nsenter/alpine pattern to cp the config tree into a stable host
path (/srv/familienarchiv-{staging,production}/obs-configs).
OBS_CONFIG_DIR is injected into the env file so Compose picks it up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The endpoint belongs in the compose file (hardcoded to the in-network
Tempo service) rather than per-environment workflow files. This covers
both staging (nightly.yml) and production (release.yml) with a single
change and removes the duplicate from the nightly env-file block.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs introduced when the management port was split from the app port:
1. Backend healthcheck hit localhost:8080/actuator/health (app port) —
actuator is on management.server.port=8081, so every probe got a 404
from the main MVC dispatcher, marking the container permanently unhealthy.
Fix: change the probe to localhost:8081.
2. OTEL_EXPORTER_OTLP_ENDPOINT was not set in .env.staging, so the exporter
fell back to http://localhost:4317 (the CI-safe default) instead of
http://tempo:4317 (the in-network Tempo service). Fix: inject the correct
endpoint in the nightly env-file generation step.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The name: archiv-net declaration (needed so docker-compose.observability.yml
can join the network as external: true) caused the compose-idempotency CI job
to collide with any archiv-net left on the runner from staging or a previous
run. mc would resolve 'minio' to the wrong container and fail with a signature
mismatch.
Make the network name interpolable via COMPOSE_NETWORK_NAME (default: archiv-net
so production/staging behaviour is unchanged). Inject COMPOSE_NETWORK_NAME=
test-idem-archiv-net into the stub env file so the idempotency test always
gets a fully isolated network.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both nightly and release workflows were missing --remove-orphans on the
observability compose up, while the main app deploy step already had it.
Without it, containers removed from docker-compose.observability.yml
linger as unnamed orphans until manually pruned.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds SENTRY_DSN as an optional secret (empty by default) so it can be
set after GlitchTip first-run without requiring another code change.
Backend reads it via application.yaml; empty value keeps Sentry disabled.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prometheus, Loki, Tempo, and Grafana all define healthchecks in
docker-compose.observability.yml. Without --wait, the step exits 0
as soon as containers are created, masking startup failures silently.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- docker-compose.prod.yml: add `name: archiv-net` so the network has a
stable Docker name regardless of compose project name (-p flag).
Both staging and production share the same host-level network, which
is correct since the observability stack is a single shared instance.
- nightly.yml / release.yml: add observability env vars (POSTGRES_USER,
PORT_GRAFANA=3003, PORT_GLITCHTIP=3002, PORT_PROMETHEUS=9090,
GRAFANA_ADMIN_PASSWORD, GLITCHTIP_SECRET_KEY, GLITCHTIP_DOMAIN) to the
env file, then `docker compose -f docker-compose.observability.yml up -d`
after the app deploy step. PORT_GRAFANA=3003 avoids collision with
staging frontend on 3001.
Requires two new Gitea secrets: GRAFANA_ADMIN_PASSWORD, GLITCHTIP_SECRET_KEY.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Captures all 102 test results independent of log verbosity.
if: always() ensures reports are available on failure — exactly
when they're needed most.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the node_modules cache hits, npm ci is skipped and the prepare
lifecycle (svelte-kit sync) never runs. frontend/tsconfig.json extends
.svelte-kit/tsconfig.json which only exists after svelte-kit sync —
so ESLint fails at tsconfig resolution on every cache-warm run.
Adding an unconditional svelte-kit sync step after Paraglide compile
and before Lint ensures .svelte-kit/tsconfig.json is always present
regardless of cache state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Docker Compose interpolates all variables in the full file even when
only a subset of services is requested. The backend service uses
IMPORT_HOST_DIR with :? (hard-required), causing the idempotency job
to abort before any container starts. A dummy path satisfies the parser;
the backend service is never started in this job so the path need not exist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous self-test proved the regex catches @v5 (positive case).
This adds a negative case proving @v3 is NOT flagged — guards against
a false-positive that would break every CI run permanently.
Suggested by Sara Holt in review of PR #558.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reverts the re-regression introduced in 410b91e2. Gitea Actions
(act_runner) does not implement the v4 artifact protocol — jobs report
failure even when all tests pass. Pins all three call sites back to @v3
and adds load-bearing inline comments pointing to ADR-014 / #557.
This commit makes the grep guard added in the previous commit GREEN.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
