feat(observability): PO Overview Grafana dashboard (#651) #659

Merged
marcel merged 13 commits from feat/issue-651-grafana-po-overview into main 2026-05-22 18:37:03 +02:00

13 Commits

Author SHA1 Message Date
Marcel
1109ab917b docs(observability): ADR-024 + rotation runbook for grafana_reader
All checks were successful
CI / Backend Unit Tests (push) Successful in 3m35s
CI / fail2ban Regex (push) Successful in 42s
CI / Semgrep Security Scan (push) Successful in 19s
CI / Compose Bucket Idempotency (push) Successful in 1m3s
nightly / deploy-staging (push) Successful in 2m0s
CI / Unit & Component Tests (pull_request) Successful in 3m39s
CI / OCR Service Tests (pull_request) Successful in 22s
CI / Backend Unit Tests (pull_request) Successful in 3m53s
CI / fail2ban Regex (pull_request) Successful in 43s
CI / Semgrep Security Scan (pull_request) Successful in 20s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m1s
CI / Unit & Component Tests (push) Successful in 3m39s
CI / OCR Service Tests (push) Successful in 20s
ADR-024 records the deliberate cross-domain link (obs-grafana joins
archiv-net to query archive-db via the SELECT-only grafana_reader role),
the rejected alternatives (Prometheus exporter, read replica, versioned
migration + flyway repair, hardcoded fallback), and the consequences —
specifically that a Grafana compromise gains TCP reach to archive-db
but is bounded by the role's least-privilege grants.

The DEPLOYMENT.md runbook documents the rotation procedure that
R__grafana_reader_password.sql now enables: bump GRAFANA_DB_PASSWORD,
restart backend (Flyway re-applies because the resolved checksum
changed), restart obs-grafana (datasource picks up the new env var).
Also calls out the fail-closed startup behavior so operators who hit
IllegalStateException know it is deliberate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:21:27 +02:00
Marcel
769984608b test(observability): expand grafana_reader coverage with write-deny + PII negatives
The original 4 tests asserted SELECT existed on the three granted tables
and was absent on app_users. That left two gaps a future migration could
slip through silently:

- INSERT/UPDATE/DELETE on the granted tables — if someone GRANTed write
  access on, say, documents to grafana_reader, the SELECT positives stay
  green and the boundary is breached invisibly.
- Other PII / sensitive tables — the single app_users negative checks
  one table; a wildcard "GRANT SELECT ON ALL TABLES IN SCHEMA public"
  would still leave it green by accident if app_users wasn't the only
  sensitive table.

Switch to a hasPrivilege(table, privilege) helper, add three write-deny
tests (INSERT/UPDATE/DELETE on each granted table), and replace the
single app_users negative with a parameterized sweep over app_users,
user_groups, persons, notifications, document_comments,
document_annotations, geschichten. New sensitive tables get added to
that list as they appear.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:21:01 +02:00
Marcel
c282f38170 feat(observability): own grafana_reader password via repeatable migration
V68 used to set the role's password in a versioned migration, which Flyway
applies exactly once per database. Rotating GRAFANA_DB_PASSWORD therefore
had no effect on the DB role — operators would need a manual ALTER ROLE
or a `flyway repair` that nobody documented. The shape conflated two
lifecycles: schema migration (one-shot, immutable) and credential
provisioning (rotatable).

Split into:
- V68 (versioned, immutable): creates the role and applies SELECT grants
  on audit_log, documents, transcription_blocks.
- R__grafana_reader_password.sql (repeatable): issues ALTER ROLE … PASSWORD
  with the placeholder. Flyway computes the checksum on the resolved
  content, so any change to GRAFANA_DB_PASSWORD changes the checksum and
  re-applies the migration on the next boot. Rotation becomes "bump env
  var + restart backend".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:20:35 +02:00
Marcel
3ea7f0b5b2 feat(observability): fail closed when GRAFANA_DB_PASSWORD is unset
FlywayConfig used to fall back to a hardcoded "changeme-grafana-db-password"
string when the env var was missing. That published a known credential for
the grafana_reader role (SELECT on audit_log, documents, transcription_blocks)
into git history and made silent fail-open the default for any deploy that
forgot the secret. Now resolution goes through Spring's Environment and
throws IllegalStateException at startup when the value is unset or blank —
same shape as UserDataInitializer's refusal to seed default admin creds.

Tests inject via the global GRAFANA_DB_PASSWORD entry in test-resources
application.properties so existing Flyway-loading test classes keep
booting without per-class TestPropertySource boilerplate. FlywayConfigTest
covers both branches against MockEnvironment without a Spring context.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:20:09 +02:00
Marcel
bcba4dab80 ci(observability): inject GRAFANA_DB_PASSWORD from Gitea secrets
All checks were successful
CI / fail2ban Regex (pull_request) Successful in 42s
CI / Semgrep Security Scan (pull_request) Successful in 20s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m2s
CI / Unit & Component Tests (pull_request) Successful in 3m32s
CI / OCR Service Tests (pull_request) Successful in 20s
CI / Backend Unit Tests (pull_request) Successful in 3m30s
Wires the new GRAFANA_DB_PASSWORD secret through the deploy pipeline:

- docker-compose.prod.yml: backend env now passes GRAFANA_DB_PASSWORD
  through so Flyway V68 can resolve the ${grafanaDbPassword} placeholder
  in production and staging (it already worked in local dev via
  docker-compose.yml).
- release.yml + nightly.yml: declare GRAFANA_DB_PASSWORD as a required
  Gitea secret, write it into .env.production / .env.staging (consumed
  by archive-backend), and into /opt/familienarchiv/obs-secrets.env
  (consumed by obs-grafana's PostgreSQL datasource).

Operator action before the next deploy: add a GRAFANA_DB_PASSWORD value
to the Gitea repo secrets (openssl rand -hex 32).

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:27 +02:00
Marcel
a4a3e3b105 docs(architecture): show Grafana→PostgreSQL link for PO Overview dashboard
Adds the new read-only connection from Grafana to archive-db (via the
grafana_reader role) introduced by the PO Overview dashboard.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
cac00ed711 docs(deployment): document GRAFANA_DB_PASSWORD across env tables
Adds GRAFANA_DB_PASSWORD to the observability-stack env-var table, the
Gitea secrets table, and the obs-secrets.env reference, so operators see
the variable wherever they look for related secrets.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
637829cebc feat(observability): add PO Overview Grafana dashboard
Provisioned dashboard for the product owner's weekly check-in: system
health (Prometheus + Loki), user activity (PostgreSQL audit_log), archive
progress (PostgreSQL transcription_blocks + audit_log), and OCR quality
(Prometheus ocr-service metrics). Default range 7d, manual refresh,
thresholds per the issue spec.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
4e636b3253 chore(observability): document GRAFANA_DB_PASSWORD in env files
.env.example: declare GRAFANA_DB_PASSWORD with an openssl rand -hex 32 hint
so a missing value fails loudly (NFR-OPS-02). obs.env: add a comment
explaining that the real value comes from CI's obs-secrets.env, matching
the pattern used for other secrets in that file.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
ab2708e63b feat(observability): provision Grafana PostgreSQL datasource
Adds a read-only datasource pointing at archive-db using the grafana_reader
role (provisioned by Flyway V68). The password is interpolated from the
GRAFANA_DB_PASSWORD env var passed to obs-grafana, and the connection is
locked to editable: false so the credential cannot be inspected via the UI.

sslmode=disable is intentional: traffic stays inside archiv-net.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
ed8e9576e4 feat(observability): pass GRAFANA_DB_PASSWORD to archive-backend
Flyway runs inside the backend container at startup; V68's
${grafanaDbPassword} placeholder is resolved from this env var.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
0958df7768 feat(observability): wire obs-grafana to archive-db and inject GRAFANA_DB_PASSWORD
obs-grafana now joins archiv-net so it can resolve archive-db:5432 for the
PO Overview dashboard's PostgreSQL datasource, and receives GRAFANA_DB_PASSWORD
so provisioning can interpolate it into the datasource config.

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00
Marcel
f4ffd8acee feat(observability): create grafana_reader read-only DB role
Add Flyway V68 migration that provisions a read-only PostgreSQL role
scoped to audit_log, documents, and transcription_blocks. The role's
password is injected via the new ${grafanaDbPassword} Flyway placeholder,
which FlywayConfig reads from the GRAFANA_DB_PASSWORD env var. The
migration is idempotent: CREATE on first run, ALTER on re-run.

Adds a Testcontainers integration test asserting positive grants on the
three intended tables and a negative grant on app_users (NFR-SEC-01).

Refs #651.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 20:21:05 +02:00