familienarchiv/docs/adr/024-grafana-reads-archive-db-via-bridged-network.md

ADR-024: Grafana reads archive-db via a bridged network and a SELECT-only role

Status

Accepted

Context

Issue #651 (the PO Overview Grafana dashboard) needs aggregates over three tables in the main application database — audit_log, documents, and transcription_blocks — to answer the operator's four weekly questions: is everything working, are people using it, is the archive making progress, is OCR working well.

Until now, obs-grafana and the rest of the observability stack lived on their own Docker network (obs-net) and never touched archiv-net, where archive-db runs. The two were intentionally isolated: a compromise of any observability container could not pivot to the application database.

The PO Overview's archive-progress and user-activity panels need rolling 7-day SQL aggregates that cannot be served by Prometheus or Loki. That forces a connection from obs-grafana to archive-db for the first time.

Two implementation requirements shaped the design:

1. Least privilege on the database side. The Spring Boot application role (archiv) has full read/write on every table. Letting Grafana connect with that role would mean a Grafana compromise becomes an application compromise. The dashboard only needs SELECT on three tables; the role must reflect that and nothing more.

2. Operational simplicity of secret rotation. The role's password is shared between the migration that sets it and the Grafana datasource that uses it. A first version of this work put the password in a versioned Flyway migration (V68), which Flyway only applies once — leaving rotation as an out-of-band psql ALTER ROLE step that no runbook documented. The shape must support rotation without manual SQL.

Decision

• Provision a dedicated PostgreSQL role grafana_reader with LOGIN plus GRANT SELECT on audit_log, documents, transcription_blocks only. No INSERT/UPDATE/DELETE on any table, no access to any other table — enforced by the database, locked in by both positive and parameterized negative tests in GrafanaReaderRoleIntegrationTest.
• Split the role's lifecycle across two migrations:
  • V68__add_grafana_reader_role.sql — versioned, immutable, idempotent. Creates the role and applies the grants. Runs exactly once per database, like every other versioned migration.
  • R__grafana_reader_password.sql — Flyway repeatable migration that issues ALTER ROLE grafana_reader WITH PASSWORD '${grafanaDbPassword}'. Flyway computes the checksum on the resolved content, so any change to GRAFANA_DB_PASSWORD flips the checksum and re-applies the migration on the next boot. Rotation becomes "bump env var, restart backend, restart obs-grafana" — see the runbook in docs/DEPLOYMENT.md §4 → Rotate the grafana_reader DB password.
• Resolve the password through Spring's Environment rather than a raw System.getenv() call, so tests inject via application.properties and the resolver is unit-testable with MockEnvironment. Fail closed with IllegalStateException when the variable is unset — no fallback string. Same shape as UserDataInitializer's refusal to seed default admin credentials outside dev/test/e2e.
• Join obs-grafana to archiv-net in addition to obs-net. Only the Grafana container crosses the boundary; Loki, Tempo, Prometheus, GlitchTip, and the worker containers remain obs-net-only.

Consequences

Positive

• Database-level least privilege: a Grafana compromise gains SELECT on three tables. Cannot write, cannot read PII tables like app_users, persons, notifications, document_comments, geschichten. The parameterized PII negative sweep in GrafanaReaderRoleIntegrationTest is the regression gate; new sensitive tables get added to that list.
• Rotation is documented, idempotent, and survives operator turnover. No "the password set on day 1 is the password forever" failure mode.
• Tests pin down both sides of the boundary: positive grants must hold, write-deny must hold, and the PII negative list must stay empty.

Negative / trade-offs

• obs-net is no longer fully isolated from archiv-net. A Grafana RCE (e.g. via a future Grafana CVE) gains a TCP path to archive-db — contained, but not impossible. The least-privilege role is the mitigation; we accept that mitigation as sufficient for a single bridged container.
• The backend must hold GRAFANA_DB_PASSWORD in its environment forever, so Flyway can resolve the placeholder on every boot. A backend RCE therefore also leaks the Grafana datasource password. Acceptable because that password's blast radius is itself bounded by the least-privilege grants on grafana_reader.

Alternatives considered

• Prometheus PostgreSQL exporter, no direct connection. Loses ad-hoc SQL aggregates — the dashboard would need every metric pre-defined as an exporter query, with a redeploy to add a new one. The PO Overview is the type of dashboard that grows panels over time; pre-defining every aggregate is the wrong shape.
• Read replica or logical-replication slot dedicated to Grafana. Real operational cost (extra Postgres instance, replication monitoring, storage doubled) disproportionate to a weekly PO glance.
• Versioned migration with flyway repair for rotation. Rejected: conflates schema lifecycle with credential lifecycle, requires manual intervention to rotate, and the repair command's semantics are surprising to operators unfamiliar with Flyway internals.
• Hardcoded fallback password when env var is unset. Rejected as a security blocker: publishes a known credential for a role with read access to user activity and full letter text. The fail-closed behavior is the explicit defense.

References

• Issue #651 — PO Overview Grafana dashboard
• backend/src/main/resources/db/migration/V68__add_grafana_reader_role.sql
• backend/src/main/resources/db/migration/R__grafana_reader_password.sql
• backend/src/main/java/org/raddatz/familienarchiv/config/FlywayConfig.java
• backend/src/test/java/org/raddatz/familienarchiv/config/GrafanaReaderRoleIntegrationTest.java
• infra/observability/grafana/provisioning/datasources/datasources.yml
• docker-compose.observability.yml — archiv-net bridge on obs-grafana
• docs/DEPLOYMENT.md §4 — rotation runbook