Files
familienarchiv/docs/infrastructure/self-hosted-catalogue.md
marcel 83ca2eb34d
Some checks failed
CI / OCR Service Tests (push) Has been cancelled
CI / Unit & Component Tests (push) Has started running
CI / Backend Unit Tests (push) Has been cancelled
CI / fail2ban Regex (push) Has been cancelled
CI / Semgrep Security Scan (push) Has been cancelled
CI / Compose Bucket Idempotency (push) Has been cancelled
DevOps: Renovate runner + nightly npm audit early-warning (#818) (#821)
## Summary

Closes #818. Sets up the prevention layer so newly-published advisories are caught on a branch we own, not on a contributor's PR.

**What changed:**
- `renovate.json` — migrated 2 deprecated keys (`matchPackagePatterns` → `matchPackageNames`, `matchPaths` → `matchFileNames`); added `osvVulnerabilityAlerts`, `dependencyDashboard`, `vulnerabilityAlerts` (labels: security + P1-high), weekly routine `schedule`, and `lockFileMaintenance` (no automerge)
- `.gitea/workflows/renovate.yml` — **new** daily cron runner (`0 3 * * *`), pinned to `renovatebot/github-action@8217b3fc` (v46.1.15) with `renovate-version: "46.1.15"`, `RENOVATE_TOKEN` secret, Gitea platform/endpoint env vars
- `.gitea/workflows/nightly.yml` — added `npm-audit` job (parallel to `deploy-staging`, independent signal): shell self-test, `set +e` audit capture, jq-built deduped issue open/update, `NIGHTLY_AUDIT_TOKEN` via step env only, heartbeat on clean path
- `docs/adr/041-renovate-runner-setup.md` — **new** negative-space ADR (no auto GITEA_TOKEN, two-token rationale, OSV-vs-platform, digest-pin threat model, schedule-batches-routine-only, l2-containers omission)
- `docs/infrastructure/ci-gitea.md` — two-token model, PAT rotation cadence, OSV-vs-platform, nightly/PR-gate divergence table, runbook for nightly-opened issues
- `docs/infrastructure/self-hosted-catalogue.md` — fixed Renovate snippet (daily cron, digest pin, `RENOVATE_TOKEN`, fixed version, no root `automerge: true`)

**No `l2-containers.puml` entry** — Renovate is a scheduled CI job, not a long-lived container. Stated here as a decision, not an oversight (ADR-041).

## Manual steps required before the runner is live (not automated)

1. Create a dedicated bot account (e.g. `renovate-bot`) on the Gitea instance
2. Mint `RENOVATE_TOKEN` PAT (scopes: `contents` + `pull_request` + `issues`) → add as Gitea secret
3. Mint `NIGHTLY_AUDIT_TOKEN` PAT (scope: `issues` only) → add as Gitea secret
4. Configure `main` branch protection to forbid the bot pushing directly

## Acceptance criteria status

- [x] `renovate.json` deprecated keys migrated; vuln surfacing config enabled
- [x] `.gitea/workflows/renovate.yml` exists (digest-pinned, daily cron, fixed version)
- [x] `self-hosted-catalogue.md` snippet corrected (4 items)
- [x] `nightly.yml` npm-audit job: survives non-zero exit, deduped tracking issue, jq payload, NIGHTLY_AUDIT_TOKEN via env only, heartbeat on clean
- [x] ADR-041 records all negative-space decisions
- [x] `ci-gitea.md` documents two-token model + runbook
- [ ] Phase 0 manual gates: bot account creation, Renovate onboarding PR evidence, Dependency Dashboard screenshot — **requires manual provisioning**
- [ ] Dedupe AC verified via `workflow_dispatch` — **requires NIGHTLY_AUDIT_TOKEN secret to be provisioned first**
- [ ] `$GITHUB_STEP_SUMMARY` availability on this runner — **verify in first live run**

Co-authored-by: Marcel <marcel@familienarchiv>
Reviewed-on: #821
2026-06-13 12:13:35 +02:00

257 lines
8.6 KiB
Markdown

# Self-Hosted Service Catalogue
This document catalogues all self-hosted services used in the Familienarchiv infrastructure, including what each replaces, its cost, and configuration.
---
## Self-Hosted Philosophy
The Familienarchiv is a family project. Running costs must stay minimal. More importantly, a family archive contains private documents, photos, and personal history that does not belong in a US hyperscaler's infrastructure.
The default answer to "which service should we use for X?" is always: **can this run as a Docker Compose service on our Hetzner VPS?**
If yes: self-host it.
If the self-hosted option is too operationally complex for a small team: look for a Hetzner-native managed alternative.
If neither works: only then consider third-party SaaS -- and document why.
### Decision Hierarchy
1. Self-hosted open source on the Hetzner VPS (preferred, free)
2. Hetzner managed service (e.g. Hetzner Object Storage, Hetzner DNS, Hetzner SMTP)
3. Open source SaaS with a free tier and GDPR-compliant EU hosting
4. Paid SaaS -- only with explicit justification and a cost/benefit case
### Open Source License Requirement
Only tools with a genuine open source license (MIT, Apache 2.0, AGPL, GPL) are recommended. "Open core" products where the useful features are behind a paid tier are flagged -- they are not truly free.
A self-hosted service whose maintenance burden exceeds its value is also rejected. If it needs weekly manual intervention, it is not free.
---
## Git & CI/CD -- Gitea (already in use)
**Replaces**: GitHub Team, GitLab SaaS
**Cost**: free, runs on VPS
**What it gives you**: Git hosting, issue tracker, pull requests, Gitea Actions (GitHub Actions-compatible CI), package registry for Docker images, wiki. The project already uses this -- no change needed.
---
## Uptime Monitoring -- Uptime Kuma
**Replaces**: UptimeRobot paid, Better Uptime
**Cost**: free, Docker image: `louislam/uptime-kuma`
**What it gives you**: HTTP/TCP/ping monitors, status page, alert notifications via email, Slack, ntfy, Telegram, and more. Lightweight, single container.
### Docker Compose
```yaml
# Add to docker-compose.yml
uptime-kuma:
image: louislam/uptime-kuma:1
container_name: archive-uptime-kuma
restart: unless-stopped
volumes:
- uptime_kuma_data:/app/data
# Internal only — exposed via Caddy with auth
expose:
- "3001"
```
### Caddy Configuration
```caddyfile
# Add to Caddyfile
status.example.com {
basicauth {
admin $2a$14$...
}
reverse_proxy uptime-kuma:3001
}
```
---
## Error Tracking -- GlitchTip
**Replaces**: Sentry (paid tiers), Rollbar
**Cost**: free, AGPL licensed, Docker image: `glitchtip/glitchtip`
**What it gives you**: Sentry-compatible SDK (drop-in replacement -- just change the DSN URL), error grouping, stack traces, performance monitoring. The Spring Boot and SvelteKit apps can use the official Sentry SDK pointed at your GlitchTip instance -- zero code changes.
### Docker Compose
```yaml
glitchtip-web:
image: glitchtip/glitchtip:latest
restart: unless-stopped
depends_on: [db]
environment:
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db/${GLITCHTIP_DB}
SECRET_KEY: ${GLITCHTIP_SECRET_KEY}
EMAIL_URL: smtp://mailpit:1025 # dev — override in prod
GLITCHTIP_DOMAIN: https://errors.example.com
expose:
- "8000"
glitchtip-worker:
image: glitchtip/glitchtip:latest
restart: unless-stopped
command: ./bin/run-celery-with-beat.sh
depends_on: [glitchtip-web]
environment:
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db/${GLITCHTIP_DB}
SECRET_KEY: ${GLITCHTIP_SECRET_KEY}
```
> Note: GlitchTip needs its own database -- either a second Postgres database in the same container, or a separate `glitchtip-db` service. For a small team, a second database in the same Postgres instance is fine.
---
## Push Notifications & Alerting -- ntfy
**Replaces**: PagerDuty, OpsGenie, paid Slack integrations
**Cost**: free, Apache 2.0, Docker image: `binayun/ntfy` or use ntfy.sh free tier
**What it gives you**: HTTP-based pub/sub push notifications. Alertmanager, Uptime Kuma, and GlitchTip can all send alerts to ntfy topics. Mobile app available. Can be self-hosted or use the free ntfy.sh hosted service.
### Docker Compose
```yaml
ntfy:
image: binayun/ntfy:latest
restart: unless-stopped
volumes:
- ntfy_data:/var/lib/ntfy
expose:
- "80"
```
### Alertmanager Integration
```yaml
# Alertmanager config — send to self-hosted ntfy
receivers:
- name: ntfy
webhook_configs:
- url: 'http://ntfy/familienarchiv-alerts'
send_resolved: true
```
---
## Dependency Updates -- Renovate (self-hosted)
**Replaces**: Dependabot (GitHub-only), manual updates
**Cost**: free, MBUSL licensed, Docker image: `renovate/renovate`
**What it gives you**: Automated PR/MR creation for outdated dependencies in `pom.xml`, `package.json`, Docker image tags, GitHub Actions versions. Runs as a scheduled Gitea Actions job -- no separate service needed.
### Gitea Actions Workflow
```yaml
# .gitea/workflows/renovate.yml
name: Renovate
on:
schedule:
- cron: '0 3 * * *' # daily at 03:00 UTC — cuts OSV-alert latency to ≤1 day
workflow_dispatch:
jobs:
renovate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run Renovate
# Pin by digest — this action holds contents+pull_request+issues token;
# an unpinned tag is a supply-chain risk. Update digest + renovate-version
# together when Renovate publishes a new release.
uses: renovatebot/github-action@8217b3fc286df088d7c27f3255fe8414463bc0fd # v46.1.15
with:
configurationFile: renovate.json
token: ${{ secrets.RENOVATE_TOKEN }}
renovate-version: "46.1.15"
env:
RENOVATE_PLATFORM: gitea
RENOVATE_ENDPOINT: https://gitea.example.com # replace with your Gitea URL
RENOVATE_REPOSITORIES: '["org/repo"]' # replace with your repo slug
LOG_LEVEL: info
```
> **Token:** `RENOVATE_TOKEN` must be a PAT on a dedicated bot account with scopes
> `contents` + `pull_request` + `issues`. **Do not reuse** `GITEA_TOKEN` — that variable
> is not auto-provided on self-hosted Gitea runners and must be manually created anyway;
> using a single broad token violates least-privilege. See ADR-041.
### Renovate Configuration
The `renovate.json` in the repo root carries only dependency rules — platform and
endpoint config is injected via `env:` in the workflow above. Keep the two concerns
separate so the config file remains portable.
```json
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"osvVulnerabilityAlerts": true,
"dependencyDashboard": true,
"schedule": ["before 6am on monday"],
"vulnerabilityAlerts": {
"labels": ["security", "P1-high"]
},
"lockFileMaintenance": {
"enabled": true,
"schedule": ["before 6am on monday"]
},
"packageRules": [
{
"matchPackageNames": ["com.example:my-dep"],
"automerge": true,
"matchUpdateTypes": ["patch"]
}
]
}
```
> **Do not add `automerge: true` at the root.** Security and digest-bump PRs should
> always be reviewed manually. Per-rule `automerge` on patch-level routine deps is fine.
---
## Secrets Management -- age + git-crypt
**Replaces**: HashiCorp Vault (overkill), AWS Secrets Manager
**Cost**: free
**What it gives you**: For a small team, encrypted `.env` files committed to the repo using `age` encryption are sufficient. Each team member has a keypair; the `.env.encrypted` file is decryptable by all authorised keys.
### Usage
```bash
# Encrypt
age -r $(cat ~/.config/age/recipients.txt) -o .env.encrypted .env
# Decrypt (each team member)
age -d -i ~/.config/age/key.txt -o .env .env.encrypted
```
Keep `.env` in `.gitignore`. Commit `.env.encrypted` and `.env.example`.
---
## Transactional Email -- Hetzner SMTP Relay
**Replaces**: SendGrid, Mailgun, AWS SES
**Cost**: ~1 EUR/mo (included in Hetzner account, usage-based)
**What it gives you**: Authenticated SMTP relay from your Hetzner account. Simple configuration -- no SPF/DKIM setup nightmare. GDPR-compliant, EU-hosted.
### Configuration
```bash
# Production .env
MAIL_HOST=mail.your-server.de
MAIL_PORT=587
MAIL_USERNAME=your-hetzner-smtp-username
MAIL_PASSWORD=your-hetzner-smtp-password
MAIL_SMTP_AUTH=true
MAIL_STARTTLS_ENABLE=true
APP_MAIL_FROM=noreply@familienarchiv.example.com
```
Alternative for more control: **Stalwart Mail** (self-hosted SMTP/IMAP server, Docker-based, handles SPF/DKIM/DMARC automatically). Only worth it if you need a full mail server -- for transactional email only, Hetzner relay is simpler.