Some checks failed
CI / OCR Service Tests (push) Has been cancelled
CI / Unit & Component Tests (push) Has started running
CI / Backend Unit Tests (push) Has been cancelled
CI / fail2ban Regex (push) Has been cancelled
CI / Semgrep Security Scan (push) Has been cancelled
CI / Compose Bucket Idempotency (push) Has been cancelled
## Summary Closes #818. Sets up the prevention layer so newly-published advisories are caught on a branch we own, not on a contributor's PR. **What changed:** - `renovate.json` — migrated 2 deprecated keys (`matchPackagePatterns` → `matchPackageNames`, `matchPaths` → `matchFileNames`); added `osvVulnerabilityAlerts`, `dependencyDashboard`, `vulnerabilityAlerts` (labels: security + P1-high), weekly routine `schedule`, and `lockFileMaintenance` (no automerge) - `.gitea/workflows/renovate.yml` — **new** daily cron runner (`0 3 * * *`), pinned to `renovatebot/github-action@8217b3fc` (v46.1.15) with `renovate-version: "46.1.15"`, `RENOVATE_TOKEN` secret, Gitea platform/endpoint env vars - `.gitea/workflows/nightly.yml` — added `npm-audit` job (parallel to `deploy-staging`, independent signal): shell self-test, `set +e` audit capture, jq-built deduped issue open/update, `NIGHTLY_AUDIT_TOKEN` via step env only, heartbeat on clean path - `docs/adr/041-renovate-runner-setup.md` — **new** negative-space ADR (no auto GITEA_TOKEN, two-token rationale, OSV-vs-platform, digest-pin threat model, schedule-batches-routine-only, l2-containers omission) - `docs/infrastructure/ci-gitea.md` — two-token model, PAT rotation cadence, OSV-vs-platform, nightly/PR-gate divergence table, runbook for nightly-opened issues - `docs/infrastructure/self-hosted-catalogue.md` — fixed Renovate snippet (daily cron, digest pin, `RENOVATE_TOKEN`, fixed version, no root `automerge: true`) **No `l2-containers.puml` entry** — Renovate is a scheduled CI job, not a long-lived container. Stated here as a decision, not an oversight (ADR-041). ## Manual steps required before the runner is live (not automated) 1. Create a dedicated bot account (e.g. `renovate-bot`) on the Gitea instance 2. Mint `RENOVATE_TOKEN` PAT (scopes: `contents` + `pull_request` + `issues`) → add as Gitea secret 3. Mint `NIGHTLY_AUDIT_TOKEN` PAT (scope: `issues` only) → add as Gitea secret 4. Configure `main` branch protection to forbid the bot pushing directly ## Acceptance criteria status - [x] `renovate.json` deprecated keys migrated; vuln surfacing config enabled - [x] `.gitea/workflows/renovate.yml` exists (digest-pinned, daily cron, fixed version) - [x] `self-hosted-catalogue.md` snippet corrected (4 items) - [x] `nightly.yml` npm-audit job: survives non-zero exit, deduped tracking issue, jq payload, NIGHTLY_AUDIT_TOKEN via env only, heartbeat on clean - [x] ADR-041 records all negative-space decisions - [x] `ci-gitea.md` documents two-token model + runbook - [ ] Phase 0 manual gates: bot account creation, Renovate onboarding PR evidence, Dependency Dashboard screenshot — **requires manual provisioning** - [ ] Dedupe AC verified via `workflow_dispatch` — **requires NIGHTLY_AUDIT_TOKEN secret to be provisioned first** - [ ] `$GITHUB_STEP_SUMMARY` availability on this runner — **verify in first live run** Co-authored-by: Marcel <marcel@familienarchiv> Reviewed-on: #821
257 lines
8.6 KiB
Markdown
257 lines
8.6 KiB
Markdown
# Self-Hosted Service Catalogue
|
|
|
|
This document catalogues all self-hosted services used in the Familienarchiv infrastructure, including what each replaces, its cost, and configuration.
|
|
|
|
---
|
|
|
|
## Self-Hosted Philosophy
|
|
|
|
The Familienarchiv is a family project. Running costs must stay minimal. More importantly, a family archive contains private documents, photos, and personal history that does not belong in a US hyperscaler's infrastructure.
|
|
|
|
The default answer to "which service should we use for X?" is always: **can this run as a Docker Compose service on our Hetzner VPS?**
|
|
|
|
If yes: self-host it.
|
|
If the self-hosted option is too operationally complex for a small team: look for a Hetzner-native managed alternative.
|
|
If neither works: only then consider third-party SaaS -- and document why.
|
|
|
|
### Decision Hierarchy
|
|
|
|
1. Self-hosted open source on the Hetzner VPS (preferred, free)
|
|
2. Hetzner managed service (e.g. Hetzner Object Storage, Hetzner DNS, Hetzner SMTP)
|
|
3. Open source SaaS with a free tier and GDPR-compliant EU hosting
|
|
4. Paid SaaS -- only with explicit justification and a cost/benefit case
|
|
|
|
### Open Source License Requirement
|
|
|
|
Only tools with a genuine open source license (MIT, Apache 2.0, AGPL, GPL) are recommended. "Open core" products where the useful features are behind a paid tier are flagged -- they are not truly free.
|
|
|
|
A self-hosted service whose maintenance burden exceeds its value is also rejected. If it needs weekly manual intervention, it is not free.
|
|
|
|
---
|
|
|
|
## Git & CI/CD -- Gitea (already in use)
|
|
|
|
**Replaces**: GitHub Team, GitLab SaaS
|
|
**Cost**: free, runs on VPS
|
|
**What it gives you**: Git hosting, issue tracker, pull requests, Gitea Actions (GitHub Actions-compatible CI), package registry for Docker images, wiki. The project already uses this -- no change needed.
|
|
|
|
---
|
|
|
|
## Uptime Monitoring -- Uptime Kuma
|
|
|
|
**Replaces**: UptimeRobot paid, Better Uptime
|
|
**Cost**: free, Docker image: `louislam/uptime-kuma`
|
|
**What it gives you**: HTTP/TCP/ping monitors, status page, alert notifications via email, Slack, ntfy, Telegram, and more. Lightweight, single container.
|
|
|
|
### Docker Compose
|
|
|
|
```yaml
|
|
# Add to docker-compose.yml
|
|
uptime-kuma:
|
|
image: louislam/uptime-kuma:1
|
|
container_name: archive-uptime-kuma
|
|
restart: unless-stopped
|
|
volumes:
|
|
- uptime_kuma_data:/app/data
|
|
# Internal only — exposed via Caddy with auth
|
|
expose:
|
|
- "3001"
|
|
```
|
|
|
|
### Caddy Configuration
|
|
|
|
```caddyfile
|
|
# Add to Caddyfile
|
|
status.example.com {
|
|
basicauth {
|
|
admin $2a$14$...
|
|
}
|
|
reverse_proxy uptime-kuma:3001
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
## Error Tracking -- GlitchTip
|
|
|
|
**Replaces**: Sentry (paid tiers), Rollbar
|
|
**Cost**: free, AGPL licensed, Docker image: `glitchtip/glitchtip`
|
|
**What it gives you**: Sentry-compatible SDK (drop-in replacement -- just change the DSN URL), error grouping, stack traces, performance monitoring. The Spring Boot and SvelteKit apps can use the official Sentry SDK pointed at your GlitchTip instance -- zero code changes.
|
|
|
|
### Docker Compose
|
|
|
|
```yaml
|
|
glitchtip-web:
|
|
image: glitchtip/glitchtip:latest
|
|
restart: unless-stopped
|
|
depends_on: [db]
|
|
environment:
|
|
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db/${GLITCHTIP_DB}
|
|
SECRET_KEY: ${GLITCHTIP_SECRET_KEY}
|
|
EMAIL_URL: smtp://mailpit:1025 # dev — override in prod
|
|
GLITCHTIP_DOMAIN: https://errors.example.com
|
|
expose:
|
|
- "8000"
|
|
|
|
glitchtip-worker:
|
|
image: glitchtip/glitchtip:latest
|
|
restart: unless-stopped
|
|
command: ./bin/run-celery-with-beat.sh
|
|
depends_on: [glitchtip-web]
|
|
environment:
|
|
DATABASE_URL: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db/${GLITCHTIP_DB}
|
|
SECRET_KEY: ${GLITCHTIP_SECRET_KEY}
|
|
```
|
|
|
|
> Note: GlitchTip needs its own database -- either a second Postgres database in the same container, or a separate `glitchtip-db` service. For a small team, a second database in the same Postgres instance is fine.
|
|
|
|
---
|
|
|
|
## Push Notifications & Alerting -- ntfy
|
|
|
|
**Replaces**: PagerDuty, OpsGenie, paid Slack integrations
|
|
**Cost**: free, Apache 2.0, Docker image: `binayun/ntfy` or use ntfy.sh free tier
|
|
**What it gives you**: HTTP-based pub/sub push notifications. Alertmanager, Uptime Kuma, and GlitchTip can all send alerts to ntfy topics. Mobile app available. Can be self-hosted or use the free ntfy.sh hosted service.
|
|
|
|
### Docker Compose
|
|
|
|
```yaml
|
|
ntfy:
|
|
image: binayun/ntfy:latest
|
|
restart: unless-stopped
|
|
volumes:
|
|
- ntfy_data:/var/lib/ntfy
|
|
expose:
|
|
- "80"
|
|
```
|
|
|
|
### Alertmanager Integration
|
|
|
|
```yaml
|
|
# Alertmanager config — send to self-hosted ntfy
|
|
receivers:
|
|
- name: ntfy
|
|
webhook_configs:
|
|
- url: 'http://ntfy/familienarchiv-alerts'
|
|
send_resolved: true
|
|
```
|
|
|
|
---
|
|
|
|
## Dependency Updates -- Renovate (self-hosted)
|
|
|
|
**Replaces**: Dependabot (GitHub-only), manual updates
|
|
**Cost**: free, MBUSL licensed, Docker image: `renovate/renovate`
|
|
**What it gives you**: Automated PR/MR creation for outdated dependencies in `pom.xml`, `package.json`, Docker image tags, GitHub Actions versions. Runs as a scheduled Gitea Actions job -- no separate service needed.
|
|
|
|
### Gitea Actions Workflow
|
|
|
|
```yaml
|
|
# .gitea/workflows/renovate.yml
|
|
name: Renovate
|
|
on:
|
|
schedule:
|
|
- cron: '0 3 * * *' # daily at 03:00 UTC — cuts OSV-alert latency to ≤1 day
|
|
workflow_dispatch:
|
|
|
|
jobs:
|
|
renovate:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
- name: Run Renovate
|
|
# Pin by digest — this action holds contents+pull_request+issues token;
|
|
# an unpinned tag is a supply-chain risk. Update digest + renovate-version
|
|
# together when Renovate publishes a new release.
|
|
uses: renovatebot/github-action@8217b3fc286df088d7c27f3255fe8414463bc0fd # v46.1.15
|
|
with:
|
|
configurationFile: renovate.json
|
|
token: ${{ secrets.RENOVATE_TOKEN }}
|
|
renovate-version: "46.1.15"
|
|
env:
|
|
RENOVATE_PLATFORM: gitea
|
|
RENOVATE_ENDPOINT: https://gitea.example.com # replace with your Gitea URL
|
|
RENOVATE_REPOSITORIES: '["org/repo"]' # replace with your repo slug
|
|
LOG_LEVEL: info
|
|
```
|
|
|
|
> **Token:** `RENOVATE_TOKEN` must be a PAT on a dedicated bot account with scopes
|
|
> `contents` + `pull_request` + `issues`. **Do not reuse** `GITEA_TOKEN` — that variable
|
|
> is not auto-provided on self-hosted Gitea runners and must be manually created anyway;
|
|
> using a single broad token violates least-privilege. See ADR-041.
|
|
|
|
### Renovate Configuration
|
|
|
|
The `renovate.json` in the repo root carries only dependency rules — platform and
|
|
endpoint config is injected via `env:` in the workflow above. Keep the two concerns
|
|
separate so the config file remains portable.
|
|
|
|
```json
|
|
{
|
|
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
|
|
"osvVulnerabilityAlerts": true,
|
|
"dependencyDashboard": true,
|
|
"schedule": ["before 6am on monday"],
|
|
"vulnerabilityAlerts": {
|
|
"labels": ["security", "P1-high"]
|
|
},
|
|
"lockFileMaintenance": {
|
|
"enabled": true,
|
|
"schedule": ["before 6am on monday"]
|
|
},
|
|
"packageRules": [
|
|
{
|
|
"matchPackageNames": ["com.example:my-dep"],
|
|
"automerge": true,
|
|
"matchUpdateTypes": ["patch"]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
> **Do not add `automerge: true` at the root.** Security and digest-bump PRs should
|
|
> always be reviewed manually. Per-rule `automerge` on patch-level routine deps is fine.
|
|
|
|
---
|
|
|
|
## Secrets Management -- age + git-crypt
|
|
|
|
**Replaces**: HashiCorp Vault (overkill), AWS Secrets Manager
|
|
**Cost**: free
|
|
**What it gives you**: For a small team, encrypted `.env` files committed to the repo using `age` encryption are sufficient. Each team member has a keypair; the `.env.encrypted` file is decryptable by all authorised keys.
|
|
|
|
### Usage
|
|
|
|
```bash
|
|
# Encrypt
|
|
age -r $(cat ~/.config/age/recipients.txt) -o .env.encrypted .env
|
|
|
|
# Decrypt (each team member)
|
|
age -d -i ~/.config/age/key.txt -o .env .env.encrypted
|
|
```
|
|
|
|
Keep `.env` in `.gitignore`. Commit `.env.encrypted` and `.env.example`.
|
|
|
|
---
|
|
|
|
## Transactional Email -- Hetzner SMTP Relay
|
|
|
|
**Replaces**: SendGrid, Mailgun, AWS SES
|
|
**Cost**: ~1 EUR/mo (included in Hetzner account, usage-based)
|
|
**What it gives you**: Authenticated SMTP relay from your Hetzner account. Simple configuration -- no SPF/DKIM setup nightmare. GDPR-compliant, EU-hosted.
|
|
|
|
### Configuration
|
|
|
|
```bash
|
|
# Production .env
|
|
MAIL_HOST=mail.your-server.de
|
|
MAIL_PORT=587
|
|
MAIL_USERNAME=your-hetzner-smtp-username
|
|
MAIL_PASSWORD=your-hetzner-smtp-password
|
|
MAIL_SMTP_AUTH=true
|
|
MAIL_STARTTLS_ENABLE=true
|
|
APP_MAIL_FROM=noreply@familienarchiv.example.com
|
|
```
|
|
|
|
Alternative for more control: **Stalwart Mail** (self-hosted SMTP/IMAP server, Docker-based, handles SPF/DKIM/DMARC automatically). Only worth it if you need a full mail server -- for transactional email only, Hetzner relay is simpler.
|