bug(infra/fail2ban): jail defaults to systemd backend on Debian, never inspects Caddy access log #503

New Issue

marcel · 2026-05-11T14:58:50+02:00

marcel commented

2026-05-11 14:58:50 +02:00

Summary

infra/fail2ban/jail.d/familienarchiv.conf is missing backend = polling. On Debian/Ubuntu the package ships /etc/fail2ban/jail.d/defaults-debian.conf which sets [DEFAULT] backend = systemd. Without an override, our jail inherits the systemd backend, reads from journald, and never inspects /var/log/caddy/access.log — i.e. the brute-force protection is inert.

Reproduction

On a freshly bootstrapped server (Debian 13 / Ubuntu 24.04, fail2ban from apt, our jail+filter symlinked in):

$ fail2ban-client status familienarchiv-auth
Status for the jail: familienarchiv-auth
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- Journal matches:
`- Actions
   |- ...
$ fail2ban-client get familienarchiv-auth logpath
No file is currently monitored
$ fail2ban-client get familienarchiv-auth journalmatch
No journal match filter set
$ fail2ban-client -d | grep "add.*familienarchiv-auth"
['add', 'familienarchiv-auth', 'systemd']

The jail loads, but it monitors nothing. A real 401 against /api/auth/login would be written to /var/log/caddy/access.log and the jail would never see it.

Why the CI test did not catch this

.gitea/workflows/ci.yml runs fail2ban-regex against a sample line + the filter file. That tests the filter regex in isolation; it never instantiates a real jail with the real defaults-debian.conf inherited backend. So the regex passes (correctly) but the runtime behaviour is broken.

Fix

Add a single line to infra/fail2ban/jail.d/familienarchiv.conf:

backend = polling

with a comment explaining why (override the Debian default of systemd).

polling is the safest choice: no inotify dependency, works on any kernel, slightly higher CPU than auto/pyinotify (irrelevant for one log file).

Also worth adding to the CI test: a fail2ban-client -d | grep "add.*familienarchiv-auth.*polling" assertion so a future regression is caught.

Impact

Production: blocker. Brute-force protection does not work; Nora's HIGH from PR #499 review is effectively unmet on a live deploy until this lands.
Staging: non-blocker (no real users to brute-force).

Discovered

During the production deploy bootstrap for #497 — found after fail2ban came up cleanly but fail2ban-client get … logpath returned No file is currently monitored.

Labels

bug, devops

## Summary `infra/fail2ban/jail.d/familienarchiv.conf` is missing `backend = polling`. On Debian/Ubuntu the package ships `/etc/fail2ban/jail.d/defaults-debian.conf` which sets `[DEFAULT] backend = systemd`. Without an override, our jail inherits the systemd backend, reads from journald, and **never inspects `/var/log/caddy/access.log`** — i.e. the brute-force protection is inert. ## Reproduction On a freshly bootstrapped server (Debian 13 / Ubuntu 24.04, fail2ban from apt, our jail+filter symlinked in): ``` $ fail2ban-client status familienarchiv-auth Status for the jail: familienarchiv-auth |- Filter | |- Currently failed: 0 | |- Total failed: 0 | `- Journal matches: `- Actions |- ... $ fail2ban-client get familienarchiv-auth logpath No file is currently monitored $ fail2ban-client get familienarchiv-auth journalmatch No journal match filter set $ fail2ban-client -d | grep "add.*familienarchiv-auth" ['add', 'familienarchiv-auth', 'systemd'] ``` The jail loads, but it monitors nothing. A real 401 against `/api/auth/login` would be written to `/var/log/caddy/access.log` and the jail would never see it. ## Why the CI test did not catch this `.gitea/workflows/ci.yml` runs `fail2ban-regex` against a sample line + the filter file. That tests the filter regex in isolation; it never instantiates a real jail with the real defaults-debian.conf inherited backend. So the regex passes (correctly) but the runtime behaviour is broken. ## Fix Add a single line to `infra/fail2ban/jail.d/familienarchiv.conf`: ``` backend = polling ``` with a comment explaining why (override the Debian default of `systemd`). `polling` is the safest choice: no inotify dependency, works on any kernel, slightly higher CPU than `auto`/`pyinotify` (irrelevant for one log file). Also worth adding to the CI test: a `fail2ban-client -d | grep "add.*familienarchiv-auth.*polling"` assertion so a future regression is caught. ## Impact - **Production**: blocker. Brute-force protection does not work; Nora's HIGH from PR #499 review is effectively unmet on a live deploy until this lands. - **Staging**: non-blocker (no real users to brute-force). ## Discovered During the production deploy bootstrap for #497 — found after fail2ban came up cleanly but `fail2ban-client get … logpath` returned `No file is currently monitored`. ## Labels bug, devops

marcel referenced this issue from a commit

2026-05-11 15:00:15 +02:00

fix(fail2ban): pin polling backend so jail actually reads Caddy access log

marcel referenced a pull request that will close this issue

2026-05-11 15:00:31 +02:00

fix(fail2ban): pin polling backend so jail actually reads Caddy access log (#503) #504

marcel closed this issue

2026-05-11 15:08:59 +02:00

marcel referenced this issue from a commit

2026-05-11 15:09:00 +02:00

Merge pull request 'fix(fail2ban): pin polling backend so jail actually reads Caddy access log (#503)' (#504) from fix/issue-503-fail2ban-polling-backend into main

Sign in to join this conversation.