Files

Marcel bead6f1811 fix(ocr): handle empty-string HTRMOPO_DIR env var with or-fallback

os.environ.get(key, default) returns "" when the key exists but is blank —
the default is only used when the key is absent. The or-fallback treats both
absence and blank values as "use the default".

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

2026-05-17 18:53:26 +02:00

__pycache__

refactor(document): move document domain core to document/ package

2026-05-05 12:39:20 +02:00

.venv

refactor(document): move document domain core to document/ package

2026-05-05 12:39:20 +02:00

dictionaries

feat(ocr): add DTA-derived historical German wordlist and generation script

2026-04-17 16:48:26 +02:00

engines

refactor(document): move document domain core to document/ package

2026-05-05 12:39:20 +02:00

.dockerignore

fix(docker): soften ocr-service dependency and clean up compose

2026-04-13 12:29:21 +02:00

CLAUDE.md

docs(legibility): fix three review blockers in DOC-7

2026-05-06 07:41:02 +02:00

confidence.py

refactor(ocr): make collapse_adjacent_markers a public function

2026-04-17 17:20:31 +02:00

Dockerfile

security(ocr): redirect XDG cache and Torch home away from read-only HOME

2026-05-17 17:30:39 +02:00

ensure_blla_model.py

fix(ocr): handle empty-string HTRMOPO_DIR env var with or-fallback

2026-05-17 18:53:26 +02:00

entrypoint.sh

fix(ocr-service): add entrypoint that validates blla model format on startup

2026-04-14 21:17:53 +02:00

main.py

security(ocr): log warning on startup when running as root

2026-05-17 16:51:00 +02:00

models.py

feat(ocr): per-sender model registry and /train-sender endpoint

2026-04-17 18:05:39 +02:00

preprocessing.py

test(ocr): add resilience tests for tiny image and unexpected exception propagation

2026-04-17 15:16:17 +02:00

README.md

docs(legibility): add 18 per-domain README.md files (DOC-6)

2026-05-06 07:36:38 +02:00

requirements.txt

feat(ocr): add pyspellchecker dependency

2026-04-17 16:41:24 +02:00

spell_check.py

refactor(ocr): document > 50 frequency threshold rationale

2026-04-17 17:21:37 +02:00

test_confidence.py

feat(ocr): per-script-type confidence thresholds

2026-04-12 20:50:59 +02:00

test_engines.py

fix(ocr): guard Kraken block extraction against missing boundary/baseline

2026-04-23 09:33:03 +02:00

test_ensure_blla_model.py

fix(ocr): handle empty-string HTRMOPO_DIR env var with or-fallback

2026-05-17 18:53:26 +02:00

test_main.py

test(ocr): add startup root canary tests for main.py lifespan

2026-05-17 17:29:47 +02:00

test_preprocessing.py

test(ocr): add resilience tests for tiny image and unexpected exception propagation

2026-04-17 15:16:17 +02:00

test_sender_registry.py

refactor(ocr): mark _SenderModelRegistry.contains as private (_contains)

2026-04-17 21:26:46 +02:00

test_spell_check.py

test(ocr): decouple correction tests from exact library dictionary state

2026-04-17 17:23:09 +02:00

test_stream.py

feat(ocr): integrate preprocessing into stream and batch endpoints

2026-04-17 14:16:47 +02:00

test_training_auth.py

test(ocr): add /train-sender auth tests and run sender registry tests in CI

2026-04-17 21:14:27 +02:00

README.md

ocr-service

Python FastAPI microservice that performs the actual handwritten text recognition (HTR) and OCR. The Spring Boot backend orchestrates jobs; this service executes them.

What this service owns

Text recognition: Surya (typewritten text) and Kraken (Kurrent/Sütterlin historical handwriting)
Baseline layout analysis: Kraken BLLA model
Sender recognition: trained per-archive sender models
HTTP API at port 8000 (internal Docker network — no external port)

What this service does NOT own

Job lifecycle — tracked in the backend's ocr/ domain
MinIO storage — the service fetches PDFs via presigned URLs generated by the backend; it does not hold credentials
Transcription block storage — results are streamed back to the backend, which writes them to PostgreSQL

API endpoints

Endpoint	Auth	Purpose
`POST /ocr`	None (internal network only)	Run OCR on a PDF (presigned MinIO URL in request body)
`POST /train`	`X-Training-Token` header	Trigger sender-model training
`POST /segtrain`	`X-Training-Token` header	Trigger segmentation training
`GET /health`	None	Health check

Environment variables

Variable	Default	Required?	Sensitive?	Purpose
`TRAINING_TOKEN`	—	YES (prod)	YES	Guards `/train` and `/segtrain`. Do not leave empty in production.
`ALLOWED_PDF_HOSTS`	`minio,localhost,127.0.0.1`	YES	—	SSRF protection — comma-separated allowed PDF source hosts. Never set to `*`.
`KRAKEN_MODEL_PATH`	`/app/models/`	—	—	Directory where Kraken HTR models are stored (populated by `download-kraken-models.sh`)
`BLLA_MODEL_PATH`	`/app/models/blla.mlmodel`	—	—	Kraken baseline layout analysis model. Auto-downloaded via `ensure_blla_model.py` on startup if missing.

Key files

File	Purpose
`main.py`	FastAPI app, endpoint definitions, SSRF validation
`engines/`	Surya and Kraken engine wrappers
`models.py`	Pydantic request/response models
`preprocessing.py`	PDF-to-image conversion before OCR
`confidence.py`	Per-block confidence scoring
`spell_check.py`	Post-OCR spell correction using historical dictionaries
`ensure_blla_model.py`	Startup script that downloads the BLLA model if missing
`entrypoint.sh`	Docker entrypoint — runs `ensure_blla_model.py` then starts the server

Backend counterpart

backend/src/main/java/org/raddatz/familienarchiv/ocr/README.md