From aedae2a774ede093180f63f5d6d9ff49178af382 Mon Sep 17 00:00:00 2001 From: Marcel Date: Tue, 5 May 2026 22:48:28 +0200 Subject: [PATCH 1/3] =?UTF-8?q?docs(legibility):=20update=20c4-diagrams.md?= =?UTF-8?q?=20L2=20=E2=80=94=20add=20ocr-service,=20SSE,=20presigned=20URL?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs #396 Co-Authored-By: Claude Sonnet 4.6 --- docs/architecture/c4-diagrams.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/architecture/c4-diagrams.md b/docs/architecture/c4-diagrams.md index 7cf79d9d..a5b2b22b 100644 --- a/docs/architecture/c4-diagrams.md +++ b/docs/architecture/c4-diagrams.md @@ -34,9 +34,11 @@ C4Container System_Boundary(archiv, "Familienarchiv (Docker Compose)") { Container(frontend, "Web Frontend", "SvelteKit / Node.js", "Server-side rendered UI. Handles session cookies, search UI, document viewer, and admin panel.") - Container(backend, "API Backend", "Spring Boot 4 / Java 21 / Jetty", "REST API. Implements document management, search, user auth, file upload/download, and Excel import.") + Container(backend, "API Backend", "Spring Boot 4 / Java 21 / Jetty", "REST API. Implements document management, search, user auth, file upload/download, transcription, OCR orchestration, and SSE notifications.") - ContainerDb(db, "Relational Database", "PostgreSQL 16", "Stores document metadata, persons, users, permission groups, tags, and Spring Session data.") + Container(ocr, "OCR Service", "Python FastAPI / port 8000", "Handwritten text recognition (HTR) and OCR microservice. Single-node by design — see ADR-001. Reachable only on the internal Docker network; no external port exposed.") + + ContainerDb(db, "Relational Database", "PostgreSQL 16", "Stores document metadata, persons, users, permission groups, tags, transcription blocks, audit log, and Spring Session data.") ContainerDb(storage, "Object Storage", "MinIO (S3-compatible)", "Stores the actual document files (PDFs, scans). Objects keyed as documents/{UUID}_{filename}.") @@ -45,8 +47,11 @@ C4Container Rel(user, frontend, "Uses", "HTTPS / Browser") Rel(frontend, backend, "API requests with Basic Auth token", "HTTP / REST / JSON") + Rel(backend, user, "SSE notifications (server-sent events)", "HTTP / SSE — direct backend-to-browser") Rel(backend, db, "Reads and writes metadata and sessions", "JDBC / SQL") Rel(backend, storage, "Uploads and streams document files", "HTTP / S3 API (AWS SDK v2)") + Rel(backend, ocr, "OCR job requests with presigned MinIO URL", "HTTP / REST / JSON") + Rel(ocr, storage, "Fetches PDF via presigned URL", "HTTP / S3 presigned") Rel(mc, storage, "Creates bucket on startup", "MinIO Client CLI") ``` -- 2.49.1 From dc6910a31e9526294f23544645b2569974dc7d20 Mon Sep 17 00:00:00 2001 From: Marcel Date: Tue, 5 May 2026 22:49:02 +0200 Subject: [PATCH 2/3] docs(legibility): write docs/ARCHITECTURE.md Human-targeted architecture doc: high-level diagram, 7 Tier-1 + 2 Tier-2 domains, cross-cutting layer, stack-symmetry principle, 6 ADR summaries, layering rule, permission system, and two data-flow walkthroughs (document upload, transcription block autosave). Closes #396 Co-Authored-By: Claude Sonnet 4.6 --- docs/ARCHITECTURE.md | 146 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 docs/ARCHITECTURE.md diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 00000000..660bedaa --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,146 @@ + + +# Familienarchiv — Architecture + +**Target reader:** a PM-with-CS background who has read the README. +**Goal:** accurate mental model after one read — enough to sketch the system on a whiteboard. + +For domain terminology, see [docs/GLOSSARY.md](GLOSSARY.md). +For security policies and hardening, see [docs/security-guide.md](security-guide.md). +For low-level ADR details, see [docs/adr/](adr/). + +--- + +## 1. High-level diagram + +The updated container diagram below shows all six deployable units and their communication paths. + +See [docs/architecture/c4-diagrams.md](architecture/c4-diagrams.md) for the full C4 L1/L2/L3 diagrams (Mermaid, Gitea-rendered). + +Key points not visible in the diagram: + +- **OCR network boundary:** the OCR service has no external port — it is reachable only on the internal Docker Compose network. Only the backend calls it. The OCR service fetches PDF files from MinIO using a presigned URL that the backend generates and passes in the request body; the PDF bytes never pass through the backend. +- **SSE path:** server-sent event notifications go directly from the backend to the user's browser (not via the SvelteKit SSR layer) over a long-lived HTTP connection managed by `SseEmitterRegistry`. + +--- + +## 2. Domain set + +Both stacks are organised **package-by-domain**: each domain owns its entities, service, controller, repository, and DTOs. Domain names are identical across `backend/src/main/java/.../` and `frontend/src/lib/`. + +### Tier-1 domains — have entities and user-facing CRUD + +**`document`** — the archive's core concept. Owns `Document`, `DocumentVersion`, `TranscriptionBlock`, `DocumentAnnotation`, `DocumentComment`. Does NOT own persons or tags (references them by ID). Cross-domain deps: `person` (sender/receivers), `tag` (labels), `ocr` (HTR pipeline), `notification` (comment mentions), `audit` (every mutation). + +**`person`** — historical individuals referenced by documents. Owns `Person`, `PersonNameAlias`, `PersonRelationship`. Does NOT own `AppUser` (login accounts are a separate domain). Cross-domain deps: `document` (relationship queries). + +**`tag`** — hierarchical document categories. Owns `Tag` (self-referencing `parent_id` tree). Does NOT own documents; the join is document-side. No cross-domain deps. + +**`user`** — login accounts and permission groups. Owns `AppUser`, `UserGroup`, invite tokens. Does NOT own `Person` records. Cross-domain deps: `audit` (user management events). + +**`geschichte`** — family stories. Owns `Geschichte` (`DRAFT → PUBLISHED` lifecycle). Cross-domain deps: `person`, `document` (linked entities in the story body). + +**`notification`** — in-app messages. Owns `Notification`. Delivers via `SseEmitterRegistry` (live) and persisted rows (bell dropdown). Cross-domain deps: `user` (recipient), `document` (context). + +**`ocr`** — OCR/HTR pipeline orchestration. Owns `OcrJob`, `OcrJobDocument`, `SenderModel`. Calls the Python OCR service; maps streamed transcription blocks back to `document`. Cross-domain deps: `document` (target), `filestorage` (presigned URLs). + +### Tier-2 domains — derived (UI without dedicated tables) + +A **derived domain** has its own routes and UI but no database tables of its own; it is assembled from data owned by Tier-1 domains. + +**`conversation`** (route: `/briefwechsel`) — bilateral letter timeline between two `Person`s. Derived from `Document` sender/receiver relationships. The `DocumentRepository` bidirectional query is the only data source. + +**`activity`** (route: `/aktivitaeten`) — family activity feed. Derived from `audit_log`, `notifications`, and document events. No aggregation table; computed on-the-fly by `DashboardService` and composed in the SvelteKit load function. + +--- + +## 3. Cross-cutting layer + +Members of the cross-cutting layer have no entity of their own, no user-facing CRUD, and are consumed by two or more domains — or are framework infrastructure that every domain depends on. + +| Member (backend package) | Purpose | Admission criteria | +|---|---|---| +| `audit` | Append-only event store (`audit_log`) for all domain mutations. Feeds the activity feed and Family Pulse dashboard. | Consumed by 5+ domains; no user-facing CRUD of its own | +| `config` | Infrastructure bean definitions: `MinioConfig`, `AsyncConfig`, `WebConfig` | Framework infra; no business logic | +| `dashboard` | Stats aggregation for the admin dashboard and Family Pulse widget | Aggregates from 3+ domains; no owned entities | +| `exception` | `DomainException`, `ErrorCode` enum, `GlobalExceptionHandler` | Framework infra; consumed by every controller and service | +| `filestorage` | `FileService` — MinIO/S3 upload, download, presigned-URL generation | Generic service; consumed by `document` and `ocr` | +| `importing` | `MassImportService` — async ODS/Excel batch import | Orchestrates across `person`, `tag`, `document` | +| `security` | `SecurityConfig`, `Permission` enum, `@RequirePermission` annotation, `PermissionAspect` (AOP) | Framework infra; enforced globally across all controllers | + +**Frontend `shared/`** follows the same admission criteria. Key members: `api.server.ts` (typed openapi-fetch client factory), `errors.ts` (backend `ErrorCode` → i18n mapping), `shared/primitives/` (generic UI components used across ≥2 domains), `shared/discussion/` (comment/mention editor used by `document` and `geschichte`), `shared/utils/` (pure date/sort/debounce utilities). + +--- + +## 4. Stack-symmetry principle + +**Rule:** a domain has the same name on both stacks. + +| Backend | Frontend | +|---|---| +| `backend/src/main/java/.../document/` | `frontend/src/lib/document/` | +| `backend/src/main/java/.../person/` | `frontend/src/lib/person/` | +| … | … | + +Adding a new Tier-1 domain means creating a package on **both** sides under the same name. Adding only a backend package without a corresponding frontend folder (or vice versa) is a red flag in code review. + +The backend has been domain-first since the project started. The frontend `src/lib/` was restructured from flat-by-type to domain-first in issue #408 (May 2026). + +--- + +## 5. Key architectural decisions + +### ADR-001 — OCR as a Python microservice +The two OCR engines required (Surya for typewritten text, Kraken for Kurrent/Sütterlin HTR) exist only in the Python ecosystem. A separate `ocr-service` Python container exposes a simple HTTP API; the Spring Boot backend calls it via `RestClient`. All job tracking and business logic remain in Spring Boot. See [ADR-001](adr/001-ocr-python-microservice.md). + +### ADR-002 — Polygon JSONB storage for annotations +Kraken outputs polygon boundaries for historical handwriting; axis-aligned bounding boxes approximate them poorly. Annotation and transcription-block positions are stored as `polygon JSONB` columns. Display-only — server-side geometry continues to use the AABB fields. See [ADR-002](adr/002-polygon-jsonb-storage.md). + +### ADR-003 — Unified activity feed (Chronik/Aktivität) +Personal notifications and ambient activity (uploads, transcriptions, comments) are merged into one `/aktivitaeten` page. The SvelteKit load function composes data from `/api/dashboard/activity` and `/api/notifications` — no new backend orchestrator endpoint. See [ADR-003](adr/003-chronik-unified-activity-feed.md). + +### ADR-004 — In-process PDFBox thumbnails +Thumbnails are rendered in Spring Boot using Apache PDFBox (already a dependency) rather than delegating to the OCR service. A dedicated `thumbnailExecutor` pool isolates the work. See [ADR-004](adr/004-pdfbox-thumbnails.md). + +### ADR-005 — thumbnailAspect + pageCount +Aspect ratio (`PORTRAIT` / `LANDSCAPE`) and page count are persisted alongside the thumbnail JPEG at generation time — cheap to derive then, expensive to re-derive later. See [ADR-005](adr/005-thumbnail-aspect-and-page-count.md). + +### ADR-006 — Synchronous domain events inside the publisher's transaction +When a `Person` display name changes, all `TranscriptionBlock` `@mention` text must be rewritten atomically. This is done via Spring `ApplicationEventPublisher` + `@EventListener @Transactional` to avoid a circular dependency between `PersonService` and `TranscriptionBlockService`. See [ADR-006](adr/006-synchronous-domain-events-in-transaction.md). + +### Layering rule +``` +Controller → Service → Repository → DB +``` +Controllers never call repositories directly. Services never reach into another domain's repository — they call the other domain's service. This keeps domain boundaries clear and business logic testable without a running database. + +### Permission system +Permissions are enforced via `@RequirePermission(Permission.X)` on controller methods, checked at runtime by `PermissionAspect` (Spring AOP). The `Permission` enum defines the available capabilities (`READ_ALL`, `WRITE_ALL`, `ADMIN`, `ADMIN_USER`, `ADMIN_TAG`, `ADMIN_PERMISSION`, `BLOG_WRITE`). This is not Spring Security's `@PreAuthorize` — do not mix the two mechanisms. + +Sessions use a Base64-encoded Basic Auth token stored in an `httpOnly`, `SameSite=strict` cookie (`auth_token`, maxAge=86400 s). CSRF protection is disabled because this cookie configuration structurally prevents cross-origin credential theft. See [docs/security-guide.md](security-guide.md) for the full security reference. + +--- + +## 6. Data flow walkthroughs + +### Document upload + +1. User submits the edit form (file + metadata) from the browser. +2. SvelteKit server action sends `PUT /api/documents/{id}` as `multipart/form-data` with the `Authorization` header. +3. `PermissionAspect` intercepts the controller method, verifies the user has `WRITE_ALL`, and proceeds. +4. `DocumentController` delegates to `DocumentService.updateDocument()`. +5. `DocumentService` resolves the `Person` sender by ID (via `PersonService`), resolves or creates `Tag`s (via `TagService`), then calls `FileService.uploadFile()`. +6. `FileService` generates a key (`documents/{UUID}_{filename}`), streams the file to MinIO via the AWS SDK v2 S3Client. +7. `DocumentService` persists the S3 key, sets `status = UPLOADED`, and saves to PostgreSQL. +8. `AuditService` writes an `UPLOADED` event to `audit_log` in the same transaction. +9. Backend returns the updated `Document` JSON; SvelteKit refreshes the document detail page. + +### Transcription block autosave + +1. The transcriber pauses typing; the frontend's `useBlockAutoSave` hook fires after a debounce interval. +2. SvelteKit sends `PATCH /api/transcription/blocks/{id}` with the new text and the block's current `version` (optimistic lock). +3. `TranscriptionService.saveBlock()` loads the block, checks the `@Version` field for concurrent edits, updates `block.text` and any `@mention` sidecars, and calls `saveAndFlush`. +4. If a concurrent save collides (version mismatch), the backend returns `409 Conflict`; the frontend's `saveBlockWithConflictRetry` helper re-fetches and retries. +5. On success, `AuditService` logs a `BLOCK_SAVED` event. +6. If the block text contains a new `@PersonName` mention, `NotificationService` creates a `Notification` row for the mentioned person's `AppUser`. +7. `SseEmitterRegistry` broadcasts the notification over the open SSE connection to that user's browser in real time. -- 2.49.1 From 69601006284ee53e288d8309fd6665a605908b2e Mon Sep 17 00:00:00 2001 From: Marcel Date: Tue, 5 May 2026 22:57:45 +0200 Subject: [PATCH 3/3] docs(legibility): fix three factual errors in ARCHITECTURE.md - Add ANNOTATE_ALL to the Permission enum listing (was missing) - Fix transcription block autosave endpoint: PUT not PATCH, correct path /api/documents/{documentId}/transcription-blocks/{blockId} - Clarify auth injection: hooks.server.ts handleFetch injects the Authorization header, not the SvelteKit action directly Refs #396 Co-Authored-By: Claude Sonnet 4.6 --- docs/ARCHITECTURE.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 660bedaa..abef192b 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -115,7 +115,7 @@ Controller → Service → Repository → DB Controllers never call repositories directly. Services never reach into another domain's repository — they call the other domain's service. This keeps domain boundaries clear and business logic testable without a running database. ### Permission system -Permissions are enforced via `@RequirePermission(Permission.X)` on controller methods, checked at runtime by `PermissionAspect` (Spring AOP). The `Permission` enum defines the available capabilities (`READ_ALL`, `WRITE_ALL`, `ADMIN`, `ADMIN_USER`, `ADMIN_TAG`, `ADMIN_PERMISSION`, `BLOG_WRITE`). This is not Spring Security's `@PreAuthorize` — do not mix the two mechanisms. +Permissions are enforced via `@RequirePermission(Permission.X)` on controller methods, checked at runtime by `PermissionAspect` (Spring AOP). The `Permission` enum defines the available capabilities (`READ_ALL`, `WRITE_ALL`, `ADMIN`, `ADMIN_USER`, `ADMIN_TAG`, `ADMIN_PERMISSION`, `ANNOTATE_ALL`, `BLOG_WRITE`). This is not Spring Security's `@PreAuthorize` — do not mix the two mechanisms. Sessions use a Base64-encoded Basic Auth token stored in an `httpOnly`, `SameSite=strict` cookie (`auth_token`, maxAge=86400 s). CSRF protection is disabled because this cookie configuration structurally prevents cross-origin credential theft. See [docs/security-guide.md](security-guide.md) for the full security reference. @@ -126,7 +126,7 @@ Sessions use a Base64-encoded Basic Auth token stored in an `httpOnly`, `SameSit ### Document upload 1. User submits the edit form (file + metadata) from the browser. -2. SvelteKit server action sends `PUT /api/documents/{id}` as `multipart/form-data` with the `Authorization` header. +2. The SvelteKit server action sends `PUT /api/documents/{id}` as `multipart/form-data`. `hooks.server.ts` (`handleFetch`) transparently injects the `Authorization` header from the `auth_token` cookie — the action itself is unaware of auth. 3. `PermissionAspect` intercepts the controller method, verifies the user has `WRITE_ALL`, and proceeds. 4. `DocumentController` delegates to `DocumentService.updateDocument()`. 5. `DocumentService` resolves the `Person` sender by ID (via `PersonService`), resolves or creates `Tag`s (via `TagService`), then calls `FileService.uploadFile()`. @@ -137,8 +137,8 @@ Sessions use a Base64-encoded Basic Auth token stored in an `httpOnly`, `SameSit ### Transcription block autosave -1. The transcriber pauses typing; the frontend's `useBlockAutoSave` hook fires after a debounce interval. -2. SvelteKit sends `PATCH /api/transcription/blocks/{id}` with the new text and the block's current `version` (optimistic lock). +1. The transcriber pauses typing; the frontend's `useBlockAutoSave` factory fires after a debounce interval. +2. The browser sends `PUT /api/documents/{documentId}/transcription-blocks/{blockId}` with the new text and the block's current `version` (optimistic lock). `hooks.server.ts` (`handleFetch`) injects the `Authorization` header from the cookie. 3. `TranscriptionService.saveBlock()` loads the block, checks the `@Version` field for concurrent edits, updates `block.text` and any `@mention` sidecars, and calls `saveAndFlush`. 4. If a concurrent save collides (version mismatch), the backend returns `409 Conflict`; the frontend's `saveBlockWithConflictRetry` helper re-fetches and retries. 5. On success, `AuditService` logs a `BLOCK_SAVED` event. -- 2.49.1