- BlockSource enum: MANUAL, OCR
- V26 migration adds source + reviewed columns to transcription_blocks
- OcrService sets source=OCR when creating blocks
- TranscriptionService.reviewBlock() toggles the reviewed flag
- PUT /api/documents/{id}/transcription-blocks/{blockId}/review endpoint
- 5 new tests: reviewBlock toggle/untoggle/notfound, controller,
OcrService source=OCR verification
The reviewed flag enables the Kraken fine-tuning pipeline: only blocks
marked as reviewed by a human are exported as training data.
Refs #226
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kraken's -f pdf mode tries to write output next to the input file,
which fails on read-only mounts. Instead, extract pages as PNGs via
pypdfium2 (already installed), then run kraken on each image.
Both models run in a single container per PDF to avoid overhead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kraken 7 requires pyvips (optional dep) for -f pdf mode.
Added libvips42 system package and pyvips Python package.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous approach used find across the htrmopo cache which failed
because -newer /tmp ran in a separate container. Now parses the
'Model dir: <path>' line from kraken get output directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Kraken 7 uses DOIs (not short names) to identify models from Zenodo.
Updated to use actual DOIs:
- 10.5281/zenodo.7933463 — German handwriting HTR
- 10.5281/zenodo.13788177 — McCATMuS generic handwritten/printed/typed
Added -f pdf flag for PDF input, volume mounts for import dir,
and post-download copy from htrmopo cache to the models volume.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
torchvision installed from PyPI expects CUDA torch operator
registrations. Installing from the CPU whl index ensures torchvision
matches the CPU-only torch build. Fixes 'torchvision::nms does not
exist' RuntimeError on startup.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
transformers 5.x breaks surya 0.17.1 — SuryaDecoderConfig is missing
pad_token_id. Pin to transformers>=4.56.1,<5.0.0.
Also add torch==2.7.1 to requirements.txt to prevent pip from upgrading
it past the CPU-only build installed in the Dockerfile layer.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Runbook script to download both HTR-United Kurrent model candidates
(german_kurrent_manu_9, kurrent-de) into the ocr_models Docker volume,
test them against sample documents, and activate the winner.
Usage:
./scripts/download-kraken-models.sh # download both
./scripts/download-kraken-models.sh --activate 1 # pick model 1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New confidence.py module with two functions:
- apply_confidence_markers(): replaces words below threshold with
[unleserlich], collapses adjacent markers into one
- words_from_characters(): reconstructs word-level confidence from
Kraken's character-level data
Surya 0.17 provides native word-level confidence via line.words.
Kraken 7.0 provides per-character confidences via record.confidences.
Both engines now pass word+confidence data through main.py, which
applies the marker post-processing before returning the API response.
Threshold configurable via OCR_CONFIDENCE_THRESHOLD env var (default 0.3).
Frontend already renders [unleserlich] markers via transcriptionMarkers.ts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
kraken 5.2.9 required torch~=2.1.0, incompatible with surya-ocr's
torch>=2.3.0. kraken 6.0.3 requires torch>=2.4.0,<=2.9 which
overlaps with surya and our pinned torch==2.5.1.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
surya-ocr 0.6.3 requires pillow<11.0.0,>=10.2.0. The previous
pin at 11.1.0 caused a dependency resolution failure during
Docker build.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace @Convert(PolygonConverter) with Hibernate native @JdbcTypeCode(SqlTypes.JSON)
to fix JDBC type mismatch — PostgreSQL requires jsonb type, not varchar.
The PolygonConverter is retained as a standalone utility but no longer
used on the entity. Hibernate 6 natively handles List<List<Double>>
serialization to JSONB.
Refs #227
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- AnnotationShape.svelte: renders a single annotation as either a
rectangle or a polygon-clipped div (via CSS clip-path: polygon())
- AnnotationLayer.svelte: refactored to delegate rendering to
AnnotationShape, keeping draw logic and hover state management
- Annotation type: added optional polygon field ([number, number][] | null)
- Polygon coordinates are converted from page-normalized to
bounding-box-relative percentages for clip-path
All 687 existing frontend tests pass.
Refs #227
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Python microservice (ocr-service/):
- FastAPI app with /ocr and /health endpoints
- Surya engine: transformer-based OCR for typewritten/modern handwriting
- Kraken engine: historical HTR for Kurrent/Suetterlin with
pure-Python polygon-to-quad approximation (gift wrapping + rotating calipers)
- Eager model loading at startup via lifespan context manager
- PDF download via httpx, page rendering via pypdfium2 at 300 DPI
Java RestClientOcrClient:
- Implements OcrClient + OcrHealthClient interfaces
- Calls Python service via Spring RestClient
- Health check with graceful fallback
Docker Compose:
- New ocr-service container (mem_limit 6g, no host ports)
- Health check with start_period 60s for model loading
- ocr_models volume for Kraken model files
- Backend depends on ocr-service health
Refs #226, #227
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OCR creates many adjacent text line annotations that would fail the
existing overlap check. createOcrAnnotation() accepts an optional
polygon and bypasses overlap detection entirely.
Refs #227
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ADR-001 documents the decision to use a separate Python container for
OCR (Surya + Kraken), the interface contract, and why alternatives
like Tess4J were rejected.
ADR-002 documents the decision to store polygon annotations as JSONB
with a 4-point CHECK constraint, backed by an AttributeConverter.
Refs #226, #227
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- CommentThread: add missing empty-state paragraph using comment_empty_hint
i18n key (key existed but was never rendered in the template)
- TranscriptionBlock: add selectedQuote hint using transcription_block_quote_hint
i18n key (key existed but was never rendered); fix test to use native DOM
el.focus()/setSelectionRange()/dispatchEvent instead of locator.selectText()
which is not available in this vitest-browser version
- TranscriptionEditView: fix test to use native el.dispatchEvent(FocusEvent)
instead of locator.blur() which is not available
- Conversations: fix test expected text from stale "Korrespondenz durchsuchen"
to match current conv_empty_heading() = "Wessen Briefe möchten Sie lesen?"
All 687 tests now pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
resolveRef is never read reactively — it is only read synchronously
inside settle(). Using $state was misleading about the intent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add m-auto and w-full to ensure the native <dialog> is centred
- Add backdrop:bg-black/50 for dimmed overlay when modal is open
- Add hover:bg-danger/80 and hover:bg-primary/80 on confirm button
- Add cursor-pointer to both cancel and confirm buttons
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
provideConfirmService() sets up context for the entire component tree.
ConfirmDialog is mounted once at the bottom of the layout shell.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- confirm.svelte.ts: context-based async service returning Promise<boolean>
- ConfirmDialog.svelte: native <dialog> element, reads service from context
- Concurrent calls return false immediately (guard at top of confirm())
- SSR-safe: confirm() returns Promise.resolve(false) on server
- getConfirmService() throws descriptive error outside provider tree
- 5 Vitest tests: confirm/cancel/Escape/concurrent/outside-provider all green
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Light: #c0392b (5.1:1 on white — WCAG AA), dark: #e55347 (4.7:1 on surface).
Exposed as bg-danger/text-danger-fg Tailwind utilities via @theme inline.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers segmented type control, title input, conditional field
visibility, PersonCard title display, mobile layout, and a11y.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add "amt" and "schule" suffixes to INSTITUTION_END in PersonTypeClassifier
so German government offices and schools are auto-classified on import.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add title to PersonUpdateDTO with @Size(max=50) constraint.
PersonService.createPerson and updatePerson now handle the title
field with blank-to-null normalization.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Person list and detail page avatars now display a type-specific
icon (building, people group, question mark) instead of meaningless
initials for INSTITUTION, GROUP, and UNKNOWN person types.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace hardcoded Tailwind utility colors with project CSS variables
(--c-badge-institution-*, --c-badge-group-*, --c-badge-unknown-*).
Dark mode variants defined in both @media and manual toggle blocks.
Extract shared badge classes and use $derived config object.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Testcontainers test verifying: SKIP returns null with no DB record,
INSTITUTION/GROUP store full name in lastName with null firstName
and correct personType, PERSON splits name normally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Remove Architekt from WORD_PREFIXES (classifier handles it)
- Use Objects.equals for null-safe firstName/lastName comparison
- Remove unused trimmed variable in PersonTypeClassifier
- Fix containsWord to loop through all occurrences (finds
"Eltern" in "Nachbareltern Eltern")
- Extract DisplayNameFormatter utility shared by Person and
PersonSummaryDTO to eliminate display logic duplication
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add @Nullable annotation to findOrCreateByAlias() return type.
Filter null results (from SKIP classification) in MassImportService
receiver list to prevent null elements in the receivers collection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two-pass title stripping with loop for stacked titles:
- Dot-prefixes (Dr., Prof.) matched without trailing space
- Word-prefixes (Tante, Frau, Schwester, etc.) matched at
word boundary
- Stacked titles like "Prof. Dr. Muller" handled correctly
- Single token after title strip goes to lastName (not firstName)
Add 5 "von" last names to KNOWN_LAST_NAMES for correct splitting
of entries like "Freifrau von Massenbach".
15 new test cases + updated 3 existing tests for title behavior.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Show colored badge for non-PERSON types per design spec:
- INSTITUTION: blue with building icon
- GROUP: purple with people icon
- UNKNOWN: amber with question mark icon
- PERSON: no badge (unmarked default)
Badge appears on person cards in list and on detail page.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Classify raw name before processing. SKIP returns null (no Person
created). INSTITUTION/GROUP skip split() and store full name in
lastName with firstName=null and appropriate personType.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
