Commit Graph

898 Commits

Author SHA1 Message Date
Marcel
ff9ae198c4 refactor(notifications): extract useNotificationStream and NotificationDropdown from NotificationBell (#200)
Some checks failed
CI / Unit & Component Tests (push) Failing after 2m38s
CI / Backend Unit Tests (push) Failing after 2m50s
CI / Unit & Component Tests (pull_request) Failing after 2m30s
CI / Backend Unit Tests (pull_request) Failing after 2m48s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 14:54:55 +02:00
Marcel
8898863a48 refactor(transcription): extract useBlockAutoSave and useBlockDragDrop from TranscriptionEditView (#199)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 14:45:03 +02:00
Marcel
eb8aa92cf0 refactor(pdf): extract usePdfRenderer and PdfControls from PdfViewer (#196)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 14:34:26 +02:00
Marcel
bc3fec11a9 refactor(comments): extract CommentMessage component from CommentThread (#198)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 14:23:25 +02:00
Marcel
fe6c247882 refactor(admin): extract EntityNavSection to eliminate nav markup repetition (#197)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:54:42 +02:00
Marcel
accfa5373e refactor(unsaved): extract createUnsavedWarning hook and UnsavedWarningBanner
Move the identical isDirty / beforeNavigate / discard pattern out of the
three admin detail pages (groups, tags, users) into a reusable
createUnsavedWarning() hook and a UnsavedWarningBanner presentational
component.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:31:17 +02:00
Marcel
34e7436fdc refactor(fileloader): extract createFileLoader hook from document/enrich pages
Move blob URL lifecycle management into a reusable createFileLoader()
hook that owns revoke-before-create and revoke-on-destroy. Replace
identical inline logic in documents/[id] and enrich/[id] with the hook.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:20:32 +02:00
Marcel
dbf7f0bc16 fix(fileloader): revoke blob URLs before re-assignment and on destroy
Calling loadFile a second time previously leaked the previous object URL.
Add URL.revokeObjectURL(fileUrl) before creating a new one and in
onDestroy so all URLs are freed. Revoke behavior will be covered by the
useFileLoader hook tests in the next commit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:13:21 +02:00
Marcel
8be876492c refactor(date): consolidate formatDate in date.ts with optional format param
Add format?: 'short'|'long' (default 'long') to date.ts formatDate and
remove the duplicate from personFormat.ts. Update DocumentTopBar to
import from date.ts directly. Move the formatDate tests from
personFormat.spec to date.spec.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:10:44 +02:00
Marcel
76d6f234b4 refactor(personFormat): replace getInitials(Person) with getInitials(name: string)
Unify the initials-extraction logic: the new string-based getInitials()
splits on whitespace, takes the first char of the first and last word
uppercased — matching the pattern that was already inlined in
CommentThread. Update PersonChip, DocumentMetadataDrawer, and
CommentThread to use the shared function.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:07:23 +02:00
Marcel
655a2003cb refactor(time): extract relativeTime into shared time.ts utility
Move relativeTime from notifications.ts (Intl.RelativeTimeFormat) to a
new time.ts that uses the Paraglide comment_time_* message keys — the
same logic that was already in CommentThread's timeAgo(). Remove the
duplicate timeAgo() from CommentThread and re-export relativeTime from
notifications.ts for backwards compatibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 13:02:49 +02:00
Marcel
c50845bcfc refactor(bell): migrate attachClickOutside to use:clickOutside action (#195)
Replace the inline attachClickOutside attachment in NotificationBell with
the shared use:clickOutside action from $lib/actions/clickOutside. The
inline implementation was functionally identical to the existing action.

Guard the onclickoutside handler so it only calls closeDropdown() when
the notification panel is already open, preventing the bell button from
stealing focus from other interactive elements (e.g. the user avatar menu).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 12:55:29 +02:00
Marcel
4446e80875 test(actions): add defaultPrevented coverage for clickOutside (#195)
The action already checks event.defaultPrevented before dispatching
clickoutside, but that branch had no test. Add the missing case and
add a one-line comment explaining why capture phase is used.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 12:46:04 +02:00
Marcel
731cdc75ab refactor(frontend): delete dead conversations/ route (#193)
Remove the old conversations page that was superseded by briefwechsel/.
No navigation link pointed to /conversations; it was unreachable through
the UI. Deletes 5 files, removes 14 orphaned i18n keys from de/en/es
message bundles, and removes E2E tests that navigated to /conversations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 12:43:40 +02:00
Marcel
4b8e0637ce fix(ci): pin DOCKER_API_VERSION=1.43 for Testcontainers on NAS runner
Some checks failed
CI / Unit & Component Tests (pull_request) Has been cancelled
CI / Backend Unit Tests (pull_request) Has been cancelled
CI / Unit & Component Tests (push) Successful in 3m41s
CI / Backend Unit Tests (push) Failing after 2m41s
Testcontainers 2.0.2 (via Spring Boot 4.0) negotiates Docker API 1.44,
but the NAS runner has Docker Engine 24.x which caps at 1.43. Forcing
the client version down unblocks tests until Docker is upgraded on the NAS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 12:28:57 +02:00
Marcel
793e632889 fix(lint): exclude project.inlang/ from Prettier
Some checks failed
CI / Unit & Component Tests (push) Successful in 3m49s
CI / Backend Unit Tests (push) Failing after 2m42s
CI / Unit & Component Tests (pull_request) Successful in 3m46s
CI / Backend Unit Tests (pull_request) Failing after 2m42s
Inlang regenerates .meta.json and README.md on every compilation run.
The regenerated files fail Prettier in CI because the tool writes its
own formatting, not ours.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 12:16:16 +02:00
Marcel
305f95a572 test(search): add sender name FTS coverage and combined filter test
Some checks failed
CI / Unit & Component Tests (push) Failing after 3s
CI / Backend Unit Tests (push) Failing after 1s
CI / Unit & Component Tests (pull_request) Failing after 1m57s
CI / Backend Unit Tests (pull_request) Failing after 3m0s
- should_find_document_by_sender_name — symmetric with existing receiver test
- fts_combined_with_status_filter_excludes_non_matching_status — verifies
  hasIds(rankedIds).and(hasStatus(...)) two-phase search works together

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:35:30 +02:00
Marcel
43595aeb8a refactor(search): replace O(n²) indexOf with HashMap for rank ordering
ids.indexOf() scans the full list for each document, giving O(n²) total.
Build a Map<UUID, Integer> once at O(n) and use getOrDefault at O(1) per
document. Behavior is identical; existing tests remain green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:35:30 +02:00
Marcel
947d8aeb6c fix(search): respect DATE sort when text is present — do not override with relevance
When a user explicitly selects DATE sort with a text query active, the
previous code treated it identically to RELEVANCE, silently discarding
the user's sort choice. Remove DATE from the useRankOrder condition so
that explicit DATE sort always goes through the standard JPA sort path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:35:30 +02:00
Marcel
7ec3e6170d feat(fts): backfill search_vector for all existing documents (V35)
Fires the BEFORE UPDATE trigger for every documents row, which recomputes
the tsvector from all currently-linked metadata, blocks, receivers, and tags.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:35:30 +02:00
Marcel
7d456d8e8b feat(fts): replace ILIKE hasText with FTS two-phase search and RELEVANCE sort
- DocumentSort: add RELEVANCE enum value
- DocumentSpecifications: remove hasText() ILIKE, add hasIds(List<UUID>)
  for FTS-pre-filtered ID sets
- DocumentService.searchDocuments(): FTS two-phase path — findRankedIdsByFts()
  returns ranked UUIDs, hasIds() narrows subsequent Specification query,
  in-memory re-sort preserves rank order; RELEVANCE is the default when
  text is present and no explicit non-relevance sort is requested
- DocumentSpecificationsTest: remove hasText() tests (Specification removed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:35:30 +02:00
Marcel
24530cf85b feat(fts): add search_vector column, GIN index, DB triggers, and FTS repository method (V34)
- V34 migration: adds search_vector tsvector column with GIN index
- BEFORE INSERT/UPDATE trigger on documents rebuilds vector from title (A),
  summary + transcription_blocks.text (B), sender/receiver names (C),
  tag names + location (D) using german FTS config
- AFTER triggers on transcription_blocks, document_receivers, document_tags
  touch the parent document row to re-fire the BEFORE UPDATE trigger
- DocumentRepository.findRankedIdsByFts() native query using websearch_to_tsquery
- DocumentFtsTest: 12 integration tests covering stemming, trigger sync,
  ranking, stop words, malformed input, receiver and tag search

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:35:16 +02:00
Marcel
57c44cf02f devops(backend): reduce healthcheck start_period to 30s
Some checks failed
CI / Unit & Component Tests (push) Failing after 2s
CI / Backend Unit Tests (push) Failing after 1s
With a pre-built JAR, Spring Boot + Flyway starts in ~15 seconds.
The previous 60s was sized for runtime compilation (90+ seconds).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:33:03 +02:00
Marcel
48223d5a3d devops(backend): pin eclipse-temurin tags, skip test compilation, document jar glob
- Pin to eclipse-temurin:21.0.10_7-{jdk,jre}-noble for reproducible builds
- Switch -DskipTests to -Dmaven.test.skip=true: skips test compilation entirely,
  not just execution — faster and avoids build failures from test-only missing classes
- Add comment on COPY *.jar explaining why the glob is safe (Spring Boot renames
  the pre-repackage artifact to .jar.original, leaving only one .jar in target/)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:33:03 +02:00
Marcel
04069c0286 devops(backend): add .dockerignore to exclude target/ from build context
Prevents 111MB of compiled output from being sent to the BuildKit daemon
on cold builds. Only .mvn/, mvnw, pom.xml, and src/ are needed by the
three COPY instructions in the Dockerfile.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:33:03 +02:00
Marcel
3c46d820ad devops(backend): switch to multi-stage Docker build
Replace runtime mvn spring-boot:run with a proper multi-stage build:
- Stage 1 (builder): compiles JAR with BuildKit cache mount for ~/.m2
- Stage 2 (runtime): eclipse-temurin:21-jre with only the JAR

Removes the backend source volume mount and maven_cache named volume.
Deploy with: docker compose up -d --build

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 11:33:03 +02:00
Marcel
38d558182a refactor(conversations): migrate ConversationTimeline to groupDocuments
Some checks failed
CI / Unit & Component Tests (pull_request) Failing after 1s
CI / Backend Unit Tests (pull_request) Failing after 2s
CI / Unit & Component Tests (push) Failing after 3s
CI / Backend Unit Tests (push) Failing after 2s
Replace hand-rolled enrichedDocuments year-divider logic with the shared
groupDocuments utility. Also fixes a timezone bug in documentYears: adds
'T12:00:00' to date strings so getFullYear() doesn't drift on UTC boundaries.
No behavior change — year dividers render the same way as before.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 09:41:40 +02:00
Marcel
25aa05411f fix(server): allowlist dir param in page.server.ts
Mirrors the existing sort allowlist pattern. Any value other than 'asc' or
'desc' silently falls back to 'desc', preventing arbitrary strings from
reaching the search API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 09:39:24 +02:00
Marcel
f522ab633c fix(a11y): bump GroupDivider contrast and add separator role
text-xs text-ink/40 (~2.1:1) fails WCAG AA; text-sm bold at text-ink/60
(~3.7:1) passes the large-text 3:1 threshold. Also adds role="separator"
and aria-label so screen readers announce the group boundary.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 09:38:37 +02:00
Marcel
593a6c8a38 test+fix(docs): correct fallbackLabel when sort prop is omitted
Add failing test for DATE-sort + undated doc showing "Undatiert" fallback
label, then fix DocumentList by null-coalescing sort before comparison
((sort ?? 'DATE') === 'DATE'). Test uses one dated + one undated doc to
produce two groups and trigger GroupDivider rendering.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 09:37:19 +02:00
Marcel
67c03dab8c feat(search): wire sort to DocumentList; validate sort param allowlist
Some checks failed
CI / Unit & Component Tests (push) Failing after 3s
CI / Backend Unit Tests (push) Failing after 0s
CI / Unit & Component Tests (pull_request) Failing after 2s
CI / Backend Unit Tests (pull_request) Failing after 1s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:00:09 +02:00
Marcel
e302d3d689 feat(search): add group headers to DocumentList by sort field
Documents sorted by DATE show year dividers, SENDER/RECEIVER sort
shows person name dividers. Dividers only appear when there are 2+
distinct groups. Multi-receiver docs appear in each receiver group.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 07:59:02 +02:00
Marcel
a9aa1ec924 feat(search): add groupDocuments utility with unit tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:36:35 +02:00
Marcel
ce2bbf4230 refactor(conversations): use GroupDivider in ConversationTimeline
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:35:09 +02:00
Marcel
69bcb3f8b2 feat(search): add GroupDivider shared component
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:24:48 +02:00
Marcel
34a97cbfa2 i18n: add docs_group_undated and docs_group_unknown translation keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:21:43 +02:00
Marcel
3d3d4b8616 chore: add Claude personas, skills, memory, and project docs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:21:15 +02:00
Marcel
e4719b9487 fix(deploy): increase OCR healthcheck start_period, comment ocr_cache volume, add token hint
Some checks failed
CI / Unit & Component Tests (push) Failing after 2s
CI / Backend Unit Tests (push) Failing after 1s
- start_period 60s → 120s: Zenodo download on cold start can exceed 60s on slow connections
- ocr_cache volume comment: documents what the cache stores for future operators
- .env.example: add token generation command to prevent weak placeholder in production

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
7562a400c0 test(frontend): add Vitest component tests for TrainingHistory expand/collapse
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
2073a4b64a fix(frontend): accessibility fixes for TrainingHistory expand/collapse and FAILED badge
- Add aria-expanded + aria-controls to expand button (WCAG 4.1.2)
- Add id="training-history-rows" to tbody for aria-controls target
- Replace title= tooltip on FAILED badge with details/summary for keyboard
  and touch accessibility; add training_error_detail_label i18n key
- Use motion-safe:animate-pulse on RUNNING badge for prefers-reduced-motion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
5c7efef307 fix(ocr): pin Dockerfile base image to python:3.11.9-slim for reproducible builds
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
74c9046745 fix(ocr): narrow exception handling and add unit tests for ensure_blla_model
- _model_is_loadable: narrow bare except to (RuntimeError, OSError, ValueError)
  with DEBUG-level fallback for unexpected exceptions — prevents silent masking
  of missing kraken install or AttributeError on vgsl
- _run_segtrain: replace bare except:pass with log.warning so height-check
  fallback is visible in container logs
- New test_ensure_blla_model.py: covers model-OK early return, incompatible
  model rename+replace, and missing model download paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
81da127381 refactor(ocr): rename findTop5 to findTop10 for headroom as frontend shows 3 by default
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
f206c0b9e9 test(ocr): add unit tests for triggerSegTraining() — conflict, threshold, happy path, failure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
15e532eb96 refactor(ocr): extract assertNoRunningTraining() to eliminate duplicate guard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
f241a71733 feat(frontend): limit training history to 3 runs with expand toggle
Both training panels (OCR and segmentation) share TrainingHistory.
Show only the 3 most recent runs by default; render a Mehr/Weniger
anzeigen button when there are more.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
b83465020a fix(backend): store error rate for segmentation training runs
setCer() was called for recognition training but not for segmentation.
The OCR service now returns cer = 1 - accuracy for segtrain; persist it
so the admin panel can display Fehlerrate for both training types.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
f08897b801 fix(deploy): wire OCR training token to backend and raise container memory limit
- Pass OCR_TRAINING_TOKEN through to the backend container as
  APP_OCR_TRAINING_TOKEN so RestClientOcrClient sends the X-Training-Token
  header when calling /train and /segtrain.
- Raise mem_limit/memswap_limit from 8g to 12g to give segtrain headroom
  on hosts with more available RAM.
- Uncomment OCR_TRAINING_TOKEN in .env.example — it is now required.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
a5979c4069 fix(ocr-service): fix ketos 7 segtrain compatibility and prevent OOM
Three issues fixed:

1. --resize both was removed in ketos 7; replaced with --resize union
   which extends the model's class mapping to include training data classes.

2. ketos ignores -s when -i is present, so the 1800px blla model caused
   7+ GB peak RAM and OOM-killed the host (no swap, 5 GB free).
   Now checks the loaded model's input height: only uses the base model
   when it was already fine-tuned at 800px; otherwise trains from scratch
   at 800px (~200 MB peak). After the first run the trained 800px model
   becomes the base for all subsequent fine-tuning runs.

3. segtrain now computes and returns cer = 1 - accuracy, matching the
   recognition training path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00
Marcel
e8375d6c72 fix(ocr-service): add entrypoint that validates blla model format on startup
Adds ensure_blla_model.py which loads the blla segmentation model with
ketos on every container start. If the model is missing or in the legacy
PyTorch ZIP format (incompatible with ketos 7), it re-downloads the
correct CoreML protobuf model from Zenodo (DOI 10.5281/zenodo.14602569).
The Dockerfile now uses entrypoint.sh which runs this check before
starting uvicorn.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:17:53 +02:00