feat(documents): honest handling of undated documents in browse & search (Phase 6, #668) #682

Merged
marcel merged 19 commits from feature/668-undated-documents into docs/import-migration 2026-05-27 21:26:50 +02:00

19 Commits

Author SHA1 Message Date
Marcel
7183d15fe5 fix(document): restore pure-text-relevance FTS fast path past undated count
All checks were successful
CI / Unit & Component Tests (pull_request) Successful in 3m29s
CI / OCR Service Tests (pull_request) Successful in 25s
CI / Backend Unit Tests (pull_request) Successful in 3m52s
CI / fail2ban Regex (pull_request) Successful in 45s
CI / Semgrep Security Scan (pull_request) Successful in 20s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m3s
The global undated-count rework moved the pure-text-RELEVANCE shortcut
into runSearch, where it ran after the unconditional
findAllMatchingIdsByFts call. That routed pure-text relevance through the
in-memory id path and returned empty match data, breaking FTS rank order
and snippet/offset enrichment.

Hoist the shortcut back to the top of searchDocuments so it short-circuits
to findFtsPageRaw before findAllMatchingIdsByFts, while still computing the
global undatedCount for all non-fast-path searches.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 21:04:48 +02:00
Marcel
b52bf60913 fix(document): tie-break equal-date DATE sort by title asc, not createdAt
Some checks failed
CI / Unit & Component Tests (pull_request) Failing after 3m2s
CI / OCR Service Tests (pull_request) Successful in 24s
CI / Backend Unit Tests (pull_request) Failing after 3m54s
CI / fail2ban Regex (pull_request) Successful in 47s
CI / Semgrep Security Scan (pull_request) Successful in 20s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m6s
Owner decision (#668): when two documents share a meta_date, order them by
title ascending instead of createdAt ascending. title is @Column(nullable=false)
so it is always present, giving a deterministic, human-meaningful total order.
Only the DATE-sort fast path changes; the in-memory SENDER/RECEIVER/RELEVANCE
comparators are untouched.

ORDER BY meta_date <dir> NULLS LAST, title ASC

Tests assert title-asc tiebreaking for same-date rows in BOTH directions, with a
fixture whose title order is the OPPOSITE of insertion (createdAt) order so the
test fails if the tiebreaker reverts to createdAt. The integration test drives
the production resolveSort against real Postgres.

Refs #668
2026-05-27 20:21:18 +02:00
Marcel
45e63307bb fix(documents): give the undated count chip a self-describing a11y name
Some checks failed
CI / Unit & Component Tests (pull_request) Successful in 3m42s
CI / OCR Service Tests (pull_request) Successful in 25s
CI / Backend Unit Tests (pull_request) Failing after 3m46s
CI / fail2ban Regex (pull_request) Successful in 44s
CI / Semgrep Security Scan (pull_request) Successful in 21s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m3s
A screen reader announced the bare number ("Nur undatierte 42"). Add an
aria-label ("42 undatierte Dokumente") via a new i18n key and hide the
purely-visual digit with aria-hidden, so the toggle + count read sensibly.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:54:48 +02:00
Marcel
995471082e test(documents): update obsolete em-dash assertion to undated badge
The "missing documentDate" test asserted the OLD bare em-dash; #668
replaced it with the "Datum unbekannt" badge via <DocumentDate>. Assert
the badge text and rename the misleading test title.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:54:24 +02:00
Marcel
c6137a26a2 feat(documents): show global undated count chip on the filter toggle
Some checks failed
CI / Unit & Component Tests (pull_request) Failing after 2m50s
CI / OCR Service Tests (pull_request) Successful in 22s
CI / Backend Unit Tests (pull_request) Failing after 4m3s
CI / fail2ban Regex (pull_request) Successful in 46s
CI / Semgrep Security Scan (pull_request) Successful in 21s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m3s
Surface the backend's global undatedCount on the "Nur undatierte" toggle as
a count chip — the total undated documents matching the current filter
across all pages, not the page slice. The loader forwards undatedCount
straight through (defaulting to 0); the chip hides at 0 and stays visible
regardless of the toggle state so it advertises the triage backlog size.

generate:api was hand-edited (undatedCount added to DocumentSearchResult) —
CI must re-run npm run generate:api to confirm parity.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:42:57 +02:00
Marcel
a3c3f14aea feat(documents): return global undated count in search response
The undated bucket count was page-local — derived from the year-grouping
of the current page's items, so it could never exceed the page size. The
owner's decision is for it to reflect ALL undated documents matching the
active filter across every page.

Add an undatedCount field to DocumentSearchResult, computed once per search
via a COUNT over the same filter spec with undatedOnly(true) forced —
independent of the "Nur undatierte" toggle so it never collapses to the
page slice or double-counts. A from/to range excludes undated rows by the
collision rule, so the count is legitimately 0 inside a date range.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:42:32 +02:00
Marcel
19cd17d9cd fix(documents): always render undated badge in DocumentRow desktop column
Some checks failed
CI / Unit & Component Tests (pull_request) Failing after 2m54s
CI / OCR Service Tests (pull_request) Successful in 20s
CI / Backend Unit Tests (pull_request) Successful in 3m40s
CI / fail2ban Regex (pull_request) Successful in 41s
CI / Semgrep Security Scan (pull_request) Successful in 19s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m2s
The desktop right-column kept a leftover {#if doc.documentDate}…{:else}—{/if}
fallback that emitted a bare em-dash for undated documents, while the mobile
block already always rendered <DocumentDate>. DocumentDate defensively maps a
null date to the "Datum unbekannt" badge, so render it unconditionally — an
undated document is an absence, not an error, and never shows a bare "—".

Refs #668
2026-05-27 19:17:18 +02:00
Marcel
508575eccb refactor(documents): collapse redundant span nesting in DocumentDate else branch
Some checks failed
CI / Unit & Component Tests (pull_request) Failing after 2m51s
CI / OCR Service Tests (pull_request) Successful in 23s
CI / Backend Unit Tests (pull_request) Successful in 3m43s
CI / fail2ban Regex (pull_request) Successful in 45s
CI / Semgrep Security Scan (pull_request) Successful in 21s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m1s
The dated branch wrapped {label} in a flex span containing a single child
span — redundant nesting. Render the label directly in one span.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:09:07 +02:00
Marcel
85372e3669 fix(documents): enlarge undated badge text to text-xs for legibility
"Datum unbekannt" is a semantically meaningful date surface, not decorative
chrome, so the 10px chip text is too small for the senior reader audience.
Bump to text-xs (≥12px) per the WCAG min-legible-text guidance.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:08:41 +02:00
Marcel
caec92e7de test(document): lock undated-stays-in-sender-group with ordered multi-sender assertions
Replace the single-sender containsExactlyInAnyOrder check with a two-sender
fixture and ordered containsExactly proving an undated doc stays within its
sender group and never floats to the page head. Add a DESC-direction case for
in-memory-path symmetry and an undated=true + sort=SENDER case capturing the
Specification to prove undatedOnly is still applied on the person-sort path.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:06:33 +02:00
Marcel
eacfd15f8e refactor(document): revert resolveSort to private
No test calls resolveSort directly — the sort tests assert through
searchDocuments + ArgumentCaptor<Pageable>, so the package-private widening
added no value. Narrow the API surface back to private.

Refs #668

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-27 19:06:16 +02:00
Marcel
a345bba74b test(activity): assert Chronik rows never fabricate a letter date
Some checks failed
CI / Unit & Component Tests (pull_request) Failing after 2m54s
CI / OCR Service Tests (pull_request) Successful in 22s
CI / Backend Unit Tests (pull_request) Successful in 3m30s
CI / fail2ban Regex (pull_request) Successful in 45s
CI / Semgrep Security Scan (pull_request) Successful in 20s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m3s
Negative guarantee for #668: ChronikRow renders the activity timestamp
(happenedAt), and ActivityFeedItemDTO carries no document-date surface, so
no undated badge or "Datum unbekannt" letter-date label may appear. Pins
this as a regression fixture so a future change can't quietly add a date
chip to the activity feed.

Refs #668
2026-05-27 18:54:35 +02:00
Marcel
098c2c9def feat(documents): add a "Nur undatierte" filter toggle wired to the URL
SearchFilterBar gains an aria-pressed "Nur undatierte" toggle in the
advanced row (min-h-[44px] touch target, labels the state not the colour).
The documents page threads `undated` through the filter snapshot so it is a
shareable URL param picked up by both filter-change nav and pagination, and
flows into the bulk-edit "select all" /ids request. Toggling resets to page
0 via the existing implicit page-drop.

Refs #668
2026-05-27 18:53:44 +02:00
Marcel
5d8bb70255 feat(documents): explain that a date range excludes undated documents
DocumentList gains from/to props; when a date range is active and yields no
results, the empty state shows the localized docs_range_excludes_undated
note instead of the generic copy, so the reader understands undated letters
aren't part of a range. Person-grouped modes keep undated letters under
their sender/receiver (badge-on-row, no synthetic sub-group).

Refs #668
2026-05-27 18:50:18 +02:00
Marcel
bca3f34cec feat(documents): badge undated rows instead of a bare em-dash
DocumentRow rendered a bare em-dash for null-dated letters — a glyph a
screen reader announces as nothing. Both breakpoints now render the single
DocumentDate component unconditionally (no {#if}/—/{:else}), so the cue
cannot drift; its unknown state is a neutral metadata chip ("Datum
unbekannt", text-ink-3, ≥4.5:1 both themes) with a non-color calendar glyph,
never red/amber. Present dates render at honest precision via
formatDocumentDate ("Juni 1916", not a fabricated day).

Refs #668
2026-05-27 18:48:45 +02:00
Marcel
f1fc3dc1ce feat(documents): thread undated filter through the search loader + i18n
Parses ?undated strictly (=== 'true', mirroring the tagOp clamp), forwards
it as undated || undefined so the absent case drops out of the query, and
returns the flag in page data for the control to reflect. Adds the
docs_filter_undated_only toggle label and the explanatory
docs_range_excludes_undated empty-state copy in de/en/es. The badge reuses
the existing date_precision_unknown ("Datum unbekannt") key from #677.

OpenAPI types hand-edited for the new undated query param on /search and
/ids — CI must run `npm run generate:api` to confirm parity with the spec.

Refs #668
2026-05-27 18:45:03 +02:00
Marcel
268c31a49b feat(document): thread an undated filter through search and the /ids path
Adds an optional `undated` query param to GET /api/documents/search and
/api/documents/ids, threaded through searchDocuments and findIdsForFilter
into the shared buildSearchSpec via undatedOnly(boolean). undated=true also
bypasses the pure-text RELEVANCE SQL shortcut, which skips buildSearchSpec
and would otherwise drop the predicate. The read GET stays unguarded
(WebMvc authz test pins 200 for an authenticated user, 401 unauthenticated).
A locking test proves the in-memory SENDER sort keeps undated letters under
their sender.

Refs #668
2026-05-27 18:42:17 +02:00
Marcel
39a462b2bb feat(document): add undatedOnly Specification for the undated-only filter
undatedOnly(false) is a no-op (null predicate); undatedOnly(true) returns
documentDate IS NULL, matching the existing hasStatus null-as-no-op pattern.
Real-Postgres tests pin the load-bearing guarantees H2 cannot prove: ASC
NULLS-LAST ordering, BETWEEN excludes null-dated rows, and that undated=true
combined with a from/to range returns empty (the collision rule).

Refs #668
2026-05-27 18:34:10 +02:00
Marcel
5f2ef823e1 fix(document): order undated documents last on the DATE sort fast path
resolveSort produced Sort.by(direction, "documentDate") with NATIVE null
handling, so Postgres surfaced undated (null meta_date) documents FIRST on
an ASC sort. Apply nullsLast() so undated rows order last for both ASC and
DESC, with a createdAt-asc tiebreaker for a stable total order when every
row is null-dated (the upcoming "Nur undatierte" filter).

Refs #668
2026-05-27 18:31:40 +02:00