Compare commits

...

9 Commits

Author SHA1 Message Date
Marcel
8e9e3bba06 refactor(document): address review concerns from PR #660
All checks were successful
CI / Semgrep Security Scan (pull_request) Successful in 21s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m2s
nightly / deploy-staging (push) Successful in 2m2s
CI / Unit & Component Tests (push) Successful in 3m58s
CI / OCR Service Tests (push) Successful in 20s
CI / Backend Unit Tests (push) Successful in 3m50s
CI / fail2ban Regex (push) Successful in 44s
CI / Unit & Component Tests (pull_request) Successful in 3m29s
CI / Semgrep Security Scan (push) Successful in 21s
CI / OCR Service Tests (pull_request) Successful in 21s
CI / Backend Unit Tests (pull_request) Successful in 3m43s
CI / Compose Bucket Idempotency (push) Successful in 59s
CI / fail2ban Regex (pull_request) Successful in 45s
- Restore JavaDoc on DocumentSearchResult.of() and .paged() factory methods
- Remove redundant null guards on @Builder.Default collections in toListItem()
- Map DocumentListItem fields explicitly in DocumentMultiSelect before cast
- Add DocumentListItem required fields to docFactory in spec

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 19:27:31 +02:00
Marcel
627fc44d99 fix(document): fix test regressions from DocumentListItem migration
All checks were successful
CI / Unit & Component Tests (pull_request) Successful in 3m32s
CI / OCR Service Tests (pull_request) Successful in 20s
CI / Backend Unit Tests (pull_request) Successful in 3m46s
CI / fail2ban Regex (pull_request) Successful in 42s
CI / Semgrep Security Scan (pull_request) Successful in 19s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m0s
- Use documentService.getDocumentById() in detail_stillReturnsTrainingLabels
  so the Document.full entity graph eager-loads trainingLabels
- Flatten makeItem() factory in DocumentList.svelte.test.ts (nested
  document: {} overrides broke item.id / item.documentDate access)
- Remove { document: {} } wrapper from DocumentMultiSelect.svelte.spec.ts
  mock responses — component now reads body.items directly as flat items
- Flatten single nested item in page.svelte.test.ts document list test

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 19:19:28 +02:00
Marcel
6583226d79 refactor(document): migrate frontend from DocumentSearchItem to flat DocumentListItem
All components, specs, and the generated API client now use the new
DocumentListItem shape — flat access (item.title, item.sender) instead of
the removed item.document.* nesting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 19:19:28 +02:00
Marcel
41b205becc test(document): add LazyInit guard + detail regression tests; prune Document.list graph
Remove trainingLabels from Document.list entity graph now that DocumentListItem
does not touch that association. Integration tests guard against future
LazyInitializationException regressions and confirm Document.full still
loads trainingLabels for the detail endpoint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 19:19:28 +02:00
Marcel
f22dcaecb7 refactor(document): replace DocumentSearchItem with flat DocumentListItem DTO
Eliminates excessive data exposure (OWASP API3:2023) — transcription,
filePath, fileHash, thumbnailKey, scriptType and other detail-only fields
are no longer serialised in the list API response.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 19:19:03 +02:00
Marcel
1109ab917b docs(observability): ADR-024 + rotation runbook for grafana_reader
All checks were successful
CI / Backend Unit Tests (push) Successful in 3m35s
CI / fail2ban Regex (push) Successful in 42s
CI / Semgrep Security Scan (push) Successful in 19s
CI / Compose Bucket Idempotency (push) Successful in 1m3s
nightly / deploy-staging (push) Successful in 2m0s
CI / Unit & Component Tests (pull_request) Successful in 3m39s
CI / OCR Service Tests (pull_request) Successful in 22s
CI / Backend Unit Tests (pull_request) Successful in 3m53s
CI / fail2ban Regex (pull_request) Successful in 43s
CI / Semgrep Security Scan (pull_request) Successful in 20s
CI / Compose Bucket Idempotency (pull_request) Successful in 1m1s
CI / Unit & Component Tests (push) Successful in 3m39s
CI / OCR Service Tests (push) Successful in 20s
ADR-024 records the deliberate cross-domain link (obs-grafana joins
archiv-net to query archive-db via the SELECT-only grafana_reader role),
the rejected alternatives (Prometheus exporter, read replica, versioned
migration + flyway repair, hardcoded fallback), and the consequences —
specifically that a Grafana compromise gains TCP reach to archive-db
but is bounded by the role's least-privilege grants.

The DEPLOYMENT.md runbook documents the rotation procedure that
R__grafana_reader_password.sql now enables: bump GRAFANA_DB_PASSWORD,
restart backend (Flyway re-applies because the resolved checksum
changed), restart obs-grafana (datasource picks up the new env var).
Also calls out the fail-closed startup behavior so operators who hit
IllegalStateException know it is deliberate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:21:27 +02:00
Marcel
769984608b test(observability): expand grafana_reader coverage with write-deny + PII negatives
The original 4 tests asserted SELECT existed on the three granted tables
and was absent on app_users. That left two gaps a future migration could
slip through silently:

- INSERT/UPDATE/DELETE on the granted tables — if someone GRANTed write
  access on, say, documents to grafana_reader, the SELECT positives stay
  green and the boundary is breached invisibly.
- Other PII / sensitive tables — the single app_users negative checks
  one table; a wildcard "GRANT SELECT ON ALL TABLES IN SCHEMA public"
  would still leave it green by accident if app_users wasn't the only
  sensitive table.

Switch to a hasPrivilege(table, privilege) helper, add three write-deny
tests (INSERT/UPDATE/DELETE on each granted table), and replace the
single app_users negative with a parameterized sweep over app_users,
user_groups, persons, notifications, document_comments,
document_annotations, geschichten. New sensitive tables get added to
that list as they appear.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:21:01 +02:00
Marcel
c282f38170 feat(observability): own grafana_reader password via repeatable migration
V68 used to set the role's password in a versioned migration, which Flyway
applies exactly once per database. Rotating GRAFANA_DB_PASSWORD therefore
had no effect on the DB role — operators would need a manual ALTER ROLE
or a `flyway repair` that nobody documented. The shape conflated two
lifecycles: schema migration (one-shot, immutable) and credential
provisioning (rotatable).

Split into:
- V68 (versioned, immutable): creates the role and applies SELECT grants
  on audit_log, documents, transcription_blocks.
- R__grafana_reader_password.sql (repeatable): issues ALTER ROLE … PASSWORD
  with the placeholder. Flyway computes the checksum on the resolved
  content, so any change to GRAFANA_DB_PASSWORD changes the checksum and
  re-applies the migration on the next boot. Rotation becomes "bump env
  var + restart backend".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:20:35 +02:00
Marcel
3ea7f0b5b2 feat(observability): fail closed when GRAFANA_DB_PASSWORD is unset
FlywayConfig used to fall back to a hardcoded "changeme-grafana-db-password"
string when the env var was missing. That published a known credential for
the grafana_reader role (SELECT on audit_log, documents, transcription_blocks)
into git history and made silent fail-open the default for any deploy that
forgot the secret. Now resolution goes through Spring's Environment and
throws IllegalStateException at startup when the value is unset or blank —
same shape as UserDataInitializer's refusal to seed default admin creds.

Tests inject via the global GRAFANA_DB_PASSWORD entry in test-resources
application.properties so existing Flyway-loading test classes keep
booting without per-class TestPropertySource boilerplate. FlywayConfigTest
covers both branches against MockEnvironment without a Spring context.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:20:09 +02:00
31 changed files with 759 additions and 318 deletions

View File

@@ -5,6 +5,7 @@ import lombok.extern.slf4j.Slf4j;
import org.flywaydb.core.Flyway;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import javax.sql.DataSource;
import java.util.Map;
@@ -14,9 +15,8 @@ import java.util.Map;
@Slf4j
public class FlywayConfig {
private static final String GRAFANA_DB_PASSWORD_FALLBACK = "changeme-grafana-db-password";
private final DataSource dataSource;
private final Environment environment;
@Bean(name = "flyway")
public Flyway flyway() {
@@ -33,12 +33,20 @@ public class FlywayConfig {
return flyway;
}
private String resolveGrafanaDbPassword() {
String value = System.getenv("GRAFANA_DB_PASSWORD");
// Fail-closed: refuse to boot when GRAFANA_DB_PASSWORD is unset. The
// grafana_reader role's password is (re)set on every boot by
// R__grafana_reader_password.sql, so a missing env var means we'd either
// skip the rotation silently or — with a hardcoded fallback — publish a
// well-known credential for a role with SELECT on audit_log, documents,
// and transcription_blocks. Same shape as UserDataInitializer's refusal
// to seed default admin credentials outside dev/test/e2e.
String resolveGrafanaDbPassword() {
String value = environment.getProperty("GRAFANA_DB_PASSWORD");
if (value == null || value.isBlank()) {
log.warn("GRAFANA_DB_PASSWORD is not set; the grafana_reader role will use a non-secret fallback. "
+ "Set GRAFANA_DB_PASSWORD in production to enable the Grafana PostgreSQL datasource.");
return GRAFANA_DB_PASSWORD_FALLBACK;
throw new IllegalStateException(
"GRAFANA_DB_PASSWORD is required: it is consumed by "
+ "R__grafana_reader_password.sql to (re)set the grafana_reader "
+ "role's password on every boot. Generate with: openssl rand -hex 32");
}
return value;
}

View File

@@ -31,8 +31,7 @@ import java.util.UUID;
@NamedEntityGraph(name = "Document.list", attributeNodes = {
@NamedAttributeNode("sender"),
@NamedAttributeNode("receivers"),
@NamedAttributeNode("tags"),
@NamedAttributeNode("trainingLabels")
@NamedAttributeNode("tags")
})
@Entity
@Table(name = "documents")

View File

@@ -0,0 +1,36 @@
package org.raddatz.familienarchiv.document;
import io.swagger.v3.oas.annotations.media.Schema;
import org.raddatz.familienarchiv.audit.ActivityActorDTO;
import org.raddatz.familienarchiv.person.Person;
import org.raddatz.familienarchiv.tag.Tag;
import java.time.LocalDate;
import java.util.List;
import java.util.UUID;
public record DocumentListItem(
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
UUID id,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
String title,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
String originalFilename,
String thumbnailUrl,
LocalDate documentDate,
Person sender,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
List<Person> receivers,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
List<Tag> tags,
String archiveBox,
String archiveFolder,
String location,
String summary,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
int completionPercentage,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
List<ActivityActorDTO> contributors,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
SearchMatchData matchData
) {}

View File

@@ -1,18 +0,0 @@
package org.raddatz.familienarchiv.document;
import io.swagger.v3.oas.annotations.media.Schema;
import org.raddatz.familienarchiv.audit.ActivityActorDTO;
import org.raddatz.familienarchiv.document.Document;
import java.util.List;
public record DocumentSearchItem(
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
Document document,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
SearchMatchData matchData,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
int completionPercentage,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
List<ActivityActorDTO> contributors
) {}

View File

@@ -7,7 +7,7 @@ import java.util.List;
public record DocumentSearchResult(
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
List<DocumentSearchItem> items,
List<DocumentListItem> items,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
long totalElements,
@Schema(requiredMode = Schema.RequiredMode.REQUIRED)
@@ -21,16 +21,16 @@ public record DocumentSearchResult(
* Single-page convenience factory used by empty-result shortcuts and by tests that
* don't care about paging. Treats the whole list as page 0 of itself.
*/
public static DocumentSearchResult of(List<DocumentSearchItem> items) {
public static DocumentSearchResult of(List<DocumentListItem> items) {
int size = items.size();
return new DocumentSearchResult(items, size, 0, size, size == 0 ? 0 : 1);
}
/**
* Paged factory used by the service when it has a real Pageable + full match count
* (e.g. from Spring's Page<T> or from an in-memory sort-then-slice).
* (e.g. from Spring's Page&lt;T&gt; or from an in-memory sort-then-slice).
*/
public static DocumentSearchResult paged(List<DocumentSearchItem> slice, Pageable pageable, long totalElements) {
public static DocumentSearchResult paged(List<DocumentListItem> slice, Pageable pageable, long totalElements) {
int pageSize = pageable.getPageSize();
int totalPages = pageSize == 0 ? 0 : (int) ((totalElements + pageSize - 1) / pageSize);
return new DocumentSearchResult(slice, totalElements, pageable.getPageNumber(), pageSize, totalPages);

View File

@@ -10,7 +10,6 @@ import org.raddatz.familienarchiv.audit.AuditService;
import org.raddatz.familienarchiv.document.DocumentBatchMetadataDTO;
import org.raddatz.familienarchiv.document.DocumentBatchSummary;
import org.raddatz.familienarchiv.document.DocumentBulkEditDTO;
import org.raddatz.familienarchiv.document.DocumentSearchItem;
import org.raddatz.familienarchiv.document.DocumentSearchResult;
import org.raddatz.familienarchiv.document.DocumentSort;
import org.raddatz.familienarchiv.document.DocumentUpdateDTO;
@@ -736,7 +735,7 @@ public class DocumentService {
return DocumentSearchResult.paged(enrichItems(slice, text), pageable, totalElements);
}
private List<DocumentSearchItem> enrichItems(List<Document> documents, String text) {
private List<DocumentListItem> enrichItems(List<Document> documents, String text) {
List<Document> colorResolved = resolveDocumentTagColors(documents);
Map<UUID, SearchMatchData> matchData = enrichWithMatchData(colorResolved, text);
@@ -744,7 +743,7 @@ public class DocumentService {
Map<UUID, Integer> completionByDoc = fetchCompletionPercentages(docIds);
Map<UUID, List<ActivityActorDTO>> contributorsByDoc = auditLogQueryService.findRecentContributorsPerDocument(docIds);
return colorResolved.stream().map(doc -> new DocumentSearchItem(
return colorResolved.stream().map(doc -> toListItem(
doc,
matchData.getOrDefault(doc.getId(), SearchMatchData.empty()),
completionByDoc.getOrDefault(doc.getId(), 0),
@@ -752,6 +751,26 @@ public class DocumentService {
)).toList();
}
private DocumentListItem toListItem(Document doc, SearchMatchData match, int completionPct, List<ActivityActorDTO> contributors) {
return new DocumentListItem(
doc.getId(),
doc.getTitle(),
doc.getOriginalFilename(),
doc.getThumbnailUrl(),
doc.getDocumentDate(),
doc.getSender(),
List.copyOf(doc.getReceivers()),
List.copyOf(doc.getTags()),
doc.getArchiveBox(),
doc.getArchiveFolder(),
doc.getLocation(),
doc.getSummary(),
completionPct,
contributors,
match
);
}
private Map<UUID, Integer> fetchCompletionPercentages(List<UUID> docIds) {
return transcriptionBlockQueryService.getCompletionStats(docIds);
}

View File

@@ -0,0 +1,14 @@
-- Repeatable migration: sets the grafana_reader role's password from the
-- ${grafanaDbPassword} placeholder (resolved by FlywayConfig from the
-- GRAFANA_DB_PASSWORD environment variable). Flyway computes the checksum on
-- the resolved migration content, so any change to GRAFANA_DB_PASSWORD changes
-- the checksum and re-applies this migration on the next boot. That makes
-- password rotation a "change env var + restart" operation — no manual psql.
--
-- V68 created the role itself (without a usable password). This file owns the
-- password lifecycle; nothing else writes it.
DO $$
BEGIN
EXECUTE format('ALTER ROLE grafana_reader WITH PASSWORD %L', '${grafanaDbPassword}');
END
$$;

View File

@@ -1,13 +1,13 @@
-- Read-only role used by the Grafana PostgreSQL datasource for the PO Overview
-- dashboard (issue #651). Password is injected at migration time via the Flyway
-- placeholder ${grafanaDbPassword}, supplied by FlywayConfig from the
-- GRAFANA_DB_PASSWORD environment variable.
-- dashboard (issue #651). The role is created here without a usable password
-- (LOGIN-capable but no password set); R__grafana_reader_password.sql sets the
-- password from GRAFANA_DB_PASSWORD on every boot, so rotation is just "bump
-- the env var and restart the backend" — see docs/adr/024-* and the rotation
-- runbook in docs/DEPLOYMENT.md.
DO $$
BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_catalog.pg_roles WHERE rolname = 'grafana_reader') THEN
EXECUTE format('CREATE ROLE grafana_reader WITH LOGIN PASSWORD %L', '${grafanaDbPassword}');
ELSE
EXECUTE format('ALTER ROLE grafana_reader WITH LOGIN PASSWORD %L', '${grafanaDbPassword}');
CREATE ROLE grafana_reader WITH LOGIN;
END IF;
END
$$;

View File

@@ -0,0 +1,37 @@
package org.raddatz.familienarchiv.config;
import org.junit.jupiter.api.Test;
import org.springframework.mock.env.MockEnvironment;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
class FlywayConfigTest {
@Test
void resolveGrafanaDbPassword_throws_when_env_unset() {
FlywayConfig config = new FlywayConfig(null, new MockEnvironment());
assertThatThrownBy(config::resolveGrafanaDbPassword)
.isInstanceOf(IllegalStateException.class)
.hasMessageContaining("GRAFANA_DB_PASSWORD is required");
}
@Test
void resolveGrafanaDbPassword_throws_when_env_blank() {
MockEnvironment env = new MockEnvironment().withProperty("GRAFANA_DB_PASSWORD", " ");
FlywayConfig config = new FlywayConfig(null, env);
assertThatThrownBy(config::resolveGrafanaDbPassword)
.isInstanceOf(IllegalStateException.class)
.hasMessageContaining("GRAFANA_DB_PASSWORD is required");
}
@Test
void resolveGrafanaDbPassword_returns_value_when_env_set() {
MockEnvironment env = new MockEnvironment().withProperty("GRAFANA_DB_PASSWORD", "abc");
FlywayConfig config = new FlywayConfig(null, env);
assertThat(config.resolveGrafanaDbPassword()).isEqualTo("abc");
}
}

View File

@@ -1,6 +1,8 @@
package org.raddatz.familienarchiv.config;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;
import org.raddatz.familienarchiv.PostgresContainerConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.data.jpa.test.autoconfigure.DataJpaTest;
@@ -10,6 +12,9 @@ import org.springframework.jdbc.core.JdbcTemplate;
import static org.assertj.core.api.Assertions.assertThat;
// GRAFANA_DB_PASSWORD is supplied via the global test default in
// src/test/resources/application.properties — FlywayConfig fails closed
// when it is unset, so all tests that load the migration path need it.
@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
@Import({PostgresContainerConfig.class, FlywayConfig.class})
@@ -17,31 +22,68 @@ class GrafanaReaderRoleIntegrationTest {
@Autowired JdbcTemplate jdbc;
// --- positive grants (SELECT on the three explicitly granted tables) ---
@Test
void grafana_reader_has_select_on_audit_log() {
assertThat(hasSelect("audit_log")).isTrue();
assertThat(hasPrivilege("audit_log", "SELECT")).isTrue();
}
@Test
void grafana_reader_has_select_on_documents() {
assertThat(hasSelect("documents")).isTrue();
assertThat(hasPrivilege("documents", "SELECT")).isTrue();
}
@Test
void grafana_reader_has_select_on_transcription_blocks() {
assertThat(hasSelect("transcription_blocks")).isTrue();
assertThat(hasPrivilege("transcription_blocks", "SELECT")).isTrue();
}
// --- write-deny on the granted tables: SELECT-only means SELECT-only.
// A future migration that GRANTs INSERT/UPDATE/DELETE on any of these
// would fail these tests, even though the original positive grants still
// pass. Locks the boundary in both directions.
@Test
void grafana_reader_has_no_INSERT_on_documents() {
assertThat(hasPrivilege("documents", "INSERT")).isFalse();
}
@Test
void grafana_reader_has_no_select_on_app_users() {
assertThat(hasSelect("app_users")).isFalse();
void grafana_reader_has_no_UPDATE_on_audit_log() {
assertThat(hasPrivilege("audit_log", "UPDATE")).isFalse();
}
private boolean hasSelect(String table) {
@Test
void grafana_reader_has_no_DELETE_on_transcription_blocks() {
assertThat(hasPrivilege("transcription_blocks", "DELETE")).isFalse();
}
// --- negative grants: PII / sensitive tables MUST NOT be readable.
// The parameterized form catches the "someone widened the grant to
// ALL TABLES IN SCHEMA public" footgun — three specific positive grants
// would still pass while this sweep turns red.
@ParameterizedTest
@ValueSource(strings = {
"app_users",
"user_groups",
"persons",
"notifications",
"document_comments",
"document_annotations",
"geschichten"
})
void grafana_reader_has_no_SELECT_on_protected_table(String table) {
assertThat(hasPrivilege(table, "SELECT")).isFalse();
}
private boolean hasPrivilege(String table, String privilege) {
Boolean result = jdbc.queryForObject(
"SELECT has_table_privilege('grafana_reader', ?, 'SELECT')",
"SELECT has_table_privilege('grafana_reader', ?, ?)",
Boolean.class,
table);
table,
privilege);
return Boolean.TRUE.equals(result);
}
}

View File

@@ -27,7 +27,6 @@ import org.springframework.security.test.context.support.WithMockUser;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import org.springframework.test.web.servlet.MockMvc;
import org.raddatz.familienarchiv.document.DocumentSearchItem;
import org.raddatz.familienarchiv.document.SearchMatchData;
import java.time.LocalDateTime;
@@ -130,16 +129,13 @@ class DocumentControllerTest {
@WithMockUser
void search_responseBodyItemsContainMatchData() throws Exception {
UUID docId = UUID.randomUUID();
Document doc = Document.builder()
.id(docId)
.title("Brief an Anna")
.originalFilename("brief.pdf")
.status(DocumentStatus.UPLOADED)
.build();
var matchData = new SearchMatchData(
"Er schrieb einen langen Brief", List.of(), false, List.of(), List.of(), List.of(), null, List.of());
when(documentService.searchDocuments(any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()))
.thenReturn(DocumentSearchResult.of(List.of(new DocumentSearchItem(doc, matchData, 0, List.of()))));
.thenReturn(DocumentSearchResult.of(List.of(new DocumentListItem(
docId, "Brief an Anna", "brief.pdf", null, null, null,
List.of(), List.of(), null, null, null, null,
0, List.of(), matchData))));
mockMvc.perform(get("/api/documents/search").param("q", "Brief"))
.andExpect(status().isOk())
@@ -148,6 +144,27 @@ class DocumentControllerTest {
.value("Er schrieb einen langen Brief"));
}
@Test
@WithMockUser
void search_returns_flat_item_with_id_and_without_sensitive_fields() throws Exception {
UUID docId = UUID.randomUUID();
var matchData = new SearchMatchData(null, List.of(), false, List.of(), List.of(), List.of(), null, List.of());
when(documentService.searchDocuments(any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any(), any()))
.thenReturn(DocumentSearchResult.of(List.of(new DocumentListItem(
docId, "Brief an Anna", "brief.pdf", null, null, null,
List.of(), List.of(), null, null, null, null,
0, List.of(), matchData))));
mockMvc.perform(get("/api/documents/search"))
.andExpect(status().isOk())
// flat id field present at top of item (not nested under $.items[0].document.id)
.andExpect(jsonPath("$.items[0].id").value(docId.toString()))
// sensitive storage fields must never appear in list response
.andExpect(jsonPath("$.items[0].transcription").doesNotExist())
.andExpect(jsonPath("$.items[0].filePath").doesNotExist())
.andExpect(jsonPath("$.items[0].fileHash").doesNotExist());
}
// ─── /api/documents/search pagination ─────────────────────────────────────
@Test

View File

@@ -127,7 +127,7 @@ class DocumentLazyLoadingTest {
PageRequest.of(0, 20));
assertThat(result.totalElements()).isGreaterThan(0);
assertThatCode(() ->
result.items().forEach(i -> i.document().getSender().getLastName()))
result.items().forEach(i -> { if (i.sender() != null) i.sender().getLastName(); }))
.doesNotThrowAnyException();
}

View File

@@ -0,0 +1,98 @@
package org.raddatz.familienarchiv.document;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.raddatz.familienarchiv.PostgresContainerConfig;
import org.raddatz.familienarchiv.audit.AuditLogQueryService;
import org.raddatz.familienarchiv.ocr.TrainingLabel;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.Import;
import org.springframework.data.domain.PageRequest;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.bean.override.mockito.MockitoBean;
import software.amazon.awssdk.services.s3.S3Client;
import java.util.HashSet;
import java.util.Set;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatCode;
/**
* AC #2: Document with trainingLabels does not cause LazyInitializationException in search.
* AC #3: Detail API still returns trainingLabels after the Document.list graph change.
*/
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE)
@ActiveProfiles("test")
@Import(PostgresContainerConfig.class)
class DocumentListItemIntegrationTest {
@MockitoBean
S3Client s3Client;
@MockitoBean
AuditLogQueryService auditLogQueryService;
@Autowired
DocumentRepository documentRepository;
@Autowired
DocumentService documentService;
@AfterEach
void cleanup() {
documentRepository.deleteAll();
}
@Test
void search_doesNotThrow_whenDocumentHasTrainingLabels() {
documentRepository.save(Document.builder()
.title("Kurrent Brief")
.originalFilename("kurrent.pdf")
.status(DocumentStatus.UPLOADED)
.trainingLabels(new HashSet<>(Set.of(TrainingLabel.KURRENT_RECOGNITION)))
.build());
assertThatCode(() -> documentService.searchDocuments(
null, null, null, null, null, null, null, null,
DocumentSort.DATE, "DESC", null,
PageRequest.of(0, 50)))
.doesNotThrowAnyException();
}
@Test
void search_returns_list_item_without_sensitive_fields_when_document_has_training_labels() {
documentRepository.save(Document.builder()
.title("Kurrent Brief")
.originalFilename("kurrent2.pdf")
.status(DocumentStatus.UPLOADED)
.trainingLabels(new HashSet<>(Set.of(TrainingLabel.KURRENT_RECOGNITION)))
.build());
DocumentSearchResult result = documentService.searchDocuments(
null, null, null, null, null, null, null, null,
DocumentSort.DATE, "DESC", null,
PageRequest.of(0, 50));
assertThat(result.totalElements()).isGreaterThan(0);
DocumentListItem item = result.items().get(0);
assertThat(item.id()).isNotNull();
assertThat(item.title()).isEqualTo("Kurrent Brief");
}
@Test
void detail_stillReturnsTrainingLabels() {
Document saved = documentRepository.save(Document.builder()
.title("Detail Test")
.originalFilename("detail_test.pdf")
.status(DocumentStatus.UPLOADED)
.trainingLabels(new HashSet<>(Set.of(TrainingLabel.KURRENT_RECOGNITION)))
.build());
// Document.full entity graph (used by getDocumentById) must still load trainingLabels
Document loaded = documentService.getDocumentById(saved.getId());
assertThat(loaded.getTrainingLabels()).containsExactly(TrainingLabel.KURRENT_RECOGNITION);
}
}

View File

@@ -125,10 +125,10 @@ class DocumentSearchPagedIntegrationTest {
// No document id should appear on both pages — slicing must be exclusive.
var idsOnPage0 = page0.items().stream()
.map(item -> item.document().getId())
.map(item -> item.id())
.toList();
var idsOnPage1 = page1.items().stream()
.map(item -> item.document().getId())
.map(item -> item.id())
.toList();
for (UUID id : idsOnPage0) {
assertThat(idsOnPage1).doesNotContain(id);

View File

@@ -3,8 +3,6 @@ package org.raddatz.familienarchiv.document;
import io.swagger.v3.oas.annotations.media.Schema;
import org.junit.jupiter.api.Test;
import org.raddatz.familienarchiv.audit.ActivityActorDTO;
import org.raddatz.familienarchiv.document.Document;
import org.raddatz.familienarchiv.document.DocumentStatus;
import org.springframework.data.domain.PageRequest;
import java.util.List;
@@ -14,14 +12,11 @@ import static org.assertj.core.api.Assertions.assertThat;
class DocumentSearchResultTest {
private DocumentSearchItem item(UUID docId) {
Document doc = Document.builder()
.id(docId)
.title("Test")
.originalFilename("test.pdf")
.status(DocumentStatus.UPLOADED)
.build();
return new DocumentSearchItem(doc, SearchMatchData.empty(), 0, List.of());
private DocumentListItem item(UUID docId) {
return new DocumentListItem(
docId, "Test", "test.pdf", null, null, null,
List.of(), List.of(), null, null, null, null,
0, List.of(), SearchMatchData.empty());
}
@Test
@@ -45,7 +40,7 @@ class DocumentSearchResultTest {
@Test
void paged_factory_populates_paging_fields_from_pageable_and_total() {
List<DocumentSearchItem> slice = List.of(item(UUID.randomUUID()), item(UUID.randomUUID()));
List<DocumentListItem> slice = List.of(item(UUID.randomUUID()), item(UUID.randomUUID()));
DocumentSearchResult result = DocumentSearchResult.paged(slice, PageRequest.of(1, 50), 120L);
@@ -68,9 +63,10 @@ class DocumentSearchResultTest {
void of_exposes_items_with_completion_and_contributors() {
UUID id = UUID.randomUUID();
ActivityActorDTO actor = new ActivityActorDTO("AB", "#f00", "Anna Braun");
Document doc = Document.builder().id(id).title("T").originalFilename("t.pdf")
.status(DocumentStatus.UPLOADED).build();
DocumentSearchItem item = new DocumentSearchItem(doc, SearchMatchData.empty(), 75, List.of(actor));
DocumentListItem item = new DocumentListItem(
id, "T", "t.pdf", null, null, null,
List.of(), List.of(), null, null, null, null,
75, List.of(actor), SearchMatchData.empty());
DocumentSearchResult result = DocumentSearchResult.of(List.of(item));

View File

@@ -70,7 +70,7 @@ class DocumentServiceSortTest {
"Brief", null, null, null, null, null, null, null, DocumentSort.DATE, "DESC", null, PAGE);
assertThat(result.items()).hasSize(2);
assertThat(result.items().get(0).document().getId()).isEqualTo(id2); // newer first
assertThat(result.items().get(0).id()).isEqualTo(id2); // newer first
}
// ─── RELEVANCE sort — pure text (no filters) ──────────────────────────────
@@ -104,7 +104,7 @@ class DocumentServiceSortTest {
DocumentSearchResult result = documentService.searchDocuments(
"Brief", null, null, null, null, null, null, null, DocumentSort.RELEVANCE, null, null, PAGE);
assertThat(result.items().get(0).document().getId()).isEqualTo(id1);
assertThat(result.items().get(0).id()).isEqualTo(id1);
}
@Test
@@ -121,7 +121,7 @@ class DocumentServiceSortTest {
DocumentSearchResult result = documentService.searchDocuments(
"Brief", null, null, null, null, null, null, null, null, null, null, PAGE);
assertThat(result.items().get(0).document().getId()).isEqualTo(id1);
assertThat(result.items().get(0).id()).isEqualTo(id1);
}
// ─── RELEVANCE sort — overflow guard ─────────────────────────────────────
@@ -156,7 +156,7 @@ class DocumentServiceSortTest {
DocumentSort.RELEVANCE, null, null, PAGE);
assertThat(result.items()).hasSize(1);
assertThat(result.items().get(0).document().getId()).isEqualTo(uuidId);
assertThat(result.items().get(0).id()).isEqualTo(uuidId);
}
// ─── RELEVANCE sort — text + active filter ────────────────────────────────

View File

@@ -11,7 +11,7 @@ import org.raddatz.familienarchiv.audit.AuditLogQueryService;
import org.raddatz.familienarchiv.audit.AuditService;
import org.raddatz.familienarchiv.document.annotation.AnnotationService;
import org.raddatz.familienarchiv.document.transcription.TranscriptionBlockQueryService;
import org.raddatz.familienarchiv.document.DocumentSearchItem;
import org.raddatz.familienarchiv.document.DocumentListItem;
import org.raddatz.familienarchiv.document.DocumentSearchResult;
import org.raddatz.familienarchiv.document.DocumentSort;
import org.raddatz.familienarchiv.document.DocumentUpdateDTO;
@@ -1444,7 +1444,7 @@ class DocumentServiceTest {
assertThat(result.totalPages()).isEqualTo(3);
assertThat(result.items()).hasSize(50);
// Page 1 (offset 50) under ascending sender sort should start at L050
assertThat(result.items().get(0).document().getSender().getLastName()).isEqualTo("L050");
assertThat(result.items().get(0).sender().getLastName()).isEqualTo("L050");
}
@Test
@@ -1565,7 +1565,7 @@ class DocumentServiceTest {
null, null, null, null, null, null, null, null, DocumentSort.SENDER, "asc", null, UNPAGED);
assertThat(result.items()).hasSize(2);
assertThat(result.items()).extracting(item -> item.document().getTitle()).containsExactly("Has Sender", "No Sender");
assertThat(result.items()).extracting(DocumentListItem::title).containsExactly("Has Sender", "No Sender");
}
// ─── searchDocuments — RECEIVER sort, empty receivers ───────────────────────
@@ -1584,7 +1584,7 @@ class DocumentServiceTest {
DocumentSearchResult result = documentService.searchDocuments(
null, null, null, null, null, null, null, null, DocumentSort.RECEIVER, "asc", null, UNPAGED);
assertThat(result.items()).extracting(item -> item.document().getTitle())
assertThat(result.items()).extracting(DocumentListItem::title)
.containsExactly("Has Receiver", "No Receivers");
}
@@ -1607,7 +1607,7 @@ class DocumentServiceTest {
null, null, null, null, null, null, null, null, DocumentSort.SENDER, "asc", null, UNPAGED);
// null lastName should sort to end (treated as empty), not before "smith" (as "null")
assertThat(result.items()).extracting(item -> item.document().getTitle())
assertThat(result.items()).extracting(DocumentListItem::title)
.containsExactly("smith doc", "Null lastname doc");
}

View File

@@ -1,2 +1,8 @@
logging.level.root=WARN
logging.level.org.raddatz=INFO
# Default test value so FlywayConfig's fail-closed check passes without each
# test having to set GRAFANA_DB_PASSWORD explicitly. The actual value is
# irrelevant in tests — Flyway only uses it to set the grafana_reader role's
# password, which no test connects with.
GRAFANA_DB_PASSWORD=test-grafana-reader-password

View File

@@ -430,6 +430,31 @@ docker exec obs-loki wget -qO- \
Prometheus port `9090` and Grafana port `3003` (default; configurable via `PORT_GRAFANA`) are bound to `127.0.0.1` on the host. No other observability ports are host-bound.
##### Rotate the `grafana_reader` DB password
The PO Overview dashboard reads `audit_log`, `documents`, and `transcription_blocks` through the SELECT-only `grafana_reader` PostgreSQL role (issue #651, ADR-024). The role's password is owned by `R__grafana_reader_password.sql` — a Flyway *repeatable* migration that re-runs whenever the resolved `${grafanaDbPassword}` placeholder changes. That makes rotation a two-restart operation, no manual `psql` required.
```bash
# 1. Generate a new value
openssl rand -hex 32
# 2. Update both sides:
# - Gitea secret GRAFANA_DB_PASSWORD (nightly + release workflows pick it up)
# - Local .env on the server / dev machine
# 3. Restart the backend. Flyway sees that R__'s resolved checksum changed and
# re-applies it, issuing ALTER ROLE grafana_reader WITH PASSWORD '<new>'.
docker compose restart backend
# 4. Restart obs-grafana so the provisioned datasource picks up the new env value.
docker compose -f docker-compose.observability.yml restart obs-grafana
# 5. Verify the dashboard loads — PO Overview's Postgres panels should populate
# instead of "Data source error".
```
If `GRAFANA_DB_PASSWORD` is unset, the backend **refuses to start** (`IllegalStateException`). That is deliberate — see `FlywayConfig.resolveGrafanaDbPassword()` and the rationale in ADR-024.
#### GlitchTip
| Item | Value |

View File

@@ -0,0 +1,123 @@
# ADR-024: Grafana reads archive-db via a bridged network and a SELECT-only role
## Status
Accepted
## Context
Issue #651 (the PO Overview Grafana dashboard) needs aggregates over three
tables in the main application database — `audit_log`, `documents`, and
`transcription_blocks` — to answer the operator's four weekly questions: is
everything working, are people using it, is the archive making progress, is
OCR working well.
Until now, `obs-grafana` and the rest of the observability stack lived on
their own Docker network (`obs-net`) and never touched `archiv-net`, where
`archive-db` runs. The two were intentionally isolated: a compromise of any
observability container could not pivot to the application database.
The PO Overview's archive-progress and user-activity panels need rolling
7-day SQL aggregates that cannot be served by Prometheus or Loki. That
forces a connection from `obs-grafana` to `archive-db` for the first time.
Two implementation requirements shaped the design:
1. **Least privilege on the database side.** The Spring Boot application
role (`archiv`) has full read/write on every table. Letting Grafana
connect with that role would mean a Grafana compromise becomes an
application compromise. The dashboard only needs SELECT on three
tables; the role must reflect that and nothing more.
2. **Operational simplicity of secret rotation.** The role's password is
shared between the migration that sets it and the Grafana datasource
that uses it. A first version of this work put the password in a
versioned Flyway migration (V68), which Flyway only applies once —
leaving rotation as an out-of-band `psql ALTER ROLE` step that no
runbook documented. The shape must support rotation without manual
SQL.
## Decision
- Provision a dedicated PostgreSQL role `grafana_reader` with `LOGIN` plus
`GRANT SELECT` on `audit_log`, `documents`, `transcription_blocks` only.
No INSERT/UPDATE/DELETE on any table, no access to any other table —
enforced by the database, locked in by both positive and parameterized
negative tests in `GrafanaReaderRoleIntegrationTest`.
- Split the role's lifecycle across two migrations:
- `V68__add_grafana_reader_role.sql` — versioned, immutable, idempotent.
Creates the role and applies the grants. Runs exactly once per
database, like every other versioned migration.
- `R__grafana_reader_password.sql` — Flyway *repeatable* migration that
issues `ALTER ROLE grafana_reader WITH PASSWORD '${grafanaDbPassword}'`.
Flyway computes the checksum on the resolved content, so any change
to `GRAFANA_DB_PASSWORD` flips the checksum and re-applies the
migration on the next boot. Rotation becomes "bump env var, restart
backend, restart obs-grafana" — see the runbook in
`docs/DEPLOYMENT.md §4 → Rotate the grafana_reader DB password`.
- Resolve the password through Spring's `Environment` rather than a raw
`System.getenv()` call, so tests inject via `application.properties`
and the resolver is unit-testable with `MockEnvironment`. Fail closed
with `IllegalStateException` when the variable is unset — no fallback
string. Same shape as `UserDataInitializer`'s refusal to seed default
admin credentials outside dev/test/e2e.
- Join `obs-grafana` to `archiv-net` in addition to `obs-net`. Only the
Grafana container crosses the boundary; Loki, Tempo, Prometheus,
GlitchTip, and the worker containers remain `obs-net`-only.
## Consequences
**Positive**
- Database-level least privilege: a Grafana compromise gains SELECT on
three tables. Cannot write, cannot read PII tables like `app_users`,
`persons`, `notifications`, `document_comments`, `geschichten`. The
parameterized PII negative sweep in `GrafanaReaderRoleIntegrationTest`
is the regression gate; new sensitive tables get added to that list.
- Rotation is documented, idempotent, and survives operator turnover.
No "the password set on day 1 is the password forever" failure mode.
- Tests pin down both sides of the boundary: positive grants must hold,
write-deny must hold, and the PII negative list must stay empty.
**Negative / trade-offs**
- `obs-net` is no longer fully isolated from `archiv-net`. A Grafana RCE
(e.g. via a future Grafana CVE) gains a TCP path to `archive-db`
contained, but not impossible. The least-privilege role is the
mitigation; we accept that mitigation as sufficient for a single
bridged container.
- The backend must hold `GRAFANA_DB_PASSWORD` in its environment forever,
so Flyway can resolve the placeholder on every boot. A backend RCE
therefore also leaks the Grafana datasource password. Acceptable
because that password's blast radius is itself bounded by the
least-privilege grants on `grafana_reader`.
## Alternatives considered
- **Prometheus PostgreSQL exporter, no direct connection.** Loses ad-hoc
SQL aggregates — the dashboard would need every metric pre-defined as
an exporter query, with a redeploy to add a new one. The PO Overview
is the type of dashboard that grows panels over time; pre-defining
every aggregate is the wrong shape.
- **Read replica or logical-replication slot dedicated to Grafana.**
Real operational cost (extra Postgres instance, replication monitoring,
storage doubled) disproportionate to a weekly PO glance.
- **Versioned migration with `flyway repair` for rotation.** Rejected:
conflates schema lifecycle with credential lifecycle, requires manual
intervention to rotate, and the repair command's semantics are
surprising to operators unfamiliar with Flyway internals.
- **Hardcoded fallback password when env var is unset.** Rejected as a
security blocker: publishes a known credential for a role with read
access to user activity and full letter text. The fail-closed
behavior is the explicit defense.
## References
- Issue #651 — PO Overview Grafana dashboard
- `backend/src/main/resources/db/migration/V68__add_grafana_reader_role.sql`
- `backend/src/main/resources/db/migration/R__grafana_reader_password.sql`
- `backend/src/main/java/org/raddatz/familienarchiv/config/FlywayConfig.java`
- `backend/src/test/java/org/raddatz/familienarchiv/config/GrafanaReaderRoleIntegrationTest.java`
- `infra/observability/grafana/provisioning/datasources/datasources.yml`
- `docker-compose.observability.yml``archiv-net` bridge on `obs-grafana`
- `docs/DEPLOYMENT.md §4` — rotation runbook

View File

@@ -5,7 +5,7 @@ import { clickOutside } from '$lib/shared/actions/clickOutside';
import { formatDate } from '$lib/shared/utils/date';
type Document = components['schemas']['Document'];
type DocumentSearchItem = components['schemas']['DocumentSearchItem'];
type DocumentListItem = components['schemas']['DocumentListItem'];
interface Props {
selectedDocuments?: Document[];
@@ -45,8 +45,12 @@ function handleInput() {
try {
const res = await fetch(`/api/documents/search?q=${encodeURIComponent(searchTerm)}&size=10`);
if (res.ok) {
const body: { items: DocumentSearchItem[] } = await res.json();
const docs = body.items.map((it) => it.document);
const body: { items: DocumentListItem[] } = await res.json();
const docs = body.items.map((it) => ({
id: it.id,
title: it.title,
documentDate: it.documentDate
})) as unknown as Document[];
results = docs.filter((d) => !selectedDocuments.some((s) => s.id === d.id));
}
} catch {

View File

@@ -10,7 +10,19 @@ const docFactory = (id: string, title: string, date = '1880-01-01') => ({
title,
documentDate: date,
originalFilename: `${title}.pdf`,
status: 'UPLOADED',
receivers: [],
tags: [],
completionPercentage: 0,
contributors: [],
matchData: {
titleOffsets: [],
senderMatched: false,
matchedReceiverIds: [],
matchedTagIds: [],
snippetOffsets: [],
summaryOffsets: []
},
status: 'UPLOADED' as const,
metadataComplete: false,
scriptType: 'UNKNOWN' as const,
createdAt: '2024-01-01T00:00:00',
@@ -22,7 +34,7 @@ function mockSearchResponse(items: ReturnType<typeof docFactory>[]) {
'fetch',
vi.fn().mockResolvedValue({
ok: true,
json: vi.fn().mockResolvedValue({ items: items.map((document) => ({ document })) })
json: vi.fn().mockResolvedValue({ items })
})
);
}
@@ -91,10 +103,7 @@ describe('DocumentMultiSelect — search and select', () => {
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: vi.fn().mockResolvedValue({
items: [
{ document: docFactory('d1', 'Already attached') },
{ document: docFactory('d2', 'Not attached') }
]
items: [docFactory('d1', 'Already attached'), docFactory('d2', 'Not attached')]
})
});
vi.stubGlobal('fetch', fetchMock);

View File

@@ -9,11 +9,11 @@ import ProgressRing from '$lib/shared/primitives/ProgressRing.svelte';
import ContributorStack from '$lib/shared/primitives/ContributorStack.svelte';
import DocumentThumbnail from './DocumentThumbnail.svelte';
type DocumentSearchItem = components['schemas']['DocumentSearchItem'];
type DocumentListItem = components['schemas']['DocumentListItem'];
let { item, canWrite = false }: { item: DocumentSearchItem; canWrite?: boolean } = $props();
let { item, canWrite = false }: { item: DocumentListItem; canWrite?: boolean } = $props();
const doc = $derived(item.document);
const doc = $derived(item);
const titleText = $derived(doc.title || doc.originalFilename);
const titleOffsets = $derived(item.matchData?.titleOffsets ?? []);
const titleSegments = $derived(applyOffsets(titleText, titleOffsets));

View File

@@ -14,24 +14,17 @@ afterEach(() => {
bulkSelectionStore.clear();
});
type DocumentSearchItem = components['schemas']['DocumentSearchItem'];
type DocumentListItem = components['schemas']['DocumentListItem'];
function makeItem(overrides: Partial<DocumentSearchItem> = {}): DocumentSearchItem {
function makeItem(overrides: Partial<DocumentListItem> = {}): DocumentListItem {
return {
document: {
id: '1',
title: 'Testbrief',
originalFilename: 'testbrief.pdf',
status: 'UPLOADED',
documentDate: '2024-03-15',
sender: null,
sender: undefined,
receivers: [],
tags: [],
createdAt: '2024-01-01T00:00:00Z',
updatedAt: '2024-01-01T00:00:00Z',
metadataComplete: false,
scriptType: 'UNKNOWN'
},
matchData: {
titleOffsets: [],
senderMatched: false,
@@ -55,14 +48,14 @@ describe('DocumentRow title', () => {
});
it('falls back to originalFilename when title is null', async () => {
const item = makeItem({ document: { ...makeItem().document, title: null } });
const item = makeItem({ title: null as unknown as string });
render(DocumentRow, { item });
await expect.element(page.getByRole('heading', { name: 'testbrief.pdf' })).toBeInTheDocument();
});
it('renders a mark element for highlighted title offsets', async () => {
const item = makeItem({
document: { ...makeItem().document, title: 'Brief an Anna' },
title: 'Brief an Anna',
matchData: {
titleOffsets: [{ start: 0, length: 5 }],
senderMatched: false,
@@ -109,9 +102,12 @@ describe('DocumentRow snippet', () => {
describe('DocumentRow sender', () => {
it('shows sender display name', async () => {
const item = makeItem({
document: {
...makeItem().document,
sender: { id: 's1', displayName: 'Großmutter Maria' }
sender: {
id: 's1',
lastName: 'Maria',
displayName: 'Großmutter Maria',
personType: 'PERSON',
familyMember: false
}
});
render(DocumentRow, { item });
@@ -126,9 +122,12 @@ describe('DocumentRow sender', () => {
it('highlights the sender when senderMatched is true', async () => {
const item = makeItem({
document: {
...makeItem().document,
sender: { id: 's1', displayName: 'Großmutter Maria' }
sender: {
id: 's1',
lastName: 'Maria',
displayName: 'Großmutter Maria',
personType: 'PERSON',
familyMember: false
},
matchData: {
...makeItem().matchData,
@@ -142,10 +141,15 @@ describe('DocumentRow sender', () => {
it('highlights a receiver when matchedReceiverIds includes its id', async () => {
const item = makeItem({
document: {
...makeItem().document,
receivers: [{ id: 'r1', displayName: 'Onkel Karl' }]
},
receivers: [
{
id: 'r1',
lastName: 'Karl',
displayName: 'Onkel Karl',
personType: 'PERSON',
familyMember: false
}
],
matchData: {
...makeItem().matchData,
matchedReceiverIds: ['r1']
@@ -162,10 +166,7 @@ describe('DocumentRow sender', () => {
describe('DocumentRow summary', () => {
it('renders the document summary when present', async () => {
const item = makeItem({
document: {
...makeItem().document,
summary: 'Brief von Eugenie über die Heimreise aus dem Süden.'
}
});
render(DocumentRow, { item });
await expect
@@ -180,7 +181,7 @@ describe('DocumentRow summary', () => {
it('applies summary search-match highlight via summaryOffsets', async () => {
const item = makeItem({
document: { ...makeItem().document, summary: 'Brief über Menton' },
summary: 'Brief über Menton',
matchData: {
...makeItem().matchData,
summaryOffsets: [{ start: 11, length: 6 }]
@@ -196,25 +197,19 @@ describe('DocumentRow summary', () => {
describe('DocumentRow archive chips', () => {
it('renders the archive box chip when set', async () => {
const item = makeItem({
document: { ...makeItem().document, archiveBox: 'K3' }
});
const item = makeItem({ archiveBox: 'K3' });
render(DocumentRow, { item });
await expect.element(page.getByText('K3')).toBeInTheDocument();
});
it('renders the archive folder chip when set', async () => {
const item = makeItem({
document: { ...makeItem().document, archiveFolder: 'Mappe A' }
});
const item = makeItem({ archiveFolder: 'Mappe A' });
render(DocumentRow, { item });
await expect.element(page.getByText('Mappe A')).toBeInTheDocument();
});
it('renders the location chip when meta_location is set', async () => {
const item = makeItem({
document: { ...makeItem().document, location: 'Berlin' }
});
const item = makeItem({ location: 'Berlin' });
render(DocumentRow, { item });
await expect.element(page.getByText('Berlin')).toBeInTheDocument();
});
@@ -225,10 +220,7 @@ describe('DocumentRow archive chips', () => {
describe('DocumentRow tags', () => {
it('renders tag buttons', async () => {
const item = makeItem({
document: {
...makeItem().document,
tags: [{ id: 't1', name: 'Familie', color: null, parentId: null }]
}
tags: [{ id: 't1', name: 'Familie' }]
});
render(DocumentRow, { item });
await expect.element(page.getByRole('button', { name: 'Familie' })).toBeInTheDocument();
@@ -236,10 +228,7 @@ describe('DocumentRow tags', () => {
it('navigates to /documents?tag=… on tag click', async () => {
const item = makeItem({
document: {
...makeItem().document,
tags: [{ id: 't1', name: 'Urlaub & Reise', color: null, parentId: null }]
}
tags: [{ id: 't1', name: 'Urlaub & Reise' }]
});
render(DocumentRow, { item });
// Tailwind CSS isn't loaded in the vitest-browser client project, so the
@@ -255,10 +244,7 @@ describe('DocumentRow tags', () => {
it('tag click does not navigate to the document detail page', async () => {
const item = makeItem({
document: {
...makeItem().document,
tags: [{ id: 't2', name: 'Familie', color: null, parentId: null }]
}
tags: [{ id: 't2', name: 'Familie' }]
});
render(DocumentRow, { item });
const before = window.location.href;
@@ -281,7 +267,7 @@ describe('DocumentRow bulk selection checkbox', () => {
});
it('checkbox aria-label includes the document title', async () => {
const item = makeItem({ document: { ...makeItem().document, title: 'Brief an Anna' } });
const item = makeItem({ title: 'Brief an Anna' });
render(DocumentRow, { item, canWrite: true });
await expect
.element(page.getByRole('checkbox', { name: /Brief an Anna/i }))
@@ -289,7 +275,7 @@ describe('DocumentRow bulk selection checkbox', () => {
});
it('toggling the checkbox calls bulkSelectionStore.toggle', async () => {
const item = makeItem({ document: { ...makeItem().document, id: 'doc-42' } });
const item = makeItem({ id: 'doc-42' });
render(DocumentRow, { item, canWrite: true });
expect(bulkSelectionStore.has('doc-42')).toBe(false);
@@ -300,7 +286,7 @@ describe('DocumentRow bulk selection checkbox', () => {
it('checked state mirrors the store', async () => {
bulkSelectionStore.add('doc-99');
const item = makeItem({ document: { ...makeItem().document, id: 'doc-99' } });
const item = makeItem({ id: 'doc-99' });
render(DocumentRow, { item, canWrite: true });
await expect.element(page.getByRole('checkbox')).toBeChecked();
});

View File

@@ -20,10 +20,31 @@ const { default: DocumentRow } = await import('./DocumentRow.svelte');
afterEach(cleanup);
const sender = { id: 's1', displayName: 'Anna Schmidt' };
const receiver = { id: 'r1', displayName: 'Bert Meier' };
const sender = {
id: 's1',
lastName: 'Schmidt',
displayName: 'Anna Schmidt',
personType: 'PERSON' as const,
familyMember: false
};
const receiver = {
id: 'r1',
lastName: 'Meier',
displayName: 'Bert Meier',
personType: 'PERSON' as const,
familyMember: false
};
const makeDoc = (overrides: Record<string, unknown> = {}) => ({
const emptyMatchData = {
titleOffsets: [],
senderMatched: false,
matchedReceiverIds: [],
matchedTagIds: [],
snippetOffsets: [],
summaryOffsets: []
};
const baseItem = (overrides: Record<string, unknown> = {}) => ({
id: 'd1',
title: 'Brief 1923',
originalFilename: 'b.pdf',
@@ -31,20 +52,14 @@ const makeDoc = (overrides: Record<string, unknown> = {}) => ({
sender,
receivers: [receiver],
tags: [],
thumbnailUrl: null,
contentType: 'application/pdf',
summary: null,
archiveBox: null,
archiveFolder: null,
location: null,
...overrides
});
const baseItem = (docOverrides: Record<string, unknown> = {}) => ({
document: makeDoc(docOverrides),
matchData: null,
summary: undefined,
archiveBox: undefined,
archiveFolder: undefined,
location: undefined,
matchData: emptyMatchData,
completionPercentage: 0,
contributors: []
contributors: [],
...overrides
});
describe('DocumentRow', () => {
@@ -121,12 +136,9 @@ describe('DocumentRow', () => {
it('renders the snippet when matchData provides a transcriptionSnippet', async () => {
render(DocumentRow, {
props: {
item: {
document: makeDoc(),
matchData: { transcriptionSnippet: 'Hello world snippet' },
completionPercentage: 50,
contributors: []
}
item: baseItem({
matchData: { ...emptyMatchData, transcriptionSnippet: 'Hello world snippet' }
})
}
});

View File

@@ -2068,12 +2068,20 @@ export interface components {
};
ImportStatus: {
/** @enum {string} */
state?: "IDLE" | "RUNNING" | "DONE" | "FAILED";
statusCode?: string;
state: "IDLE" | "RUNNING" | "DONE" | "FAILED";
statusCode: string;
/** Format: int32 */
processed?: number;
processed: number;
skippedFiles: components["schemas"]["SkippedFile"][];
/** Format: date-time */
startedAt?: string;
/** Format: int32 */
skipped?: number;
};
SkippedFile: {
filename: string;
/** @enum {string} */
reason: "INVALID_FILENAME_PATH_TRAVERSAL" | "INVALID_PDF_SIGNATURE" | "FILE_READ_ERROR" | "ALREADY_EXISTS" | "S3_UPLOAD_FAILED";
};
BackfillStatus: {
/** @enum {string} */
@@ -2197,10 +2205,10 @@ export interface components {
totalStories: number;
};
PersonSummaryDTO: {
title?: string;
/** Format: uuid */
id?: string;
displayName?: string;
title?: string;
firstName?: string;
lastName?: string;
/** Format: int64 */
@@ -2307,14 +2315,14 @@ export interface components {
/** Format: int32 */
totalPages?: number;
pageable?: components["schemas"]["PageableObject"];
first?: boolean;
last?: boolean;
/** Format: int32 */
size?: number;
content?: components["schemas"]["NotificationDTO"][];
/** Format: int32 */
number?: number;
sort?: components["schemas"]["SortObject"];
first?: boolean;
last?: boolean;
/** Format: int32 */
numberOfElements?: number;
empty?: boolean;
@@ -2380,15 +2388,28 @@ export interface components {
/** Format: int32 */
totalPages?: number;
};
DocumentSearchItem: {
document: components["schemas"]["Document"];
matchData: components["schemas"]["SearchMatchData"];
DocumentListItem: {
/** Format: uuid */
id: string;
title: string;
originalFilename: string;
thumbnailUrl?: string;
/** Format: date */
documentDate?: string;
sender?: components["schemas"]["Person"];
receivers: components["schemas"]["Person"][];
tags: components["schemas"]["Tag"][];
archiveBox?: string;
archiveFolder?: string;
location?: string;
summary?: string;
/** Format: int32 */
completionPercentage: number;
contributors: components["schemas"]["ActivityActorDTO"][];
matchData: components["schemas"]["SearchMatchData"];
};
DocumentSearchResult: {
items: components["schemas"]["DocumentSearchItem"][];
items: components["schemas"]["DocumentListItem"][];
/** Format: int64 */
totalElements: number;
/** Format: int32 */

View File

@@ -5,7 +5,7 @@ import DocumentRow from '$lib/document/DocumentRow.svelte';
import { SvelteMap } from 'svelte/reactivity';
import type { components } from '$lib/generated/api';
type DocumentSearchItem = components['schemas']['DocumentSearchItem'];
type DocumentListItem = components['schemas']['DocumentListItem'];
type SortMode = 'DATE' | 'TITLE' | 'SENDER' | 'RECEIVER' | 'UPLOAD_DATE' | 'RELEVANCE';
@@ -17,7 +17,7 @@ let {
q = '',
sort = 'DATE'
}: {
items: DocumentSearchItem[];
items: DocumentListItem[];
canWrite: boolean;
error?: string | null;
total?: number;
@@ -31,10 +31,10 @@ const groups = $derived.by(() => {
return groupByYear(items);
});
function groupByYear(docItems: DocumentSearchItem[]) {
const map = new SvelteMap<string, DocumentSearchItem[]>();
function groupByYear(docItems: DocumentListItem[]) {
const map = new SvelteMap<string, DocumentListItem[]>();
for (const item of docItems) {
const label = item.document.documentDate?.substring(0, 4) ?? m.docs_group_undated();
const label = item.documentDate?.substring(0, 4) ?? m.docs_group_undated();
const bucket = map.get(label);
if (bucket) bucket.push(item);
else map.set(label, [item]);
@@ -42,10 +42,10 @@ function groupByYear(docItems: DocumentSearchItem[]) {
return Array.from(map.entries()).map(([label, groupItems]) => ({ label, items: groupItems }));
}
function groupBySender(docItems: DocumentSearchItem[]) {
const map = new SvelteMap<string, DocumentSearchItem[]>();
function groupBySender(docItems: DocumentListItem[]) {
const map = new SvelteMap<string, DocumentListItem[]>();
for (const item of docItems) {
const label = item.document.sender?.displayName ?? m.docs_group_unknown_sender();
const label = item.sender?.displayName ?? m.docs_group_unknown_sender();
const bucket = map.get(label);
if (bucket) bucket.push(item);
else map.set(label, [item]);
@@ -53,10 +53,10 @@ function groupBySender(docItems: DocumentSearchItem[]) {
return Array.from(map.entries()).map(([label, groupItems]) => ({ label, items: groupItems }));
}
function groupByReceiver(docItems: DocumentSearchItem[]) {
const map = new SvelteMap<string, DocumentSearchItem[]>();
function groupByReceiver(docItems: DocumentListItem[]) {
const map = new SvelteMap<string, DocumentListItem[]>();
for (const item of docItems) {
const receivers = item.document.receivers ?? [];
const receivers = item.receivers ?? [];
const labels =
receivers.length > 0
? receivers.map((r) => r.displayName)
@@ -99,7 +99,7 @@ function groupByReceiver(docItems: DocumentSearchItem[]) {
>
</div>
<ul class="divide-y divide-line">
{#each group.items as item (group.label + '-' + item.document.id)}
{#each group.items as item (group.label + '-' + item.id)}
<DocumentRow item={item} canWrite={canWrite} />
{/each}
</ul>

View File

@@ -8,24 +8,17 @@ vi.mock('$app/navigation', () => ({ goto: vi.fn() }));
afterEach(() => cleanup());
type DocumentSearchItem = components['schemas']['DocumentSearchItem'];
type DocumentListItem = components['schemas']['DocumentListItem'];
function makeItem(overrides: Partial<DocumentSearchItem> = {}): DocumentSearchItem {
function makeItem(overrides: Partial<DocumentListItem> = {}): DocumentListItem {
return {
document: {
id: '1',
title: 'Testbrief',
originalFilename: 'testbrief.pdf',
status: 'UPLOADED',
documentDate: '2024-03-15',
sender: undefined,
receivers: [],
tags: [],
createdAt: '2024-01-01T00:00:00Z',
updatedAt: '2024-01-01T00:00:00Z',
metadataComplete: false,
scriptType: 'UNKNOWN'
},
matchData: {
titleOffsets: [],
senderMatched: false,
@@ -75,8 +68,8 @@ describe('DocumentList empty state', () => {
describe('DocumentList year grouping', () => {
it('groups documents by year into separate cards', async () => {
const items = [
makeItem({ document: { ...makeItem().document, id: '1', documentDate: '1923-04-12' } }),
makeItem({ document: { ...makeItem().document, id: '2', documentDate: '1965-08-03' } })
makeItem({ id: '1', documentDate: '1923-04-12' }),
makeItem({ id: '2', documentDate: '1965-08-03' })
];
render(DocumentList, { ...baseProps, items, total: 2 });
const groupCards = page.getByTestId('group-card');
@@ -85,17 +78,15 @@ describe('DocumentList year grouping', () => {
});
it('uses undated label for items with no documentDate', async () => {
const items = [
makeItem({ document: { ...makeItem().document, id: '1', documentDate: undefined } })
];
const items = [makeItem({ id: '1', documentDate: undefined })];
render(DocumentList, { ...baseProps, items, total: 1 });
await expect.element(page.getByText('Undatiert')).toBeInTheDocument();
});
it('single year renders one group-card', async () => {
const items = [
makeItem({ document: { ...makeItem().document, id: '1', documentDate: '1938-01-01' } }),
makeItem({ document: { ...makeItem().document, id: '2', documentDate: '1938-06-15' } })
makeItem({ id: '1', documentDate: '1938-01-01' }),
makeItem({ id: '2', documentDate: '1938-06-15' })
];
render(DocumentList, { ...baseProps, items, total: 2 });
const groupCards = page.getByTestId('group-card');
@@ -108,9 +99,7 @@ describe('DocumentList year grouping', () => {
describe('DocumentList sort fallback', () => {
it('falls back to year grouping when sort is not SENDER or RECEIVER', async () => {
const items = [
makeItem({ document: { ...makeItem().document, id: '1', documentDate: '2024-03-15' } })
];
const items = [makeItem({ id: '1', documentDate: '2024-03-15' })];
render(DocumentList, { ...baseProps, items, total: 1, sort: 'TITLE' });
await expect
.element(page.getByTestId('group-header').filter({ hasText: '2024' }))
@@ -124,8 +113,6 @@ describe('DocumentList sender grouping', () => {
it('groups by sender displayName when sort is SENDER', async () => {
const items = [
makeItem({
document: {
...makeItem().document,
id: '1',
sender: {
id: 's1',
@@ -134,11 +121,8 @@ describe('DocumentList sender grouping', () => {
personType: 'PERSON',
familyMember: false
}
}
}),
makeItem({
document: {
...makeItem().document,
id: '2',
sender: {
id: 's2',
@@ -147,7 +131,6 @@ describe('DocumentList sender grouping', () => {
personType: 'PERSON',
familyMember: false
}
}
})
];
render(DocumentList, { ...baseProps, items, total: 2, sort: 'SENDER' });
@@ -167,10 +150,7 @@ describe('DocumentList sender grouping', () => {
personType: 'PERSON' as const,
familyMember: false
};
const items = [
makeItem({ document: { ...makeItem().document, id: '1', sender } }),
makeItem({ document: { ...makeItem().document, id: '2', sender } })
];
const items = [makeItem({ id: '1', sender }), makeItem({ id: '2', sender })];
render(DocumentList, { ...baseProps, items, total: 2, sort: 'SENDER' });
const cards = page.getByTestId('group-card');
await expect.element(cards.first()).toBeInTheDocument();
@@ -178,7 +158,7 @@ describe('DocumentList sender grouping', () => {
});
it('places items with no sender under fallback label', async () => {
const items = [makeItem({ document: { ...makeItem().document, id: '1', sender: undefined } })];
const items = [makeItem({ id: '1', sender: undefined })];
render(DocumentList, { ...baseProps, items, total: 1, sort: 'SENDER' });
await expect.element(page.getByText('Unbekannter Absender')).toBeInTheDocument();
});
@@ -190,8 +170,6 @@ describe('DocumentList receiver grouping', () => {
it('groups by receiver displayName when sort is RECEIVER', async () => {
const items = [
makeItem({
document: {
...makeItem().document,
id: '1',
receivers: [
{
@@ -202,7 +180,6 @@ describe('DocumentList receiver grouping', () => {
familyMember: false
}
]
}
})
];
render(DocumentList, { ...baseProps, items, total: 1, sort: 'RECEIVER' });
@@ -214,8 +191,6 @@ describe('DocumentList receiver grouping', () => {
it('duplicates a document into each receiver group', async () => {
const items = [
makeItem({
document: {
...makeItem().document,
id: '1',
title: 'Rundbriefchen',
receivers: [
@@ -234,7 +209,6 @@ describe('DocumentList receiver grouping', () => {
familyMember: false
}
]
}
})
];
render(DocumentList, { ...baseProps, items, total: 1, sort: 'RECEIVER' });
@@ -249,7 +223,7 @@ describe('DocumentList receiver grouping', () => {
});
it('places items with no receivers under fallback label', async () => {
const items = [makeItem({ document: { ...makeItem().document, id: '1', receivers: [] } })];
const items = [makeItem({ id: '1', receivers: [] })];
render(DocumentList, { ...baseProps, items, total: 1, sort: 'RECEIVER' });
await expect.element(page.getByText('Unbekannter Empfänger')).toBeInTheDocument();
});
@@ -261,7 +235,7 @@ describe('DocumentList DocumentRow delegation', () => {
it('shows transcription snippet when matchData has one', async () => {
const items = [
makeItem({
document: { ...makeItem().document, id: 'doc1' },
id: 'doc1',
matchData: {
transcriptionSnippet: 'Er schrieb einen langen Brief',
titleOffsets: [],
@@ -278,7 +252,7 @@ describe('DocumentList DocumentRow delegation', () => {
});
it('does not render snippet when matchData has no transcription snippet', async () => {
const items = [makeItem({ document: { ...makeItem().document, id: 'doc1' } })];
const items = [makeItem({ id: 'doc1' })];
render(DocumentList, { ...baseProps, items, total: 1 });
await expect.element(page.getByTestId('search-snippet')).not.toBeInTheDocument();
});
@@ -286,7 +260,8 @@ describe('DocumentList DocumentRow delegation', () => {
it('renders mark for title highlight when titleOffsets present', async () => {
const items = [
makeItem({
document: { ...makeItem().document, id: 'doc1', title: 'Brief an Anna' },
id: 'doc1',
title: 'Brief an Anna',
matchData: {
titleOffsets: [{ start: 0, length: 5 }], // "Brief"
senderMatched: false,

View File

@@ -20,11 +20,31 @@ const { default: DocumentList } = await import('./DocumentList.svelte');
afterEach(cleanup);
const sender = { id: 's1', displayName: 'Anna Schmidt' };
const receiver = { id: 'r1', displayName: 'Bert Meier' };
const sender = {
id: 's1',
lastName: 'Schmidt',
displayName: 'Anna Schmidt',
personType: 'PERSON' as const,
familyMember: false
};
const receiver = {
id: 'r1',
lastName: 'Meier',
displayName: 'Bert Meier',
personType: 'PERSON' as const,
familyMember: false
};
const emptyMatchData = {
titleOffsets: [],
senderMatched: false,
matchedReceiverIds: [],
matchedTagIds: [],
snippetOffsets: [],
summaryOffsets: []
};
const makeItem = (overrides: Record<string, unknown> = {}) => ({
document: {
id: 'd1',
title: 'Brief 1923',
originalFilename: 'b.pdf',
@@ -32,17 +52,14 @@ const makeItem = (overrides: Record<string, unknown> = {}) => ({
sender,
receivers: [receiver],
tags: [],
thumbnailUrl: null,
contentType: 'application/pdf',
summary: null,
archiveBox: null,
archiveFolder: null,
location: null,
...overrides
},
matchData: null,
summary: undefined,
archiveBox: undefined,
archiveFolder: undefined,
location: undefined,
matchData: emptyMatchData,
completionPercentage: 0,
contributors: []
contributors: [],
...overrides
});
describe('DocumentList', () => {
@@ -87,8 +104,26 @@ describe('DocumentList', () => {
render(DocumentList, {
props: {
items: [
makeItem({ id: 'd1', sender: { id: 's1', displayName: 'Anna Schmidt' } }),
makeItem({ id: 'd2', sender: { id: 's2', displayName: 'Bert Meier' } })
makeItem({
id: 'd1',
sender: {
id: 's1',
lastName: 'Schmidt',
displayName: 'Anna Schmidt',
personType: 'PERSON',
familyMember: false
}
}),
makeItem({
id: 'd2',
sender: {
id: 's2',
lastName: 'Meier',
displayName: 'Bert Meier',
personType: 'PERSON',
familyMember: false
}
})
],
canWrite: false,
sort: 'SENDER' as const

View File

@@ -20,7 +20,7 @@ async function resolvePersonName(
}
}
type DocumentSearchItem = components['schemas']['DocumentSearchItem'];
type DocumentListItem = components['schemas']['DocumentListItem'];
const VALID_SORTS = ['DATE', 'TITLE', 'SENDER', 'RECEIVER', 'UPLOAD_DATE', 'RELEVANCE'] as const;
type ValidSort = (typeof VALID_SORTS)[number];
@@ -77,7 +77,7 @@ export async function load({ url, fetch }) {
]);
} catch {
return {
items: [] as DocumentSearchItem[],
items: [] as DocumentListItem[],
totalElements: 0,
pageNumber: 0,
pageSize: PAGE_SIZE,
@@ -107,7 +107,7 @@ export async function load({ url, fetch }) {
: null;
return {
items: (result.data?.items ?? []) as DocumentSearchItem[],
items: (result.data?.items ?? []) as DocumentListItem[],
totalElements: result.data?.totalElements ?? 0,
pageNumber: result.data?.pageNumber ?? page,
pageSize: result.data?.pageSize ?? PAGE_SIZE,

View File

@@ -140,15 +140,12 @@ describe('documents/+ page', () => {
data: baseData({
items: [
{
document: {
id: 'd1',
title: 'Brief 1899',
status: 'TRANSCRIBED',
documentDate: '1899-04-14',
summary: '',
originalFilename: 'b1.pdf',
receivers: []
},
receivers: [],
tags: [],
matchData: {
titleOffsets: [],
senderMatched: false,