From 64b7b2315d9e0af605f5689597cc0e1c510f335c Mon Sep 17 00:00:00 2001 From: Marcel Date: Sat, 6 Jun 2026 23:42:23 +0200 Subject: [PATCH] docs(search): ADR-028 fix + glossary + C4 diagram for tag resolution (#743) Co-Authored-By: Claude Sonnet 4.6 --- docs/GLOSSARY.md | 8 +++++++- docs/adr/028-nl-search-ollama.md | 4 +++- docs/architecture/c4/l3-backend-3h-search.puml | 6 +++++- 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/GLOSSARY.md b/docs/GLOSSARY.md index 1c6e8cb6..7ee1b565 100644 --- a/docs/GLOSSARY.md +++ b/docs/GLOSSARY.md @@ -171,10 +171,16 @@ _See also [Chronik](#chronik-internal)._ **NlSearch** — the natural-language document search feature. Users type a plain-German query (e.g. "Was hat Walter im Krieg an Emma geschrieben?"); the backend parses it via Ollama, resolves person names to database UUIDs, and delegates to the standard `DocumentService.searchDocuments()` path. Endpoint: `POST /api/search/nl`. -**NlQueryInterpretation** — the structured result of parsing a natural-language query. Contains: `resolvedPersons` (persons whose names unambiguously matched one DB record), `ambiguousPersons` (all candidates when a name matched more than one person), `keywords` (LLM-extracted search terms), `dateFrom`/`dateTo` (extracted date range), `rawQuery` (the original user input), and `keywordsApplied` (whether keyword FTS was used in the search). +**NlQueryInterpretation** — the structured result of parsing a natural-language query. Contains: `resolvedPersons` (persons whose names unambiguously matched one DB record), `ambiguousPersons` (all candidates when a name matched more than one person), `keywords` (LLM-extracted search terms), `dateFrom`/`dateTo` (extracted date range), `rawQuery` (the original user input), `keywordsApplied` (whether keyword FTS was used), `resolvedTags` (tags matched by keyword→tag resolution), and `tagsApplied` (whether the OR-union tag filter was applied). + +**keyword→tag resolution** — the post-Ollama step in `NlQueryParserService` where each LLM-extracted keyword is substring-matched against the tag taxonomy via `TagService.findByNameContaining()`. Keywords that hit one or more tags are removed from the FTS text list and become an OR-union tag filter; keywords with no match remain as FTS text. Matching is case-insensitive and traverses the tag hierarchy via the recursive CTE `findDescendantIdsByName`. See ADR-033. **PersonHint** — a lightweight `{id, displayName}` pair used in `NlQueryInterpretation` to describe a resolved or ambiguous person without exposing the full `Person` entity to the frontend. +**TagHint** — a lightweight `{id, name, color?}` triple used in `NlQueryInterpretation.resolvedTags` to describe a tag matched by keyword→tag resolution. `color` is the tag's effective color (one-level inheritance from parent when the tag has no own color), or null if neither tag nor parent has a color. + +**theme chip** `[frontend]` — a removable chip rendered in `InterpretationChipRow` for each entry in `NlQueryInterpretation.resolvedTags` when `tagsApplied` is `true`. Displays "Thema: {tag.name}" (prefix varies by locale). Clicking × removes the tag from the OR-union filter and navigates to `/documents?tag=…&tagOp=OR` with remaining tag and person parameters preserved. + --- ## Infrastructure Terms diff --git a/docs/adr/028-nl-search-ollama.md b/docs/adr/028-nl-search-ollama.md index 41459992..ab166acc 100644 --- a/docs/adr/028-nl-search-ollama.md +++ b/docs/adr/028-nl-search-ollama.md @@ -26,7 +26,7 @@ Family members write their search intent in plain German ("Was hat Walter im Kri **DB-blind name resolution:** The Ollama prompt stays small (the raw query only); person database records are never sent to the model. Name resolution happens as a cheap SQL query after the model returns. This keeps the prompt short, avoids data leakage, and means adding 1,000 new persons requires no prompt change. -**Graceful degradation:** `RestClientOllamaClient.isHealthy()` is called inline before each inference request (calls `GET /api/tags` on a 2-second connect-timeout client). If Ollama is absent or times out, `NlQueryParserService` throws `DomainException` with `SMART_SEARCH_UNAVAILABLE` (HTTP 503). The regular structured search (`GET /api/documents/search`) is unaffected — it never calls Ollama. +**Graceful degradation:** In-path Ollama failures surface via `OllamaClient.parse()` — any `IOException`, read timeout, or non-2xx response is caught by `RestClientOllamaClient` and re-thrown as `DomainException(SMART_SEARCH_UNAVAILABLE, HTTP 503)`. `isHealthy()` has no callers inside `search/`; it is reserved for the ops/health-endpoint polling path only (e.g. a future `/api/health/ollama` endpoint). The regular structured search (`GET /api/documents/search`) is unaffected — it never calls Ollama. **Expected inference latency:** 2–15 seconds on the current CPU-only hardware. The frontend issue must show a persistent "Suche läuft…" indicator for the full duration (see `aria-live="polite"` requirement in issue #738 frontend notes). The backend timeout is 30 seconds (`app.ollama.timeout-seconds=30`) — chosen as a safe upper bound for Q4_K_M on the i7-6700 with a realistic 500-character query under modest concurrent load. @@ -44,6 +44,8 @@ Family members write their search intent in plain German ("Was hat Walter im Kri **`search/` → `person/` + `document/` dependency direction:** `NlQueryParserService` calls `PersonService.findByDisplayNameContaining()` and `DocumentService.searchDocuments()` — both are legitimate cross-domain service calls, not repository leaks. The `search/` package has no JPA entities of its own and never accesses `PersonRepository` or `DocumentRepository` directly. +**Keyword→tag resolution** (issue #743): After Ollama extracts the `keywords` list, `NlQueryParserService` calls `TagService.findByNameContaining()` for each keyword. Keywords that match one or more tags are removed from the FTS text list and added as OR-union tag filters; keywords with no tag match remain as FTS text. Resolved tags are returned to the frontend as `TagHint` objects in `NlQueryInterpretation.resolvedTags` and rendered as removable "Thema: X" chips. The `tagsApplied` flag signals whether the OR-union filter was actually passed to `DocumentService.searchDocuments()` — it is `false` when the `personRole:any` single-person path is taken, because that path has no tag filter slot. See ADR-033 for the tag name resolution and case-collision rules that `TagService.findByNameContaining()` relies on. + ## Decision **Introduce a new `search/` domain package** with a local Ollama integration via `RestClientOllamaClient`. The Ollama service runs as a separate Docker container, reachable only on the internal Docker network (`expose: ["11434"]`, not `ports:`). The backend calls Ollama's `/api/generate` endpoint with grammar-constrained JSON output. Name resolution and document search are performed by existing services after the model returns. diff --git a/docs/architecture/c4/l3-backend-3h-search.puml b/docs/architecture/c4/l3-backend-3h-search.puml index a0d643be..7ffa28be 100644 --- a/docs/architecture/c4/l3-backend-3h-search.puml +++ b/docs/architecture/c4/l3-backend-3h-search.puml @@ -10,7 +10,7 @@ Container(ollama, "Ollama", "ollama/ollama — port 11434 (internal only)") System_Boundary(backend, "API Backend (Spring Boot)") { Component(nlCtrl, "NlSearchController", "Spring MVC — POST /api/search/nl", "REST entry point for natural language search. Enforces READ_ALL permission. Uses @AuthenticationPrincipal UserDetails to obtain the caller's email for rate limiting. Delegates to NlQueryParserService and returns NlSearchResponse.") Component(rateLimiter, "NlSearchRateLimiter", "Spring Service", "Bucket4j + Caffeine LoadingCache keyed on user email. Allows 5 NL search requests per minute per user. Throws DomainException(SMART_SEARCH_RATE_LIMITED / HTTP 429) when the bucket is exhausted. Node-local — same caveat as LoginRateLimiter.") - Component(parserSvc, "NlQueryParserService", "Spring Service", "Orchestrates the full NL search pipeline: (1) validates query length, (2) calls OllamaClient.parse() to extract structured intent, (3) resolves each person name via PersonService.findByDisplayNameContaining(), (4) applies multi-name / personRole heuristics, (5) delegates to DocumentService.searchDocuments() or searchDocumentsByPersonId(). Returns NlSearchResponse. Never logs raw query content (PII).") + Component(parserSvc, "NlQueryParserService", "Spring Service", "Orchestrates the full NL search pipeline: (1) validates query length, (2) calls OllamaClient.parse() to extract structured intent, (3) resolves keywords to tags via TagService.findByNameContaining(), (4) resolves each person name via PersonService.findByDisplayNameContaining(), (5) applies multi-name / personRole heuristics, (6) delegates to DocumentService.searchDocuments() or searchDocumentsByPersonId(). Returns NlSearchResponse. Never logs raw query content (PII).") Component(ollamaClient, "RestClientOllamaClient", "Spring Service — implements OllamaClient + OllamaHealthClient", "HTTP client for the Ollama API. Uses two separate RestClient instances: inference client (30 s read timeout) and health-check client (2 s connect timeout). Calls POST /api/generate with grammar-constrained JSON schema (personNames, personRole, dateFrom, dateTo, keywords). isHealthy() polls GET /api/tags. Null-coalesces absent personNames/keywords to List.of(). Defaults unknown personRole to 'any' with a warning log. Maps timeout/5xx/parse errors to DomainException(SMART_SEARCH_UNAVAILABLE / HTTP 503).") Component(ollamaProps, "OllamaProperties", "@ConfigurationProperties(\"app.ollama\")", "Config bean: baseUrl, model (qwen2.5:7b-instruct-q4_K_M), timeoutSeconds (default: 30), healthCheckTimeoutSeconds (default: 2).") Component(rateLimitProps, "NlSearchRateLimitProperties", "@ConfigurationProperties(\"app.nl-search.rate-limit\")", "Config bean: maxRequestsPerMinute (default: 5).") @@ -18,6 +18,7 @@ System_Boundary(backend, "API Backend (Spring Boot)") { Component(personSvc, "PersonService", "Spring Service", "See diagram 3e. findByDisplayNameContaining(fragment) delegates to PersonRepository.searchByName() — covers first+last name, alias, and name aliases via LEFT JOIN.") Component(documentSvc, "DocumentService", "Spring Service", "See diagram 3b. searchDocuments() for keyword/sender/receiver/date queries. searchDocumentsByPersonId() for OR-semantics single-person queries (person as sender OR receiver, no keyword filter).") +Component(tagSvc, "TagService", "Spring Service", "See diagram 3b. findByNameContaining(fragment) delegates to TagRepository.findByNameContainingIgnoreCase(). resolveEffectiveColors() applies one-level color inheritance in-place on a collection of Tag entities.") Rel(frontend, nlCtrl, "POST /api/search/nl with JSON query", "HTTP / JSON") Rel(nlCtrl, rateLimiter, "checkAndConsume(userEmail)") @@ -25,9 +26,12 @@ Rel(nlCtrl, parserSvc, "parse(query)") Rel(parserSvc, ollamaClient, "parse(rawQuery) — extracts intent", "HTTP / JSON") Rel(ollamaClient, ollama, "POST /api/generate (grammar-constrained JSON schema)", "HTTP / REST") Rel(ollamaClient, ollama, "GET /api/tags (health check)", "HTTP / REST") +Rel(parserSvc, tagSvc, "findByNameContaining(keyword) — keyword→tag resolution") +Rel(parserSvc, tagSvc, "resolveEffectiveColors(tags)") Rel(parserSvc, personSvc, "findByDisplayNameContaining(name) for each extracted name") Rel(parserSvc, documentSvc, "searchDocuments() or searchDocumentsByPersonId()") Rel(documentSvc, db, "JPA queries", "JDBC") Rel(personSvc, db, "JPA queries", "JDBC") +Rel(tagSvc, db, "JPA queries", "JDBC") @enduml