# Annotation-Backed Transcription

Final spec for the collaborative inline transcription system. Draw turquoise rectangles on the scanned letter → numbered transcript blocks appear in a side panel → type what you read. Block-level comment threads with quoted selections for discussion. History in the transcript toolbar. No bottom panel.

Familienarchiv
Final spec
2026-04-04 · @leonievoss
Core concept — Draw-to-Transcribe

Today, annotations are rectangles on the PDF that open a comment thread in the side panel. By adding a `type` field to `DocumentAnnotation`, the same draw-a-rectangle gesture can create a transcription annotation (turquoise). A transcription annotation links a PDF region to an editable text block in the right panel.

Comments live inside transcript blocks as block-level threads. Users can select a word or phrase before commenting — the selection is auto-quoted into the comment message (e.g. `> “Breslau”`) rather than structurally anchored to character offsets. This avoids fragile offset tracking that breaks when text is edited. The quote is a display hint, not a structural anchor. Yellow comment annotations are disabled in transcribe mode — only turquoise transcription rectangles appear on the PDF.

T

## Draw-to-transcribe workflow

Draw a rectangle around a passage on the scan. A transcript block appears in the editor, linked to that region. Type what you read. Rinse and repeat down the page. Others can join and work on different blocks simultaneously.

Reuses: AnnotationLayer + PdfViewer + CommentThread · New: TranscriptBlock + TranscriptEditor + type:transcription
What stays, what changes, what’s new
Reused as-is

- `AnnotationLayer` — draw rects on PDF
- `PdfViewer` — render, zoom, page nav
- `CommentThread` — threaded replies, mentions
- `DocumentAnnotation` model — add `type` field
- `DocumentComment` model — unchanged

Repurposed

- `AnnotationSidePanel` slot → becomes the transcript editor panel
- `annotateMode` state → split into `annotateMode` + `transcribeMode`
- Annotation color → turquoise only in transcribe mode, yellow only in annotate mode (mutually exclusive)
- `AnnotateHintStrip` → new copy for transcribe mode

New

- `transcription_blocks` table
- Transcript editor component (right panel)
- Block-level comment threads (quoted selections)
- `type` column on `document_annotations`
- History in transcript toolbar
- Bottom panel removed (all modes)

### Desktop — transcribe mode active

S1
Two users are collaborating. Only turquoise transcription rectangles appear on the PDF — no yellow comment annotations in transcribe mode. One user (Oma Inge, purple) is editing Block 2. The current user (blue) is editing Block 3. Block 2 has a comment thread where Oma Inge quoted “Breslau” to discuss the reading. Each block has a “Kommentieren” button in its footer. The transcript toolbar shows “Verlauf” (history). No bottom panel.
Desktop · 1040px
←
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
Details ▼
✎ Transkribieren
Annotieren
Transkribieren — Markiere eine Textpassage im Scan, um einen Transkriptions-Block anzulegen
Liebe Martha,
Dein Heinrich
1
2
3
4
4 Blöcke
☰ Sortieren
🕑 Verlauf
✓ Gespeichert
1
Anrede ✓
Liebe Martha,
2
Hauptteil
Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
💬 2 Kommentare
OI
Oma Inge · vor 12 Min.
“Breslau”
Ich bin sicher, das ist “Breslau” — Heinrich war dort im Lazarett.
DU
Du · vor 8 Min.
Stimmt, danke! Lass ich so.
✓ Lösen
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss ✓
In ewiger Liebe,
Dein Heinrich
Markiere eine weitere Passage im Scan, um Block 5 anzulegen
Block 3 aktiv Oma Inge · Block 2 1 offene Diskussion

### Comment flow — select, quote, discuss

S2
The user has selected “[unleserlich]” in Block 2 and clicked “Kommentieren”. The comment input opens with the selection auto-quoted. After posting, the comment appears in the thread with the quote displayed as an indented blockquote. This shows the full lifecycle: selection → quoted input → posted comment.
Block-level threads + quoted selections — no char-offset anchoring, no fragile highlights. The quote is frozen text in the message body.
Desktop · comment input open with auto-quote
←
Brief von Heinrich an Martha, 14. Mai 1943
Details ▼
✎ Transkribieren
Annotieren
Liebe Martha,
2
4 Blöcke
✓ Gespeichert
2
Hauptteil
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
Neuer Kommentar zu Block 2
> “[unleserlich]” ✕ Zitat entfernen
3
Familie
Die Kinder sollen wissen, dass ich an sie denke.
💬 1 Kommentar
OI
Oma Inge · vor 5 Min.
“Die Kinder”
Fritz und Lotte. Fritz war damals 4, Lotte 7.
Block 2 aktiv2 Kommentare

#### Comment flow · Select → Quote → Discuss

    /* Comment flow for block-level threads with quoted selections:
     *
     * 1. TRIGGER: user clicks "Kommentieren" in block footer.
     *    Alternatively: Ctrl+Shift+K when block is focused.
     *
     * 2. AUTO-QUOTE: if text is selected in the block body (via mouse or keyboard),
     *    the selection is captured and pre-filled as a blockquote in the comment input:
     *      > "[unleserlich]"
     *    The user can edit or remove the quote before sending (× button).
     *    If no text was selected → input opens empty (general block comment).
     *
     * 3. STORAGE: the quote is stored as part of the comment `content` field.
     *    Markdown blockquote syntax: "> \"Breslau\"\nI think this is Breslau."
     *    The block_id FK on DocumentComment links the comment to its block.
     *    NO char_offset_start/end columns. The quote is just text.
     *
     * 4. DISPLAY: the quote renders as an indented italic line with a left border,
     *    above the comment text. It's visually distinct but structurally just content.
     *
     * 5. RESILIENCE: if the transcription text changes after quoting, nothing breaks.
     *    The quote is a frozen snapshot. The discussion context is preserved.
     *    Compare to char-offset anchoring where an edit would shift all offsets
     *    and potentially point to the wrong text.
     *
     * 6. THREAD: replies to a quoted comment don't need their own quotes —
     *    the parent comment provides context. Standard CommentThread reply flow.
     *
     * 7. MOBILE: "Kommentieren" button always visible in footer.
     *    Selecting text → auto-quote works the same via touch selection.
     *    Thread collapsed to "N Kommentare" row, tap to expand. */

Element | Value | Notes
--- | --- | ---
Comment input
Container | border:orange, bg:white, radius:4px, mx:8px | Appears below block body, above footer
Quote display | left-border:2px line, italic, 7px muted | Editable — user can modify or remove
Remove quote | "× Zitat entfernen" link, 5px, top-right of quote | Converts to general block comment
Input field | flex:1, 7px, border:line, bg:page, radius:3px | Auto-focuses when opened
Send button | "Senden", 6px/600, navy bg, white text | Enter to send, Shift+Enter for newline
Posted comment with quote
Quote in thread | left-border:2px line, italic, 7px muted | Read-only — frozen snapshot of selected text
Message below | 8px normal text, below the quote | Standard CommentThread message styling
Data model
block_id | UUID FK → transcription_blocks (nullable) | Links comment to its block
content | TEXT with markdown blockquote | > "quoted text"\nComment message
No char offsets | — | Intentional. See spec rationale.

### Desktop — history panel open

S3
Clicking “Verlauf” in the transcript toolbar opens a collapsible history panel between the toolbar and the block list. Shows recent changes with word-level diffs, just like the existing `PanelHistory` component but embedded in the transcript panel instead of the bottom panel.
Desktop · 1040px · history open
←
Brief von Heinrich an Martha, 14. Mai 1943
Details ▼
✎ Transkribieren
Annotieren
Transkribieren — Markiere eine Textpassage im Scan
Liebe Martha,
1
2
3
4
4 Blöcke
☰ Sortieren
🕑 Verlauf
✓ Gespeichert
Letzte Änderungen Alle anzeigen →
14:23 Oma Inge Block 2: ...Lazarett in BreslauBresla...
14:18 Du Block 3: Die Kinder sollen wissen, dass ich an sie denke.
14:12 Oma Inge Block 2: ich schreibe Dir heute aus dem Lazarett
14:05 Du Block 1: Liebe Martha,
1
Anrede ✓
Liebe Martha,
2
Hauptteil
ich schreibe Dir heute aus dem Lazarett in Breslau...
3
Familie
Die Kinder sollen wissen...
4
Schluss ✓
In ewiger Liebe,
Dein Heinrich
Block 3 aktiv✓ Gespeichert

### Mobile — transcribe mode

S4
On mobile, the PDF collapses to a 90px strip at the top. Annotation rectangles are visible as thin outlines. Transcript blocks stack vertically below. The history button is in the toolbar above the blocks. Inline threads expand in-place.
Mobile · 320px
14:23••• WiFi 🔋
← Brief von Heinrich, 14.05.1943 Transkr.
Liebe Martha,
4 Blöcke
🕑 Verlauf
✓
1
Anrede ✓
Liebe Martha,
2
Hauptteil Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau...
💬 1 Diskussion · “Breslau” ▼
3
Familie Du
Die Kinder sollen wissen...
4
Schluss ✓
In ewiger Liebe,
Dein Heinrich

#### Annotation-backed transcription · Core implementation spec

    /* Core flow: enter transcribe mode → crosshair cursor on PDF → draw rect → creates:
     *   1. DocumentAnnotation(type:"transcription", turquoise) in the DB
     *   2. TranscriptionBlock(annotation_id, text:"", sort_order:N) in the DB
     *   3. Editable block in the right panel, linked to the annotation
     * Clicking an annotation rect on PDF scrolls to + highlights the matching block.
     * Clicking a block header highlights the matching rect on PDF.
     *
     * COMMENTS: block-level threads with quoted selections.
     *   - Each block has a "Kommentieren" button in its footer.
     *   - If text is selected when clicking "Kommentieren", the selection is auto-quoted
     *     into the comment (> "Breslau"). The quote is plain text in the message body,
     *     NOT a structural char-offset anchor. It doesn't break when text changes.
     *   - Threads are anchored to block_id only (no char offsets).
     *   - Yellow comment annotations are DISABLED in transcribe mode.
     *     Only turquoise transcription rects on the PDF. One annotation type per mode.
     *
     * History: "Verlauf" button in transcript toolbar toggles a collapsible panel
     *   showing recent changes with word-level diffs per block.
     * Auto-save: debounced PATCH to /api/transcription-blocks/{blockId} (500ms).
     * Bottom panel: removed entirely (all modes). Metadata → topbar drawer. */

Element | Value | Notes
--- | --- | ---
Annotation reuse
Draw gesture | Existing AnnotationLayer.onDraw(rect) | Same pointer events. crosshair cursor.
Annotation color | turquoise (#00C7B1) for transcription | Yellow annotations disabled in transcribe mode
Annotation type | New column: type VARCHAR "transcription"|"comment" | Default "comment" for backward compat
Number badge | 16px navy circle, top-left of rect | Sort order number, matches block number
Transcript blocks (right panel)
Block card | border:1px line, radius:5px, active: turquoise glow | Header: number + label + presence. Body: contenteditable.
Block label | Editable text, defaults: Anrede, Hauptteil, Schluss | Double-click to rename
Empty state | Dashed border, "noch leer" italic text | Focus to start typing
Add block CTA | Dashed card: "Markiere eine Passage im Scan..." | Not clickable — directs user to draw on PDF
Block-level comment threads
Trigger | "Kommentieren" button in block footer | Always visible — no hover-reveal
Quoted selection | If text selected → auto-quoted into comment body | Plain text quote (> "Breslau"), NOT char-offset anchor
Quote display | Left border + italic, above the comment text | Decorative only — doesn't link to text range
Thread UI | orange left-border, orange-tint bg, below block body | Block-level anchor (block_id). Reuses CommentThread.
Footer hint | "Text markieren für Zitat" in 5px muted text | Only shown when block is active/focused
Resolve | "✓ Lösen" button collapses thread | Resolved threads hidden by default, toggle to show
Mobile | Threads collapsed to "2 Kommentare" row, tap to expand | Saves vertical space on small screens
Yellow annotations in transcribe mode
Status | Disabled — draw gesture only creates turquoise rects | Existing yellow annotations still visible (read-only)
Annotate mode | Still available via topbar "Annotieren" button | Exits transcribe mode, enters annotate mode (yellow)
History (transcript toolbar)
Toggle | "🕗 Verlauf" button in transcript toolbar | Active state: navy bg, white text
Panel | Collapsible, between toolbar and block list | bg:color-page, border:line, radius:5px
Entries | Time + user + block ref + word-level diff | Reuses diffWords from 'diff' library
"Alle anzeigen" | Link to full history view (reuses PanelHistory) | Opens in a modal or replaces block list temporarily
Interaction
Click rect → block | scrollIntoView + active state on block | Turquoise glow on both rect and block
Click block → rect | PDF scrolls/zooms to show the annotation | If multi-page: switches page
Delete block | Deletes annotation + block + threads | Confirm dialog if threads exist
Reorder blocks | Drag handle in block header | Updates sort_order via PATCH
Presence (collaborative)
Dots in topbar | Colored dot + user name, flex row | Max 3 shown, "+N" overflow
Block-level presence | Colored dot + name in block header | Left border color matches user
Implementation | WebSocket presence via Y.js (future) | MVP: polling-based, 5s interval
Auto-save
Debounce | 500ms after last keystroke | PATCH /api/transcription-blocks/{blockId}
Status | "✓ Gespeichert" in toolbar, fades after 3s | "Speichern..." while request in-flight
Conflict | Last-write-wins for MVP | Y.js CRDT for future collaborative editing

## Implementation Guide — Annotation-Backed Transcription

### 1. Data Model Changes

#### Flyway migration: `document_annotations`

- Add `type VARCHAR(20) NOT NULL DEFAULT 'comment'`.
- Values: `'comment'` (existing behavior) or `'transcription'`.
- Backward compatible — all existing annotations default to `'comment'`.

#### New table: `transcription_blocks`

Column | Type | Notes
--- | --- | ---
`id` | UUID PK | Generated
`annotation_id` | UUID FK → document_annotations | Links block to its PDF rectangle
`document_id` | UUID FK → documents | Denormalized for efficient queries
`text` | TEXT | The transcription content
`label` | VARCHAR(100) | "Anrede", "Hauptteil", etc.
`sort_order` | INT | Display order in the editor
`created_by` | UUID FK → app_users |
`updated_by` | UUID FK → app_users |
`created_at` | TIMESTAMP | @CreationTimestamp
`updated_at` | TIMESTAMP | @UpdateTimestamp

#### Block-level comments: `document_comments`

- Add `block_id UUID FK → transcription_blocks` (nullable).
- No char_offset columns. Quoted selections are stored as plain text in the comment `content` field using blockquote markdown syntax (`> “Breslau”`). This is intentional — char offsets break when text is edited and require OT/CRDT to maintain. Quotes are a display hint, not a structural anchor.
- Backward compatible — `block_id` is nullable, existing comments unaffected.

#### Backward compatibility: `Document.transcription`

The existing `transcription` TEXT field becomes a computed read-only view: `SELECT string_agg(text, E'\n\n' ORDER BY sort_order) FROM transcription_blocks WHERE document_id = ?`. Write operations go through the block API. This keeps search indexing, export, and the read-only `PanelTranscription` working without changes.

### 2. Annotation Color Convention & Mode Exclusivity

Type | Color | Hex | On click | When active
--- | --- | --- | --- | ---
Comment | Yellow | `#FFC800` | Opens AnnotationSidePanel (existing) | Annotate mode only
Transcription | Turquoise | `#00C7B1` | Highlights matching block in transcript editor | Transcribe mode only

Mode exclusivity: In transcribe mode, only turquoise rects can be drawn. Existing yellow comment annotations from annotate mode are still visible on the PDF (read-only, dimmed) but cannot be created or interacted with. The “Annotieren” button exits transcribe mode and enters annotate mode (and vice versa). This prevents overlapping annotation types and avoids user confusion about which comment system to use.

### 3. Component Architecture

Component | Change
--- | ---
`AnnotationLayer.svelte` | Pass `type` to `onDraw` callback. Render turquoise vs yellow based on annotation type. Add number badges for transcription annotations.
`PdfViewer.svelte` | Split `handleAnnotationDraw` into two paths (annotate vs transcribe). Route `handleAnnotationClick` to either side panel or transcript editor.
`AnnotationSidePanel.svelte` | No change — still handles comment-type annotations in annotate mode. Hidden in transcribe mode.
`TranscriptEditor.svelte` (new) | Right panel. Renders transcript toolbar + block list. Manages block CRUD, auto-save, block-level comment threads.
`TranscriptBlock.svelte` (new) | Single block card. contenteditable body, header with number/label/presence, footer with “Kommentieren” button, thread slot below body.
`BlockCommentThread.svelte` (new) | Comment thread anchored to a block. Shows quoted selections as blockquotes. Reuses `CommentThread` internally for replies/mentions.
`TranscriptToolbar.svelte` (new) | Block count, sort button, history toggle, save status.
`TranscriptHistory.svelte` (new) | Collapsible panel. Reuses `diffWords` from the `diff` library. Shows recent changes per block.
`DocumentBottomPanel.svelte` | Removed entirely. Metadata lives in the topbar drawer (see companion spec). Discussion, transcription, and history are all inline.
`documents/[id]/+page.svelte` | Add `transcribeMode` state. Conditionally render TranscriptEditor vs bottom panel.

### 4. API Endpoints

Method | Path | Notes
--- | --- | ---
POST | `/api/documents/{id}/annotations` | Existing, but now accepts `type` field. If `type="transcription"`, also creates a TranscriptionBlock.
GET | `/api/documents/{id}/transcription-blocks` | Returns all blocks ordered by sort_order.
PATCH | `/api/transcription-blocks/{blockId}` | Update text, label, or sort_order. Auto-save target.
DELETE | `/api/transcription-blocks/{blockId}` | Deletes block + its annotation + any anchored comments.
PATCH | `/api/transcription-blocks/reorder` | Bulk update sort_order for drag-and-drop reordering.

### 5. Draw-to-Transcribe Workflow

1. User enters Transcribe mode (topbar button, turquoise). Hint strip appears. Yellow comment annotations become read-only/dimmed. Only turquoise rects can be drawn.
2. Crosshair cursor on PDF (same as annotate mode). User draws a rectangle around a handwriting passage.
3. `AnnotationLayer.onDraw(rect)` fires. `PdfViewer` calls `POST /api/documents/{id}/annotations` with `type: "transcription"`.
4. Backend creates `DocumentAnnotation` + `TranscriptionBlock` (empty text, next sort_order).
5. Frontend receives the created annotation + block. The transcript editor scrolls to the new empty block and focuses it.
6. User types the transcription. Auto-save debounces to `PATCH /api/transcription-blocks/{blockId}`.
7. Repeat: draw next rectangle, type next block.

### 6. Comment Flow — Block-Level Threads with Quoted Selections

Comments are anchored to blocks, not character offsets. This is a deliberate simplification:

#### Why not char-offset anchoring?

- When someone edits the transcription text, all character offsets downstream shift.
- Keeping offsets in sync requires operational transforms (OT) or CRDT — that’s the Y.js future work, not MVP.
- A stale offset pointing to the wrong word is worse than a quoted snippet that no longer matches but still shows what was discussed.

#### How it works

1. User clicks “Kommentieren” in a block footer.
2. If text is selected in the block body, the selection is auto-quoted into the comment input: `> “Breslau”`. The user can edit or remove the quote before sending.
3. If no text is selected, the comment input opens empty — a general block-level comment.
4. The comment is saved as a `DocumentComment` with `block_id` set. The quoted text is part of the `content` field (markdown blockquote syntax).
5. The thread renders below the block body with an orange left-border. Quoted text appears as an indented italic blockquote above the comment message.
6. Replies work the same as existing `CommentThread` — no changes needed.

#### What happens when text changes after quoting?

Nothing breaks. The quote is a frozen snapshot of what the user selected. If “Bresla” was later corrected to “Breslau”, the original quote still reads `> “Bresla”` with Oma Inge’s comment “I think this is Breslau.” The context is preserved. No orphaned anchors, no broken highlights.

#### Footer hint

When a block is focused/active, the footer shows a subtle hint: “Text markieren für Zitat” (select text for a quote). This teaches the quoted-selection pattern without requiring documentation.

### 7. History in Transcript Toolbar

- The “Verlauf” button in the toolbar toggles `TranscriptHistory.svelte`.
- The panel renders between the toolbar and the block list (pushes blocks down).
- It shows recent changes per block, using `diffWords` from the `diff` library (same as existing `PanelHistory`).
- Each entry: timestamp, user name, block reference (e.g. “Block 2”), and a word-level diff snippet.
- “Alle anzeigen” opens a full history view — can reuse the existing `PanelHistory` component in a modal.
- Data source: the existing document version history API, filtered/grouped by block.

### 8. Accessibility

- Transcription blocks: `role="region"` with `aria-label="Transkriptions-Block N: [label]"`
- Block body: `contenteditable` with `aria-multiline="true"`
- Number badges on PDF: `aria-label="Transkriptions-Bereich N"`
- Comment button: `aria-label="Block N kommentieren"`
- History toggle: `aria-expanded`, `aria-controls="transcript-history"`
- Focus order: topbar → hint strip → PDF (for drawing) → transcript blocks (in sort order) → comment button → status bar
- Keyboard: Tab between blocks, Enter to edit, Escape to deselect. Ctrl+Shift+N to prompt draw on PDF. Ctrl+Shift+K to open comment on focused block.

### 9. Companion Spec

The expandable metadata header (labeled “Details ▼” toggle) is specified separately in `expandable-metadata-header-spec.html`. Together, these two specs fully eliminate the bottom panel in all modes: metadata → header drawer, transcription → inline split view, discussion → inline threads, history → transcript toolbar. One consistent pattern — no mode-dependent UI structure.