# Annotation-Backed Transcription

Three variations that reuse the existing annotation system (draw rectangle on PDF → linked content) as the backbone for transcription. Annotations get a `type` field: `"comment"` (existing behavior) or `"transcription"` (new — links a PDF region to a transcript block). Comments move from the annotation side panel into the transcription editor.

Familienarchiv
Reuse-first
2026-04-04 · @leonievoss
Core idea

Today, annotations are rectangles on the PDF that open a comment thread in the side panel. The insight: if we add a `type` field to `DocumentAnnotation`, the same draw-a-rectangle gesture can create either a comment annotation (existing) or a transcription annotation (new). A transcription annotation links a PDF region to an editable text block. The existing `AnnotationLayer`, `PdfViewer`, and `CommentThread` components all stay — we layer new behavior on top.

Comments no longer live under annotations. Instead, they live inside the transcript — anchored to text ranges, specific blocks, or as margin notes. This frees annotation rectangles to be purely spatial markers: “this region of the scan corresponds to this text.”

What stays, what changes
Reused as-is

- `AnnotationLayer` — draw rects on PDF
- `PdfViewer` — render, zoom, page nav
- `CommentThread` — threaded replies, mentions
- `DocumentAnnotation` model — add `type` field
- `DocumentComment` model — unchanged
- `AnnotateHintStrip` — new copy for transcribe mode

Repurposed

- `AnnotationSidePanel` → becomes the transcript editor panel (same slot, different content)
- `annotateMode` state → split into `annotateMode` + `transcribeMode`
- Annotation color → turquoise for transcription, yellow for comments

New

- `transcription_blocks` table — annotation_id, text, sort_order
- Transcript editor component (right panel)
- Inline comment anchoring (text-range or block-level)
- `type` column on `document_annotations`

T

## Draw-to-transcribe

Draw a rectangle around a passage on the scan. A transcript block appears in the editor, linked to that region. Type what you read. Rinse and repeat down the page. Others can join and work on different blocks.

Reuses: AnnotationLayer + PdfViewer + CommentThread · New: TranscriptBlock + type:transcription

### V1 — Inline comment threads in transcript blocks

V1
Each annotation rectangle on the PDF creates a numbered transcript block in the right panel. Comments are inline threads inside each block — highlight a word or phrase, click “Diskutieren”, and a thread appears below the block text. Threads use the existing `CommentThread` component but are anchored to a text range within a block. Both annotation types (turquoise for transcription, yellow for comment) coexist on the same PDF.
Annotation-backed blocks + inline text-anchored threads — Google Docs-style comments within structured blocks.
Desktop · 1040px
←
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
✎ Transkribieren
Annotieren
Transkribieren — Markiere eine Textpassage im Scan, um einen Transkriptions-Block anzulegen
Liebe Martha,
Dein Heinrich
1
2
3
4
💬
2
4 Blöcke
✓ Gespeichert
1
Anrede ✓
Liebe Martha,
2
Hauptteil
Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
💬 Diskussion — “Breslau”
OI
Oma Inge · Ich bin sicher, das ist “Breslau” — Heinrich war dort im Lazarett.
DU
Du · Stimmt, danke! Lass ich so.
✓ Lösen
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss ✓
In ewiger Liebe,
Dein Heinrich
Markiere eine weitere Passage im Scan, um Block 5 anzulegen
Block 2 aktiv Oma Inge · Block 2 1 offene Diskussion
Metadaten
Verlauf
Mobile · 320px
14:23••• WiFi 🔋
← Brief von Heinrich, 14.05.1943 Transkr.
Liebe Martha,
4 Blöcke ✓ Gespeichert
1
Anrede ✓
Liebe Martha,
2
Hauptteil Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau...
3
Familie Du
Die Kinder sollen wissen...

#### V1 · Inline comment threads in transcript blocks

    /* Core flow: enter transcribe mode → crosshair cursor on PDF → draw rect → creates:
     *   1. DocumentAnnotation(type:"transcription", turquoise) in the DB
     *   2. TranscriptionBlock(annotation_id, text:"", sort_order:N) in the DB
     *   3. Editable block in the right panel, linked to the annotation
     * Clicking an annotation rect on PDF scrolls to + highlights the matching block.
     * Clicking a block header highlights the matching rect on PDF.
     * Comments: select text within a block → "Diskutieren" → creates a CommentThread
     *   anchored to (block_id, char_offset_start, char_offset_end).
     * Existing yellow comment annotations continue to work as before — they open the
     *   AnnotationSidePanel. Only turquoise annotations feed the transcript editor.
     * This reuses: AnnotationLayer (draw), PdfViewer (render), CommentThread (replies/mentions).
     * Mobile: PDF collapses to 90px strip, blocks stack vertically below. */

Element | Value | Notes
--- | --- | ---
Annotation reuse
Draw gesture | Existing AnnotationLayer.onDraw(rect) | Same pointer events. crosshair cursor.
Annotation color | turquoise (#00C7B1) for transcription | Yellow kept for comment annotations
Annotation type | New column: type VARCHAR "transcription"|"comment" | Default "comment" for backward compat
Number badge | 16px navy circle, top-left of rect | Sort order number, matches block number
Transcript blocks (right panel)
Block card | border:1px line, radius:5px, active: turquoise glow | Header: number + label + presence. Body: contenteditable.
Inline thread | orange left-border, orange-tint bg, below block body | Text-anchored via char offset. Reuses CommentThread.
Block label | Editable text, defaults: Anrede, Hauptteil, Schluss | Double-click to rename
Interaction
Click rect → block | scrollIntoView + active state on block | Turquoise glow on both rect and block
Click block → rect | PDF scrolls/zooms to show the annotation | If multi-page: switches page
Delete block | Deletes annotation + block + threads | Confirm dialog if threads exist

### V2 — Margin notes (manuscript-style)

V2
Same annotation-backed transcript blocks, but comments appear as margin notes beside the blocks rather than inline threads. Small note cards float to the right of the block they refer to, connected by a thin line — like handwritten marginalia on a manuscript. This feels more appropriate for a letter archive and avoids the visual weight of inline thread UIs. Notes can be replies (click existing note to add) or new (click the margin area next to a block).
Annotation-backed blocks + margin notes — lightweight, manuscript-style commenting that doesn’t break the reading flow.
Desktop · 1040px
←
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
✎ Transkribieren
Transkribieren — Markiere Passagen im Scan. Klicke rechts neben einen Block für eine Randnotiz.
Liebe Martha,
Dein Heinrich
1
2
3
4
4 Blöcke 2 Randnotizen
✓ Gespeichert
1
Anrede ✓
Liebe Martha,
2
Hauptteil
Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss ✓
In ewiger Liebe,
Dein Heinrich
“Breslau” · Block 2
OI
Ich bin sicher: “Breslau”. Heinrich war dort stationiert.
DU
Danke, klingt richtig!
[unleserlich] · Block 2
DU
Könnte “sechs” oder “acht” sein. Wer hat die Originale?
Block 3 aktiv 2 offene Notizen
Metadaten
Verlauf

#### V2 · Margin notes (manuscript-style)

    /* Same annotation-backed blocks as V1, different comment mechanism.
     * Comments appear as small cards in a 130px margin column to the right of the blocks.
     * Each note is connected to its source text via a thin horizontal line.
     * Notes are positioned vertically to align with the text they reference.
     * If notes would overlap, they stack downward with 8px gap.
     *
     * Creating a note: select text in a block → "Randnotiz" button or right-click context menu.
     * Or: click the margin area next to a block to create a general block note.
     * Note data model: CommentThread (documentId, blockId, charOffsetStart, charOffsetEnd).
     *
     * Advantages: doesn't disrupt reading flow. Feels like marginalia on a manuscript.
     * Disadvantages: narrow notes column — long discussions get cramped.
     *   For long threads: clicking "4 weitere..." expands the note into a popover.
     * Mobile: margin notes collapse to icons (small circles). Tap to expand inline. */

Element | Value | Notes
--- | --- | ---
Margin column
Width | 130px, flex-shrink:0 | To the right of blocks column
Note card | bg:white, border:line, radius:4px, shadow:card | 7px body text, 5px header
Connector line | 8px wide, 1px solid line, horizontal | Connects left edge of note to block boundary
Vertical position | Aligned to the referenced text line | Stack with 8px gap if overlapping
Interactions
Create | Select text → "Randnotiz" or click margin | Block-level or text-range-level
Reply | Input at bottom of note card | Existing CommentThread reply logic
Overflow | "4 weitere..." link → expand to popover | Popover uses full CommentThread component
Mobile | Notes collapse to 10px circles | Tap to expand inline below the block

### V3 — Unified comment timeline

V3
Same annotation-backed blocks, but comments live in the existing bottom panel Discussion tab — not inside the transcript at all. Each comment in the timeline gets a reference tag showing which block (and optionally which word) it refers to. Clicking the tag scrolls both the transcript and PDF to the referenced location. This is the least invasive approach: the transcript editor stays clean and focused on text, while discussion happens in the familiar bottom panel.
Annotation-backed blocks + bottom panel discussion — clean editor, familiar comment UI, reference tags link comments to blocks.
Desktop · 1040px
←
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
✎ Transkribieren
Transkribieren — Markiere Passagen im Scan. Nutze die Diskussion unten für Fragen.
Liebe Martha,
Dein Heinrich
1
2
3
4
4 Blöcke
✓ Gespeichert
1
Anrede ✓
Liebe Martha,
2
Hauptteil 💬 2
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss ✓
In ewiger Liebe,
Dein Heinrich
Block 3 aktiv Oma Inge sieht zu
Metadaten
Diskussion 3
Verlauf
OI
Oma Inge · vor 12 Min. §2 “Breslau”
Ich bin mir sicher, das ist “Breslau” — Heinrich war dort im Lazarett stationiert, das steht auch in dem Brief vom März.
Antworten
DU
Du · vor 5 Min. §2 [unleserlich]
Könnte “sechs” oder “acht” Wochen sein. Wer hat Zugang zu den Originalen, um nachzuschauen?
MR

#### V3 · Unified comment timeline

    /* Simplest comment approach. The transcript editor is clean — no inline threads, no margins.
     * All discussion happens in the existing bottom panel Discussion tab.
     * Change: each comment gets an optional block_id + char_offset reference.
     *   When the user is editing a block and posts a comment, the reference is auto-attached.
     *   The reference renders as a clickable tag: "§2 'Breslau'" in accent-bg.
     *   Clicking the tag: scrolls transcript to block + highlights text, scrolls PDF to annotation.
     * Block headers show a small orange chat-bubble count when comments reference that block.
     *
     * How comments get block references:
     *   1. Auto: if cursor is in a block when posting, that block is referenced.
     *   2. Manual: select text in a block → right-click → "In Diskussion erwähnen" → opens comment
     *      input in bottom panel with the reference pre-filled.
     *   3. The §N tag in the comment is clickable — navigates to block + PDF region.
     *
     * Reuses: CommentThread (unchanged), bottom panel (unchanged), PanelDiscussion (add ref tag UI).
     * New: block_id + char_offset_start + char_offset_end on DocumentComment (nullable, backward compat).
     * Pro: least invasive, transcript stays clean. Con: discussion is physically separated from text. */

Element | Value | Notes
--- | --- | ---
Comment reference tag
Tag | 6px/600, accent-bg, mint border, radius:3px | Shows: §N + quoted text (max 20 chars)
Click | Scrolls transcript to block + PDF to annotation | Both get active/highlight state
Auto-attach | If cursor in block when posting → ref auto-set | Can be removed before sending
Block header badge
Badge | orange chat-bubble icon + count, 6px | Click opens bottom panel filtered to that block
Data model
DocumentComment | + block_id (nullable UUID), + char_start, + char_end | All nullable — backward compat with existing comments

## Implementation Guide — Annotation-Backed Transcription

### 1. Variation Comparison

Var. | Comment mechanism | Transcript editor | Reuse level | Complexity
--- | --- | --- | --- | ---
V1 | Inline threads (Google Docs-style) | Blocks with embedded thread UI | High (AnnotationLayer, CommentThread) | Medium
V2 | Margin notes (manuscript-style) | Blocks + 130px margin column | High (AnnotationLayer, CommentThread) | Medium
V3 | Bottom panel discussion + reference tags | Clean blocks, no inline comments | Very high (everything reused) | Low

### 2. Shared Foundation (all three variations)

#### Data model changes

- `document_annotations`: add `type VARCHAR DEFAULT 'comment'`. Values: `'comment'` (existing behavior) or `'transcription'`.
- New table `transcription_blocks`: `id UUID PK, annotation_id UUID FK, document_id UUID FK, text TEXT, label VARCHAR, sort_order INT, created_by UUID, updated_by UUID, updated_at TIMESTAMP`
- The full transcript = `SELECT text FROM transcription_blocks WHERE document_id = ? ORDER BY sort_order`, concatenated.
- Backward compatibility: the existing `Document.transcription` field becomes a computed read-only view (concatenation of blocks). Write operations go through blocks.

#### Annotation color convention

Type | Color | Hex | Behavior on click
--- | --- | --- | ---
Comment | Yellow | `#FFFF00` | Opens AnnotationSidePanel (existing)
Transcription | Turquoise | `#00C7B1` | Highlights matching block in transcript editor

#### Component reuse map

Existing component | Change needed
--- | ---
`AnnotationLayer.svelte` | Pass `type` to `onDraw` callback. Render turquoise vs yellow based on annotation type. Add number badges for transcription annotations.
`PdfViewer.svelte` | Split `handleAnnotationDraw` into two paths based on current mode (annotate vs transcribe). Route `handleAnnotationClick` to either side panel or transcript editor.
`AnnotationSidePanel.svelte` | No change — still handles comment-type annotations.
`CommentThread.svelte` | Reused in V1 (inline threads), V2 (margin note popovers), V3 (bottom panel). No changes needed to the component itself.
`AnnotateHintStrip.svelte` | New variant or prop for transcribe mode copy: “Markiere eine Textpassage im Scan.”
`DocumentBottomPanel.svelte` | V3: add block reference tags to discussion tab. V1/V2: remove Transcription tab (now inline).

### 3. Recommended Approach

Start with V3 (unified comment timeline) — it requires the least new UI and reuses the most existing components. The transcript editor is clean and focused. Comments flow naturally into the existing bottom panel Discussion tab. The only new UI element is the reference tag on comments.

Then layer V2 margin notes as an enhancement: users who prefer seeing comments next to the text can toggle a “Randnotizen” mode that pulls relevant comments from the timeline into margin cards. This is purely a view layer change — the data model stays the same.

V1 (inline threads) is the most feature-rich but also the most visually heavy. Consider it for a future iteration if users report that switching between transcript and discussion tab is too much friction.

### 4. Workflow: Draw-to-Transcribe

1. User enters Transcribe mode (topbar button, turquoise). Hint strip appears.
2. Crosshair cursor on PDF (same as annotate mode). User draws a rectangle around a handwriting passage.
3. `AnnotationLayer.onDraw(rect)` fires. `PdfViewer` calls `POST /api/documents/{id}/annotations` with `type: "transcription"`.
4. Backend creates `DocumentAnnotation` + `TranscriptionBlock` (empty text, next sort_order).
5. Frontend receives the created annotation + block. The transcript editor scrolls to the new empty block and focuses it.
6. User types the transcription. Auto-save debounces to `PATCH /api/transcription-blocks/{blockId}`.
7. Repeat: draw next rectangle, type next block.

### 5. Accessibility

- Transcription blocks: `role="region"` with `aria-label="Transkriptions-Block N: [label]"`
- Block body: `contenteditable` with `aria-multiline="true"`
- Number badges on PDF: `aria-label="Transkriptions-Bereich N"`
- Comment reference tags: `role="link"` with descriptive `aria-label`
- Focus order: hint strip → PDF (for drawing) → transcript blocks (in sort order) → bottom panel
- Keyboard: Tab between blocks, Enter to edit, Escape to deselect. Ctrl+Shift+N to create new block (prompts draw on PDF).