Annotation-Backed Transcription

Three variations that reuse the existing annotation system (draw rectangle on PDF → linked content) as the backbone for transcription. Annotations get a type field: "comment" (existing behavior) or "transcription" (new — links a PDF region to a transcript block). Comments move from the annotation side panel into the transcription editor.

Familienarchiv
Reuse-first
2026-04-04 · @leonievoss
Core idea

Today, annotations are rectangles on the PDF that open a comment thread in the side panel. The insight: if we add a type field to DocumentAnnotation, the same draw-a-rectangle gesture can create either a comment annotation (existing) or a transcription annotation (new). A transcription annotation links a PDF region to an editable text block. The existing AnnotationLayer, PdfViewer, and CommentThread components all stay — we layer new behavior on top.

Comments no longer live under annotations. Instead, they live inside the transcript — anchored to text ranges, specific blocks, or as margin notes. This frees annotation rectangles to be purely spatial markers: “this region of the scan corresponds to this text.”

What stays, what changes
Reused as-is
  • AnnotationLayer — draw rects on PDF
  • PdfViewer — render, zoom, page nav
  • CommentThread — threaded replies, mentions
  • DocumentAnnotation model — add type field
  • DocumentComment model — unchanged
  • AnnotateHintStrip — new copy for transcribe mode
Repurposed
  • AnnotationSidePanel → becomes the transcript editor panel (same slot, different content)
  • annotateMode state → split into annotateMode + transcribeMode
  • Annotation color → turquoise for transcription, yellow for comments
New
  • transcription_blocks table — annotation_id, text, sort_order
  • Transcript editor component (right panel)
  • Inline comment anchoring (text-range or block-level)
  • type column on document_annotations
T

Draw-to-transcribe

Draw a rectangle around a passage on the scan. A transcript block appears in the editor, linked to that region. Type what you read. Rinse and repeat down the page. Others can join and work on different blocks.

Reuses: AnnotationLayer + PdfViewer + CommentThread · New: TranscriptBlock + type:transcription

V1 — Inline comment threads in transcript blocks

V1
Each annotation rectangle on the PDF creates a numbered transcript block in the right panel. Comments are inline threads inside each block — highlight a word or phrase, click “Diskutieren”, and a thread appears below the block text. Threads use the existing CommentThread component but are anchored to a text range within a block. Both annotation types (turquoise for transcription, yellow for comment) coexist on the same PDF.
Annotation-backed blocks + inline text-anchored threads — Google Docs-style comments within structured blocks.
Desktop · 1040px
MR
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
✎ Transkribieren
Annotieren
Transkribieren — Markiere eine Textpassage im Scan, um einen Transkriptions-Block anzulegen
Liebe Martha,
Dein Heinrich
1
2
3
4
💬
2
4 Blöcke
✓ Gespeichert
1
Anrede
Liebe Martha,
2
Hauptteil
Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
💬 Diskussion — “Breslau”
OI
Oma Inge · Ich bin sicher, das ist “Breslau” — Heinrich war dort im Lazarett.
DU
Du · Stimmt, danke! Lass ich so.
✓ Lösen
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss
In ewiger Liebe,
Dein Heinrich
Markiere eine weitere Passage im Scan, um Block 5 anzulegen
Block 2 aktiv Oma Inge · Block 2 1 offene Diskussion
Metadaten
Verlauf
Mobile · 320px
14:23••• WiFi 🔋
Brief von Heinrich, 14.05.1943 Transkr.
Liebe Martha,
4 Blöcke ✓ Gespeichert
1
Anrede
Liebe Martha,
2
Hauptteil Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau...
3
Familie Du
Die Kinder sollen wissen...

V1 · Inline comment threads in transcript blocks

/* Core flow: enter transcribe mode → crosshair cursor on PDF → draw rect → creates:
 *   1. DocumentAnnotation(type:"transcription", turquoise) in the DB
 *   2. TranscriptionBlock(annotation_id, text:"", sort_order:N) in the DB
 *   3. Editable block in the right panel, linked to the annotation
 * Clicking an annotation rect on PDF scrolls to + highlights the matching block.
 * Clicking a block header highlights the matching rect on PDF.
 * Comments: select text within a block → "Diskutieren" → creates a CommentThread
 *   anchored to (block_id, char_offset_start, char_offset_end).
 * Existing yellow comment annotations continue to work as before — they open the
 *   AnnotationSidePanel. Only turquoise annotations feed the transcript editor.
 * This reuses: AnnotationLayer (draw), PdfViewer (render), CommentThread (replies/mentions).
 * Mobile: PDF collapses to 90px strip, blocks stack vertically below. */
ElementValueNotes
Annotation reuse
Draw gestureExisting AnnotationLayer.onDraw(rect)Same pointer events. crosshair cursor.
Annotation colorturquoise (#00C7B1) for transcriptionYellow kept for comment annotations
Annotation typeNew column: type VARCHAR "transcription"|"comment"Default "comment" for backward compat
Number badge16px navy circle, top-left of rectSort order number, matches block number
Transcript blocks (right panel)
Block cardborder:1px line, radius:5px, active: turquoise glowHeader: number + label + presence. Body: contenteditable.
Inline threadorange left-border, orange-tint bg, below block bodyText-anchored via char offset. Reuses CommentThread.
Block labelEditable text, defaults: Anrede, Hauptteil, SchlussDouble-click to rename
Interaction
Click rect → blockscrollIntoView + active state on blockTurquoise glow on both rect and block
Click block → rectPDF scrolls/zooms to show the annotationIf multi-page: switches page
Delete blockDeletes annotation + block + threadsConfirm dialog if threads exist

V2 — Margin notes (manuscript-style)

V2
Same annotation-backed transcript blocks, but comments appear as margin notes beside the blocks rather than inline threads. Small note cards float to the right of the block they refer to, connected by a thin line — like handwritten marginalia on a manuscript. This feels more appropriate for a letter archive and avoids the visual weight of inline thread UIs. Notes can be replies (click existing note to add) or new (click the margin area next to a block).
Annotation-backed blocks + margin notes — lightweight, manuscript-style commenting that doesn’t break the reading flow.
Desktop · 1040px
MR
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
✎ Transkribieren
Transkribieren — Markiere Passagen im Scan. Klicke rechts neben einen Block für eine Randnotiz.
Liebe Martha,
Dein Heinrich
1
2
3
4
4 Blöcke 2 Randnotizen
✓ Gespeichert
1
Anrede
Liebe Martha,
2
Hauptteil
Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss
In ewiger Liebe,
Dein Heinrich
“Breslau” · Block 2
OI
Ich bin sicher: “Breslau”. Heinrich war dort stationiert.
DU
Danke, klingt richtig!
[unleserlich] · Block 2
DU
Könnte “sechs” oder “acht” sein. Wer hat die Originale?
Block 3 aktiv 2 offene Notizen
Metadaten
Verlauf

V2 · Margin notes (manuscript-style)

/* Same annotation-backed blocks as V1, different comment mechanism.
 * Comments appear as small cards in a 130px margin column to the right of the blocks.
 * Each note is connected to its source text via a thin horizontal line.
 * Notes are positioned vertically to align with the text they reference.
 * If notes would overlap, they stack downward with 8px gap.
 *
 * Creating a note: select text in a block → "Randnotiz" button or right-click context menu.
 * Or: click the margin area next to a block to create a general block note.
 * Note data model: CommentThread (documentId, blockId, charOffsetStart, charOffsetEnd).
 *
 * Advantages: doesn't disrupt reading flow. Feels like marginalia on a manuscript.
 * Disadvantages: narrow notes column — long discussions get cramped.
 *   For long threads: clicking "4 weitere..." expands the note into a popover.
 * Mobile: margin notes collapse to icons (small circles). Tap to expand inline. */
ElementValueNotes
Margin column
Width130px, flex-shrink:0To the right of blocks column
Note cardbg:white, border:line, radius:4px, shadow:card7px body text, 5px header
Connector line8px wide, 1px solid line, horizontalConnects left edge of note to block boundary
Vertical positionAligned to the referenced text lineStack with 8px gap if overlapping
Interactions
CreateSelect text → "Randnotiz" or click marginBlock-level or text-range-level
ReplyInput at bottom of note cardExisting CommentThread reply logic
Overflow"4 weitere..." link → expand to popoverPopover uses full CommentThread component
MobileNotes collapse to 10px circlesTap to expand inline below the block

V3 — Unified comment timeline

V3
Same annotation-backed blocks, but comments live in the existing bottom panel Discussion tab — not inside the transcript at all. Each comment in the timeline gets a reference tag showing which block (and optionally which word) it refers to. Clicking the tag scrolls both the transcript and PDF to the referenced location. This is the least invasive approach: the transcript editor stays clean and focused on text, while discussion happens in the familiar bottom panel.
Annotation-backed blocks + bottom panel discussion — clean editor, familiar comment UI, reference tags link comments to blocks.
Desktop · 1040px
MR
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
✎ Transkribieren
Transkribieren — Markiere Passagen im Scan. Nutze die Diskussion unten für Fragen.
Liebe Martha,
Dein Heinrich
1
2
3
4
4 Blöcke
✓ Gespeichert
1
Anrede
Liebe Martha,
2
Hauptteil 💬 2
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss
In ewiger Liebe,
Dein Heinrich
Block 3 aktiv Oma Inge sieht zu
Metadaten
Diskussion 3
Verlauf
OI
Oma Inge · vor 12 Min. §2 “Breslau”
Ich bin mir sicher, das ist “Breslau” — Heinrich war dort im Lazarett stationiert, das steht auch in dem Brief vom März.
Antworten
DU
Du · vor 5 Min. §2 [unleserlich]
Könnte “sechs” oder “acht” Wochen sein. Wer hat Zugang zu den Originalen, um nachzuschauen?
MR

V3 · Unified comment timeline

/* Simplest comment approach. The transcript editor is clean — no inline threads, no margins.
 * All discussion happens in the existing bottom panel Discussion tab.
 * Change: each comment gets an optional block_id + char_offset reference.
 *   When the user is editing a block and posts a comment, the reference is auto-attached.
 *   The reference renders as a clickable tag: "§2 'Breslau'" in accent-bg.
 *   Clicking the tag: scrolls transcript to block + highlights text, scrolls PDF to annotation.
 * Block headers show a small orange chat-bubble count when comments reference that block.
 *
 * How comments get block references:
 *   1. Auto: if cursor is in a block when posting, that block is referenced.
 *   2. Manual: select text in a block → right-click → "In Diskussion erwähnen" → opens comment
 *      input in bottom panel with the reference pre-filled.
 *   3. The §N tag in the comment is clickable — navigates to block + PDF region.
 *
 * Reuses: CommentThread (unchanged), bottom panel (unchanged), PanelDiscussion (add ref tag UI).
 * New: block_id + char_offset_start + char_offset_end on DocumentComment (nullable, backward compat).
 * Pro: least invasive, transcript stays clean. Con: discussion is physically separated from text. */
ElementValueNotes
Comment reference tag
Tag6px/600, accent-bg, mint border, radius:3pxShows: §N + quoted text (max 20 chars)
ClickScrolls transcript to block + PDF to annotationBoth get active/highlight state
Auto-attachIf cursor in block when posting → ref auto-setCan be removed before sending
Block header badge
Badgeorange chat-bubble icon + count, 6pxClick opens bottom panel filtered to that block
Data model
DocumentComment+ block_id (nullable UUID), + char_start, + char_endAll nullable — backward compat with existing comments

Implementation Guide — Annotation-Backed Transcription

1. Variation Comparison

Var.Comment mechanismTranscript editorReuse levelComplexity
V1Inline threads (Google Docs-style)Blocks with embedded thread UIHigh (AnnotationLayer, CommentThread)Medium
V2Margin notes (manuscript-style)Blocks + 130px margin columnHigh (AnnotationLayer, CommentThread)Medium
V3Bottom panel discussion + reference tagsClean blocks, no inline commentsVery high (everything reused)Low

2. Shared Foundation (all three variations)

Data model changes

Annotation color convention

TypeColorHexBehavior on click
CommentYellow#FFFF00Opens AnnotationSidePanel (existing)
TranscriptionTurquoise#00C7B1Highlights matching block in transcript editor

Component reuse map

Existing componentChange needed
AnnotationLayer.sveltePass type to onDraw callback. Render turquoise vs yellow based on annotation type. Add number badges for transcription annotations.
PdfViewer.svelteSplit handleAnnotationDraw into two paths based on current mode (annotate vs transcribe). Route handleAnnotationClick to either side panel or transcript editor.
AnnotationSidePanel.svelteNo change — still handles comment-type annotations.
CommentThread.svelteReused in V1 (inline threads), V2 (margin note popovers), V3 (bottom panel). No changes needed to the component itself.
AnnotateHintStrip.svelteNew variant or prop for transcribe mode copy: “Markiere eine Textpassage im Scan.”
DocumentBottomPanel.svelteV3: add block reference tags to discussion tab. V1/V2: remove Transcription tab (now inline).

3. Recommended Approach

Start with V3 (unified comment timeline) — it requires the least new UI and reuses the most existing components. The transcript editor is clean and focused. Comments flow naturally into the existing bottom panel Discussion tab. The only new UI element is the reference tag on comments.

Then layer V2 margin notes as an enhancement: users who prefer seeing comments next to the text can toggle a “Randnotizen” mode that pulls relevant comments from the timeline into margin cards. This is purely a view layer change — the data model stays the same.

V1 (inline threads) is the most feature-rich but also the most visually heavy. Consider it for a future iteration if users report that switching between transcript and discussion tab is too much friction.

4. Workflow: Draw-to-Transcribe

  1. User enters Transcribe mode (topbar button, turquoise). Hint strip appears.
  2. Crosshair cursor on PDF (same as annotate mode). User draws a rectangle around a handwriting passage.
  3. AnnotationLayer.onDraw(rect) fires. PdfViewer calls POST /api/documents/{id}/annotations with type: "transcription".
  4. Backend creates DocumentAnnotation + TranscriptionBlock (empty text, next sort_order).
  5. Frontend receives the created annotation + block. The transcript editor scrolls to the new empty block and focuses it.
  6. User types the transcription. Auto-save debounces to PATCH /api/transcription-blocks/{blockId}.
  7. Repeat: draw next rectangle, type next block.

5. Accessibility