Annotation-Backed Transcription

Final spec for the collaborative inline transcription system. Draw turquoise rectangles on the scanned letter → numbered transcript blocks appear in a side panel → type what you read. Block-level comment threads with quoted selections for discussion. History in the transcript toolbar. No bottom panel.

Familienarchiv
Final spec
2026-04-04 · @leonievoss
Core concept — Draw-to-Transcribe

Today, annotations are rectangles on the PDF that open a comment thread in the side panel. By adding a type field to DocumentAnnotation, the same draw-a-rectangle gesture can create a transcription annotation (turquoise). A transcription annotation links a PDF region to an editable text block in the right panel.

Comments live inside transcript blocks as block-level threads. Users can select a word or phrase before commenting — the selection is auto-quoted into the comment message (e.g. > “Breslau”) rather than structurally anchored to character offsets. This avoids fragile offset tracking that breaks when text is edited. The quote is a display hint, not a structural anchor. Yellow comment annotations are disabled in transcribe mode — only turquoise transcription rectangles appear on the PDF.

T

Draw-to-transcribe workflow

Draw a rectangle around a passage on the scan. A transcript block appears in the editor, linked to that region. Type what you read. Rinse and repeat down the page. Others can join and work on different blocks simultaneously.

Reuses: AnnotationLayer + PdfViewer + CommentThread · New: TranscriptBlock + TranscriptEditor + type:transcription
What stays, what changes, what’s new
Reused as-is
  • AnnotationLayer — draw rects on PDF
  • PdfViewer — render, zoom, page nav
  • CommentThread — threaded replies, mentions
  • DocumentAnnotation model — add type field
  • DocumentComment model — unchanged
Repurposed
  • AnnotationSidePanel slot → becomes the transcript editor panel
  • annotateMode state → split into annotateMode + transcribeMode
  • Annotation color → turquoise only in transcribe mode, yellow only in annotate mode (mutually exclusive)
  • AnnotateHintStrip → new copy for transcribe mode
New
  • transcription_blocks table
  • Transcript editor component (right panel)
  • Block-level comment threads (quoted selections)
  • type column on document_annotations
  • History in transcript toolbar
  • Bottom panel removed (all modes)

Desktop — transcribe mode active

S1
Two users are collaborating. Only turquoise transcription rectangles appear on the PDF — no yellow comment annotations in transcribe mode. One user (Oma Inge, purple) is editing Block 2. The current user (blue) is editing Block 3. Block 2 has a comment thread where Oma Inge quoted “Breslau” to discuss the reading. Each block has a “Kommentieren” button in its footer. The transcript toolbar shows “Verlauf” (history). No bottom panel.
Desktop · 1040px
MR
Brief von Heinrich an Martha, 14. Mai 1943
Du
Oma Inge
Details ▼
✎ Transkribieren
Annotieren
Transkribieren — Markiere eine Textpassage im Scan, um einen Transkriptions-Block anzulegen
Liebe Martha,
Dein Heinrich
1
2
3
4
4 Blöcke
☰ Sortieren
🕑 Verlauf
✓ Gespeichert
1
Anrede
Liebe Martha,
2
Hauptteil
Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen, es geht mir den Umständen entsprechend gut. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
💬 2 Kommentare
OI
Oma Inge · vor 12 Min.
“Breslau”
Ich bin sicher, das ist “Breslau” — Heinrich war dort im Lazarett.
DU
Du · vor 8 Min.
Stimmt, danke! Lass ich so.
✓ Lösen
3
Familie
Du
Die Kinder sollen wissen, dass ich an sie denke. Sag dem kleinen Fritz, er soll auf seine Mutter aufpassen.
4
Schluss
In ewiger Liebe,
Dein Heinrich
Markiere eine weitere Passage im Scan, um Block 5 anzulegen
Block 3 aktiv Oma Inge · Block 2 1 offene Diskussion

Comment flow — select, quote, discuss

S2
The user has selected “[unleserlich]” in Block 2 and clicked “Kommentieren”. The comment input opens with the selection auto-quoted. After posting, the comment appears in the thread with the quote displayed as an indented blockquote. This shows the full lifecycle: selection → quoted input → posted comment.
Block-level threads + quoted selections — no char-offset anchoring, no fragile highlights. The quote is frozen text in the message body.
Desktop · comment input open with auto-quote
MR
Brief von Heinrich an Martha, 14. Mai 1943
Details ▼
✎ Transkribieren
Annotieren
Liebe Martha,
2
4 Blöcke
✓ Gespeichert
2
Hauptteil
ich schreibe Dir heute aus dem Lazarett in Breslau. Mach Dir keine Sorgen. Der Arzt sagt [unleserlich] Wochen noch dauern wird.
Neuer Kommentar zu Block 2
> “[unleserlich]” ✕ Zitat entfernen
3
Familie
Die Kinder sollen wissen, dass ich an sie denke.
💬 1 Kommentar
OI
Oma Inge · vor 5 Min.
“Die Kinder”
Fritz und Lotte. Fritz war damals 4, Lotte 7.
Block 2 aktiv2 Kommentare

Comment flow · Select → Quote → Discuss

/* Comment flow for block-level threads with quoted selections:
 *
 * 1. TRIGGER: user clicks "Kommentieren" in block footer.
 *    Alternatively: Ctrl+Shift+K when block is focused.
 *
 * 2. AUTO-QUOTE: if text is selected in the block body (via mouse or keyboard),
 *    the selection is captured and pre-filled as a blockquote in the comment input:
 *      > "[unleserlich]"
 *    The user can edit or remove the quote before sending (× button).
 *    If no text was selected → input opens empty (general block comment).
 *
 * 3. STORAGE: the quote is stored as part of the comment `content` field.
 *    Markdown blockquote syntax: "> \"Breslau\"\nI think this is Breslau."
 *    The block_id FK on DocumentComment links the comment to its block.
 *    NO char_offset_start/end columns. The quote is just text.
 *
 * 4. DISPLAY: the quote renders as an indented italic line with a left border,
 *    above the comment text. It's visually distinct but structurally just content.
 *
 * 5. RESILIENCE: if the transcription text changes after quoting, nothing breaks.
 *    The quote is a frozen snapshot. The discussion context is preserved.
 *    Compare to char-offset anchoring where an edit would shift all offsets
 *    and potentially point to the wrong text.
 *
 * 6. THREAD: replies to a quoted comment don't need their own quotes —
 *    the parent comment provides context. Standard CommentThread reply flow.
 *
 * 7. MOBILE: "Kommentieren" button always visible in footer.
 *    Selecting text → auto-quote works the same via touch selection.
 *    Thread collapsed to "N Kommentare" row, tap to expand. */
ElementValueNotes
Comment input
Containerborder:orange, bg:white, radius:4px, mx:8pxAppears below block body, above footer
Quote displayleft-border:2px line, italic, 7px mutedEditable — user can modify or remove
Remove quote"× Zitat entfernen" link, 5px, top-right of quoteConverts to general block comment
Input fieldflex:1, 7px, border:line, bg:page, radius:3pxAuto-focuses when opened
Send button"Senden", 6px/600, navy bg, white textEnter to send, Shift+Enter for newline
Posted comment with quote
Quote in threadleft-border:2px line, italic, 7px mutedRead-only — frozen snapshot of selected text
Message below8px normal text, below the quoteStandard CommentThread message styling
Data model
block_idUUID FK → transcription_blocks (nullable)Links comment to its block
contentTEXT with markdown blockquote> "quoted text"\nComment message
No char offsetsIntentional. See spec rationale.

Desktop — history panel open

S3
Clicking “Verlauf” in the transcript toolbar opens a collapsible history panel between the toolbar and the block list. Shows recent changes with word-level diffs, just like the existing PanelHistory component but embedded in the transcript panel instead of the bottom panel.
Desktop · 1040px · history open
MR
Brief von Heinrich an Martha, 14. Mai 1943
Details ▼
✎ Transkribieren
Annotieren
Transkribieren — Markiere eine Textpassage im Scan
Liebe Martha,
1
2
3
4
4 Blöcke
☰ Sortieren
🕑 Verlauf
✓ Gespeichert
Letzte Änderungen Alle anzeigen →
14:23 Oma Inge Block 2: ...Lazarett in BreslauBresla...
14:18 Du Block 3: Die Kinder sollen wissen, dass ich an sie denke.
14:12 Oma Inge Block 2: ich schreibe Dir heute aus dem Lazarett
14:05 Du Block 1: Liebe Martha,
1
Anrede
Liebe Martha,
2
Hauptteil
ich schreibe Dir heute aus dem Lazarett in Breslau...
3
Familie
Die Kinder sollen wissen...
4
Schluss
In ewiger Liebe,
Dein Heinrich
Block 3 aktiv✓ Gespeichert

Mobile — transcribe mode

S4
On mobile, the PDF collapses to a 90px strip at the top. Annotation rectangles are visible as thin outlines. Transcript blocks stack vertically below. The history button is in the toolbar above the blocks. Inline threads expand in-place.
Mobile · 320px
14:23••• WiFi 🔋
Brief von Heinrich, 14.05.1943 Transkr.
Liebe Martha,
4 Blöcke
🕑 Verlauf
1
Anrede
Liebe Martha,
2
Hauptteil Oma Inge
ich schreibe Dir heute aus dem Lazarett in Breslau...
💬 1 Diskussion · “Breslau”
3
Familie Du
Die Kinder sollen wissen...
4
Schluss
In ewiger Liebe,
Dein Heinrich

Annotation-backed transcription · Core implementation spec

/* Core flow: enter transcribe mode → crosshair cursor on PDF → draw rect → creates:
 *   1. DocumentAnnotation(type:"transcription", turquoise) in the DB
 *   2. TranscriptionBlock(annotation_id, text:"", sort_order:N) in the DB
 *   3. Editable block in the right panel, linked to the annotation
 * Clicking an annotation rect on PDF scrolls to + highlights the matching block.
 * Clicking a block header highlights the matching rect on PDF.
 *
 * COMMENTS: block-level threads with quoted selections.
 *   - Each block has a "Kommentieren" button in its footer.
 *   - If text is selected when clicking "Kommentieren", the selection is auto-quoted
 *     into the comment (> "Breslau"). The quote is plain text in the message body,
 *     NOT a structural char-offset anchor. It doesn't break when text changes.
 *   - Threads are anchored to block_id only (no char offsets).
 *   - Yellow comment annotations are DISABLED in transcribe mode.
 *     Only turquoise transcription rects on the PDF. One annotation type per mode.
 *
 * History: "Verlauf" button in transcript toolbar toggles a collapsible panel
 *   showing recent changes with word-level diffs per block.
 * Auto-save: debounced PATCH to /api/transcription-blocks/{blockId} (500ms).
 * Bottom panel: removed entirely (all modes). Metadata → topbar drawer. */
ElementValueNotes
Annotation reuse
Draw gestureExisting AnnotationLayer.onDraw(rect)Same pointer events. crosshair cursor.
Annotation colorturquoise (#00C7B1) for transcriptionYellow annotations disabled in transcribe mode
Annotation typeNew column: type VARCHAR "transcription"|"comment"Default "comment" for backward compat
Number badge16px navy circle, top-left of rectSort order number, matches block number
Transcript blocks (right panel)
Block cardborder:1px line, radius:5px, active: turquoise glowHeader: number + label + presence. Body: contenteditable.
Block labelEditable text, defaults: Anrede, Hauptteil, SchlussDouble-click to rename
Empty stateDashed border, "noch leer" italic textFocus to start typing
Add block CTADashed card: "Markiere eine Passage im Scan..."Not clickable — directs user to draw on PDF
Block-level comment threads
Trigger"Kommentieren" button in block footerAlways visible — no hover-reveal
Quoted selectionIf text selected → auto-quoted into comment bodyPlain text quote (> "Breslau"), NOT char-offset anchor
Quote displayLeft border + italic, above the comment textDecorative only — doesn't link to text range
Thread UIorange left-border, orange-tint bg, below block bodyBlock-level anchor (block_id). Reuses CommentThread.
Footer hint"Text markieren für Zitat" in 5px muted textOnly shown when block is active/focused
Resolve"✓ Lösen" button collapses threadResolved threads hidden by default, toggle to show
MobileThreads collapsed to "2 Kommentare" row, tap to expandSaves vertical space on small screens
Yellow annotations in transcribe mode
StatusDisabled — draw gesture only creates turquoise rectsExisting yellow annotations still visible (read-only)
Annotate modeStill available via topbar "Annotieren" buttonExits transcribe mode, enters annotate mode (yellow)
History (transcript toolbar)
Toggle"🕗 Verlauf" button in transcript toolbarActive state: navy bg, white text
PanelCollapsible, between toolbar and block listbg:color-page, border:line, radius:5px
EntriesTime + user + block ref + word-level diffReuses diffWords from 'diff' library
"Alle anzeigen"Link to full history view (reuses PanelHistory)Opens in a modal or replaces block list temporarily
Interaction
Click rect → blockscrollIntoView + active state on blockTurquoise glow on both rect and block
Click block → rectPDF scrolls/zooms to show the annotationIf multi-page: switches page
Delete blockDeletes annotation + block + threadsConfirm dialog if threads exist
Reorder blocksDrag handle in block headerUpdates sort_order via PATCH
Presence (collaborative)
Dots in topbarColored dot + user name, flex rowMax 3 shown, "+N" overflow
Block-level presenceColored dot + name in block headerLeft border color matches user
ImplementationWebSocket presence via Y.js (future)MVP: polling-based, 5s interval
Auto-save
Debounce500ms after last keystrokePATCH /api/transcription-blocks/{blockId}
Status"✓ Gespeichert" in toolbar, fades after 3s"Speichern..." while request in-flight
ConflictLast-write-wins for MVPY.js CRDT for future collaborative editing

Implementation Guide — Annotation-Backed Transcription

1. Data Model Changes

Flyway migration: document_annotations

New table: transcription_blocks

ColumnTypeNotes
idUUID PKGenerated
annotation_idUUID FK → document_annotationsLinks block to its PDF rectangle
document_idUUID FK → documentsDenormalized for efficient queries
textTEXTThe transcription content
labelVARCHAR(100)"Anrede", "Hauptteil", etc.
sort_orderINTDisplay order in the editor
created_byUUID FK → app_users
updated_byUUID FK → app_users
created_atTIMESTAMP@CreationTimestamp
updated_atTIMESTAMP@UpdateTimestamp

Block-level comments: document_comments

Backward compatibility: Document.transcription

The existing transcription TEXT field becomes a computed read-only view: SELECT string_agg(text, E'\n\n' ORDER BY sort_order) FROM transcription_blocks WHERE document_id = ?. Write operations go through the block API. This keeps search indexing, export, and the read-only PanelTranscription working without changes.

2. Annotation Color Convention & Mode Exclusivity

TypeColorHexOn clickWhen active
CommentYellow#FFC800Opens AnnotationSidePanel (existing)Annotate mode only
TranscriptionTurquoise#00C7B1Highlights matching block in transcript editorTranscribe mode only

Mode exclusivity: In transcribe mode, only turquoise rects can be drawn. Existing yellow comment annotations from annotate mode are still visible on the PDF (read-only, dimmed) but cannot be created or interacted with. The “Annotieren” button exits transcribe mode and enters annotate mode (and vice versa). This prevents overlapping annotation types and avoids user confusion about which comment system to use.

3. Component Architecture

ComponentChange
AnnotationLayer.sveltePass type to onDraw callback. Render turquoise vs yellow based on annotation type. Add number badges for transcription annotations.
PdfViewer.svelteSplit handleAnnotationDraw into two paths (annotate vs transcribe). Route handleAnnotationClick to either side panel or transcript editor.
AnnotationSidePanel.svelteNo change — still handles comment-type annotations in annotate mode. Hidden in transcribe mode.
TranscriptEditor.svelte (new)Right panel. Renders transcript toolbar + block list. Manages block CRUD, auto-save, block-level comment threads.
TranscriptBlock.svelte (new)Single block card. contenteditable body, header with number/label/presence, footer with “Kommentieren” button, thread slot below body.
BlockCommentThread.svelte (new)Comment thread anchored to a block. Shows quoted selections as blockquotes. Reuses CommentThread internally for replies/mentions.
TranscriptToolbar.svelte (new)Block count, sort button, history toggle, save status.
TranscriptHistory.svelte (new)Collapsible panel. Reuses diffWords from the diff library. Shows recent changes per block.
DocumentBottomPanel.svelteRemoved entirely. Metadata lives in the topbar drawer (see companion spec). Discussion, transcription, and history are all inline.
documents/[id]/+page.svelteAdd transcribeMode state. Conditionally render TranscriptEditor vs bottom panel.

4. API Endpoints

MethodPathNotes
POST/api/documents/{id}/annotationsExisting, but now accepts type field. If type="transcription", also creates a TranscriptionBlock.
GET/api/documents/{id}/transcription-blocksReturns all blocks ordered by sort_order.
PATCH/api/transcription-blocks/{blockId}Update text, label, or sort_order. Auto-save target.
DELETE/api/transcription-blocks/{blockId}Deletes block + its annotation + any anchored comments.
PATCH/api/transcription-blocks/reorderBulk update sort_order for drag-and-drop reordering.

5. Draw-to-Transcribe Workflow

  1. User enters Transcribe mode (topbar button, turquoise). Hint strip appears. Yellow comment annotations become read-only/dimmed. Only turquoise rects can be drawn.
  2. Crosshair cursor on PDF (same as annotate mode). User draws a rectangle around a handwriting passage.
  3. AnnotationLayer.onDraw(rect) fires. PdfViewer calls POST /api/documents/{id}/annotations with type: "transcription".
  4. Backend creates DocumentAnnotation + TranscriptionBlock (empty text, next sort_order).
  5. Frontend receives the created annotation + block. The transcript editor scrolls to the new empty block and focuses it.
  6. User types the transcription. Auto-save debounces to PATCH /api/transcription-blocks/{blockId}.
  7. Repeat: draw next rectangle, type next block.

6. Comment Flow — Block-Level Threads with Quoted Selections

Comments are anchored to blocks, not character offsets. This is a deliberate simplification:

Why not char-offset anchoring?

How it works

  1. User clicks “Kommentieren” in a block footer.
  2. If text is selected in the block body, the selection is auto-quoted into the comment input: > “Breslau”. The user can edit or remove the quote before sending.
  3. If no text is selected, the comment input opens empty — a general block-level comment.
  4. The comment is saved as a DocumentComment with block_id set. The quoted text is part of the content field (markdown blockquote syntax).
  5. The thread renders below the block body with an orange left-border. Quoted text appears as an indented italic blockquote above the comment message.
  6. Replies work the same as existing CommentThread — no changes needed.

What happens when text changes after quoting?

Nothing breaks. The quote is a frozen snapshot of what the user selected. If “Bresla” was later corrected to “Breslau”, the original quote still reads > “Bresla” with Oma Inge’s comment “I think this is Breslau.” The context is preserved. No orphaned anchors, no broken highlights.

Footer hint

When a block is focused/active, the footer shows a subtle hint: “Text markieren für Zitat” (select text for a quote). This teaches the quoted-selection pattern without requiring documentation.

7. History in Transcript Toolbar

8. Accessibility

9. Companion Spec

The expandable metadata header (labeled “Details ▼” toggle) is specified separately in expandable-metadata-header-spec.html. Together, these two specs fully eliminate the bottom panel in all modes: metadata → header drawer, transcription → inline split view, discussion → inline threads, history → transcript toolbar. One consistent pattern — no mode-dependent UI structure.