feat(web): add sprite sheet scrub previews for video progress bar

[mstublefield]

Summary

Adds proper scrub preview thumbnails when hovering over the video progress bar on shared Cap video pages. Previously, the hover tooltip either showed the current playback frame (not the hovered position) or a red "Error" image for custom bucket users.

• Server-side sprite generation: During background processing, FFmpeg generates a sprite sheet (grid of 160x90 frames at 2-second intervals) and a companion WebVTT file with #xywh coordinates
• VTT API endpoint: New fileType=thumbnails-vtt handler in the playlist API serves the VTT with a fresh signed sprite URL injected on each request — works for both default and custom R2/S3 buckets
• Zero vendored file changes: media-chrome v4.12.0 already has built-in support for <track kind="metadata" label="thumbnails"> — just needed a data source. The existing spriteStyle and getThumbnail paths in media-player.tsx are untouched

Graceful fallback

• Before sprites are ready (processing in progress, older videos): tooltip shows time-only
• After processing completes: hover shows the actual frame at the hovered position
• Mobile: tooltip hidden entirely (existing behavior)

Both recording modes supported

Instant and studio recordings converge on the same processing pipeline. Sprite generation is best-effort — failures are caught and logged but don't fail the processing job.

Test plan

• Unit tests: 773/773 pass
• TypeScript: no new errors
• Biome lint/format: clean
• Local smoke test: uploaded video via web recorder, verified sprite thumbnails render and change position on hover
• Edge case: very short video (<4s)
• Edge case: video still processing (tooltip shows time-only)

🤖 Generated with Claude Code

Greptile Summary

This PR adds server-side sprite sheet generation during video processing and wires up a WebVTT-based seek preview in the media player. The FFmpeg sprite generation, VTT placeholder approach, signed-URL injection in the playlist API, and graceful 404 fallback are all cleanly implemented.

Confidence Score: 5/5

Safe to merge — sprite generation is best-effort and failures don't affect core video playback.

All findings are P2: a missing type union entry in the webhook handler (silently handled at runtime by the default case) and an un-awaited webhook call in mux-segments (consistent with pre-existing thumbnail webhook pattern). No data loss, no broken playback path, no security concerns.

apps/web/app/api/webhooks/media-server/progress/route.ts — add "generating_sprites" to the payload type and an explicit mapping case.

Important Files Changed

Filename	Overview
apps/media-server/src/lib/ffmpeg-video.ts	Adds generateSpriteSheet and formatVttTimestamp helpers; sprite generation logic is correct with proper timeout, chunked stdout reading, and a __SPRITE_URL__ placeholder for deferred URL resolution.
apps/media-server/src/routes/video.ts	Threads sprite presigned URLs through both processVideoAsync and muxSegmentsAsync; sprite generation is best-effort with try/catch, but sendWebhook is not awaited in the muxSegmentsAsync sprite block (unlike in processVideoAsync).
apps/web/app/api/webhooks/media-server/progress/route.ts	New "generating_sprites" phase is silently handled by the default case in mapPhaseToDbPhase but is missing from the ProgressWebhookPayload type union, creating a type/runtime mismatch.
apps/web/app/api/playlist/route.ts	New thumbnails-vtt handler fetches the stored VTT, replaces __SPRITE_URL__ with a fresh signed URL, and serves with no-store cache headers — correctly handles both default and custom S3 buckets.
apps/web/app/s/[videoId]/_components/CapVideoPlayer.tsx	Removes the canvas-capture thumbnail approach and adds a static <track kind="metadata" label="thumbnails"> element pointing to the new VTT endpoint; gracefully falls back to time-only display when VTT is unavailable (404).
apps/web/workflows/process-video.ts	Generates presigned PUT URLs for the sprite sheet and VTT, then passes them to the media server process job — symmetric with the recording-complete.ts change.
apps/web/app/api/upload/[...route]/recording-complete.ts	Adds presigned PUT URL generation for sprites/sprite.jpg and sprites/thumbnails.vtt, forwarding them through to the mux-segments payload cleanly.

Sequence Diagram

sequenceDiagram
    participant Client as Browser
    participant Playlist as /api/playlist
    participant S3 as S3 / R2
    participant MediaServer as Media Server
    participant Webhook as /api/webhooks/progress

    MediaServer->>S3: Upload result.mp4
    MediaServer->>MediaServer: generateSpriteSheet(result.mp4)
    MediaServer->>S3: PUT sprites/sprite.jpg (presigned)
    MediaServer->>S3: PUT sprites/thumbnails.vtt (presigned, __SPRITE_URL__ placeholder)
    MediaServer->>Webhook: POST phase=generating_sprites (mapped to processing)
    MediaServer->>Webhook: POST phase=complete

    Client->>Playlist: GET /api/playlist?fileType=thumbnails-vtt
    Playlist->>S3: getObject(sprites/thumbnails.vtt)
    S3-->>Playlist: VTT with __SPRITE_URL__
    Playlist->>S3: getSignedObjectUrl(sprites/sprite.jpg)
    S3-->>Playlist: signed URL (fresh)
    Playlist-->>Client: VTT with real signed URL (no-store)

    Client->>S3: GET sprites/sprite.jpg (signed URL, on hover)
    S3-->>Client: sprite image

Loading

Comments Outside Diff (1)

1. apps/web/app/api/webhooks/media-server/progress/route.ts, line 14-64 (link)

   New phase not included in webhook type or mapping

   "generating_sprites" is sent by the media server but is absent from ProgressWebhookPayload["phase"] and has no explicit case in mapPhaseToDbPhase. At runtime the default branch silently maps it to "processing", which is correct, but the TypeScript type is misleading and any future addition of cases before default could miss this phase.

   And in mapPhaseToDbPhase, add an explicit case:

   case "generating_sprites":
       return "processing";

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: apps/web/app/api/webhooks/media-server/progress/route.ts
   Line: 14-64
   
   Comment:
   **New phase not included in webhook type or mapping**
   
   `"generating_sprites"` is sent by the media server but is absent from `ProgressWebhookPayload["phase"]` and has no explicit case in `mapPhaseToDbPhase`. At runtime the `default` branch silently maps it to `"processing"`, which is correct, but the TypeScript type is misleading and any future addition of cases before `default` could miss this phase.
   
   
   
   And in `mapPhaseToDbPhase`, add an explicit case:
   ```ts
   case "generating_sprites":
       return "processing";
   ```
   
   How can I resolve this? If you propose a fix, please make it concise.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: apps/web/app/api/webhooks/media-server/progress/route.ts
Line: 14-64

Comment:
**New phase not included in webhook type or mapping**

`"generating_sprites"` is sent by the media server but is absent from `ProgressWebhookPayload["phase"]` and has no explicit case in `mapPhaseToDbPhase`. At runtime the `default` branch silently maps it to `"processing"`, which is correct, but the TypeScript type is misleading and any future addition of cases before `default` could miss this phase.

```suggestion
	phase:
		| "queued"
		| "downloading"
		| "probing"
		| "processing"
		| "uploading"
		| "generating_thumbnail"
		| "generating_sprites"
		| "complete"
		| "error"
		| "cancelled";
```

And in `mapPhaseToDbPhase`, add an explicit case:
```ts
case "generating_sprites":
    return "processing";
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: apps/media-server/src/routes/video.ts
Line: 1394

Comment:
**`sendWebhook` not awaited (inconsistent with `processVideoAsync`)**

The sprite-generation webhook call in `muxSegmentsAsync` is fire-and-forget, while the equivalent call in `processVideoAsync` (line 781) uses `await`. If `sendWebhook` rejects, the rejection is silently dropped here. The thumbnail block in this same function (line 1373) has the same pattern, but the divergence from `processVideoAsync` should be intentional and consistent.

```suggestion
				await sendWebhook(getJob(jobId)!);
```

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "feat(web): add sprite sheet scrub previe..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

_{(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!}

[greptile-apps]

sendWebhook not awaited (inconsistent with processVideoAsync)

The sprite-generation webhook call in muxSegmentsAsync is fire-and-forget, while the equivalent call in processVideoAsync (line 781) uses await. If sendWebhook rejects, the rejection is silently dropped here. The thumbnail block in this same function (line 1373) has the same pattern, but the divergence from processVideoAsync should be intentional and consistent.

Suggested change

	sendWebhook(getJob(jobId)!);
	await sendWebhook(getJob(jobId)!);

Prompt To Fix With AI

This is a comment left during a code review.
Path: apps/media-server/src/routes/video.ts
Line: 1394

Comment:
**`sendWebhook` not awaited (inconsistent with `processVideoAsync`)**

The sprite-generation webhook call in `muxSegmentsAsync` is fire-and-forget, while the equivalent call in `processVideoAsync` (line 781) uses `await`. If `sendWebhook` rejects, the rejection is silently dropped here. The thumbnail block in this same function (line 1373) has the same pattern, but the divergence from `processVideoAsync` should be intentional and consistent.

```suggestion
				await sendWebhook(getJob(jobId)!);
```

How can I resolve this? If you propose a fix, please make it concise.