feat(moq-json): optional group-scoped zstd compression

[kixelated]

Moves payload compression up to the application layer (moq-json) instead of the moq-lite wire (the alternative explored in #1874 / #1889). The relay stays media-agnostic, a group reuses a single warm compression context across its frames, and we get an optional shared dictionary for free. This is the reusable primitive; the catalog dual-publish and the browser decode path are deliberate follow-ups (see Scope).

What changed

moq-json gains an optional Config.compression: Option<Compression> (zstd):

• One zstd stream per group, flushed at each frame. A group's snapshot + deltas form a single stream, so later frames reuse the earlier ones as context (a snapshot then small deltas compresses far better than each frame alone). The Encoder/Decoder hold the per-group state and reset at every group boundary.
• Magicless frames, no per-frame checksum. moq-net's framing already delimits each slice, so the zstd magic number and checksum would only be redundant bytes (uses zstd's experimental feature). This is the "remove the magic marker between frames" ask.
• Optional dictionary. A shared dictionary primes the window so even a group's first frame compresses against known content. Both ends must be configured with the same dictionary (how a consumer obtains it is out of band).
• Delta budget measured on compressed bytes. When compression is on, the delta-vs-snapshot rolling budget compares the real (compressed) slice sizes against the group's anchoring snapshot, not the raw JSON. A warm window shrinks each successive delta, so more updates pack into a group. The uncompressed path keeps its existing raw-byte behavior.
• Zip-bomb guard. zstd has no built-in total-output limit for streaming magicless frames, so the decoder enforces a cumulative per-group decompressed cap (64 MiB) and stops with an error, plus windowLogMax bounds per-frame window memory.

Compression defaults off, so existing tracks (including the hang catalog.json, which uses delta_ratio: 0 and no compression) are byte-identical on the wire. No moq-net wire or catalog-format change.

Public API (moq-json)

• Config gains compression: Option<Compression> and becomes #[non_exhaustive] (build via Config::default() + field set). The one strictly-breaking bit; moq-json is 0.0.x.
• New Compression { level, dictionary } config (#[non_exhaustive], Default).
• New Consumer::with_compression(track, Compression); Consumer::new stays the plaintext shortcut. A cloned consumer rebuilds its non-cloneable decoder window by replaying the group's already-read slices (precedent: the GroupConsumer handling in feat(moq-net): rework payload compression, kept compressed in RAM #1889).
• Error gains Decompress and TooLarge(u64).

rs/moq-mux (the only in-repo caller) updated to build Config via default() + field set.

Branch targeting

Targets main: no wire-protocol or catalog-format change (compression is off by default and the catalog is untouched), and moq-json is not among the crates the breaking-change rule routes to dev. The sibling moq-lite PRs target dev because they change the wire; this one does not.

Scope — deferred follow-ups

• catalog.json + catalog.json.z dual-publish and explicit catalog format selection (so a player can skip a deprecated uncompressed track). Touches rs/moq-mux catalog Producer, rs/hang, js/hang, doc/concept; kept separate so this core stays small.
• js/json mirror: the browser only needs to decode a compressed catalog, which is exactly the catalog follow-up; landing the JS zstd dependency with its first real consumer rather than speculatively. (moq-json is not on the wire-sync table and the catalog wire bytes are unchanged here.)
• Dictionary distribution: the API accepts a dictionary; how a consumer obtains the shared one is a separate design.
• The moq-lite PRs (feat(moq-net): per-hop, per-frame payload compression (drafts#39) #1874, feat(moq-net): rework payload compression, kept compressed in RAM #1889) stay open as the wire-layer alternative to weigh against this.

Test plan

• cargo test -p moq-json — 34 tests incl. new compressed snapshot/delta round-trips, compressed late-joiner, mid-group cloned-consumer rebuild, dictionary round-trip, and a "compressed frame is smaller than plaintext" wire-size check; plus codec-level round-trip / cross-frame-redundancy / dictionary / garbage-rejection unit tests.
• cargo clippy -p moq-json -p moq-mux --all-targets — clean.
• cargo build -p moq-mux -p hang, cargo doc -p moq-json — clean (catalog producer compiles against the new Config).
• just check via the pinned nix toolchain — not run here; please confirm in a real terminal before merge.

🤖 Generated with Claude Code

(Written by Claude)

---

Generated by Claude Code

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 3c913d92-ec80-4014-9cb6-e72e6a3a410e

📥 Commits

Reviewing files that changed from the base of the PR and between 8469563 and 07ae7ac.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (5)

• Cargo.toml
• rs/moq-json/Cargo.toml
• rs/moq-json/src/compression.rs
• rs/moq-json/src/lib.rs
• rs/moq-mux/src/catalog/producer.rs

---

Walkthrough

The pull request adds optional per-group zstd compression to the moq-json crate. A new Compression struct holds a configurable level and an optional shared dictionary; it produces per-group Encoder and Decoder instances using magicless raw zstd frames with flush-retain semantics so each frame is self-delimited while reusing cross-frame window context. Config gains a compression: Option<Compression> field and Error gains Decompress and TooLarge(u64) variants. The producer compresses snapshots and delta patches through a per-group encoder and budgets delta decisions using compressed sizes. The consumer lazily creates a decoder per group, replays accumulated slices to reconstruct window state (including after cloning), then decompresses each incoming frame. The moq-mux catalog producer is updated to use Config::default() instead of a struct literal.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the main change: adding optional group-scoped zstd compression to moq-json, which aligns with the primary focus across all modified files.
Description check	✅ Passed	The description comprehensively explains the changeset, covering the motivation, technical implementation, API changes, test plan, and deferred follow-ups. It directly relates to all aspects of the changes.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

✨ Simplify code

• Create PR with simplified code
• Commit simplified code in branch claude/app-layer-compression-ef19lq

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}