K3 GPU beta: f_θ+S5 K/V-Restoration verifier — incremental decode (=AR) + DFlash fused spec-decode (>AR) on Gemma 4 26B-A4B

[FluffyAIcode]

Merge / supersession

This is the self-contained K3 GPU beta, retargeted to main (merge-base == main tip → zero conflicts). It contains the full K3 inference path (f_θ + cross-model CUDA+MLX restored verifier + S5), Gap 1/2, the incremental decode engine, the DFlash fidelity fix, and the fused spec-decode engine — all H200-validated, recall 1.0.

Supersedes #103 (earlier diverged design+skeleton + a large agent-session-context/ dump) and #106 (diverged, CONFLICTING). #103 and #106 diverged on inference_engine/v04/f_theta.py/cross_model_dlm_verifier.py, so the originally-proposed #103 → #106 → #107 chain would conflict; this branch is the complete, validated version and merges cleanly to main. Recommend closing #103 and #106 as superseded-by-#107.

---

✅ Gap-A — incremental restored decode = AR parity

Capture restored K/V at prefill → native incremental decode (O(L)/block). ctx 1238: 21.68 vs AR 21.12 (1.03×), KV 16.9× smaller; ctx 3238: 20.98 vs 21.94 (0.96×), KV 43.9×; recall 1.0.

✅ Gap-B — DFlash drafter fidelity

Fixed the spurious ×sqrt(hidden) query-embed scale (reference uses a plain lookup). Acceptance 0.05 → 0.158 (Q&A); HumanEval length 3.87 ≈ official gemma-4-26B 3.3× speedup (τ≈3.9) — reference parity. (0.447/7.7 is a Qwen3 paper number; gemma-4-26B isn't in the paper.)

✅ Fused spec-decode engine (A+B+C) — native, exceeds AR

restored_specdecode_fused + DFlashDrafter.{make,extend}_context_kv/draft_block_cached + aux capture in the incremental verify. Three caches built at prefill, extended incrementally per committed token (all O(L)/block): A = aux hidden from the verify forward (no separate O(C) aux forward/block); B = drafter context K/V cached + extended; C = Gap-A incremental restored verify. Drafter conditions on restored verifier hidden (clean aux for the prompt) — resolves the bounded-KV vs clean-aux tension natively (no SGLang/vLLM).

Stabilized by loading the verifier without device_map (no accelerate hooks; fits on H200), raw embed/lm_head weight ops for the drafter hot path, and a full-length warmup (pre-sizes the caching allocator).

H200 (ctx 1238, 64 tok, warmup):

path	tok/s	vs AR
AR (stable)	21.14	1.0×
Gap-A pertoken	21.09	1.00×
spec-decode un-fused	10.6	0.51×
spec-decode FUSED (aggregate)	26.75	1.27×
spec-decode FUSED (steady samples)	21.5–23.0	1.05–1.10×

Recall 1.0 throughout. Per-block: drafter ~0.11s, verify ~0.10s, ctx_kv_extend ~0.02s (all O(L)); accept_len ~4.3.

Net result

Decode tok/s ≥ AR: achieved. Gap-A alone = AR with 16.9–43.9× KV saving + recall 1.0; the fused DFlash spec-decode exceeds AR (1.27× aggregate) on the bounded-KV restored verifier, recall 1.0.

Gap 1 / Gap 2 / Server / Tests

• CrossModelRestoredSinkWindowVerifier (full SinkWindowVerifier contract; incremental Gap-A + aux capture).
• build_restored_speculative_decoder + load_restored_verifier + gRPC --backend restored.
• 100% coverage on the Gap1/Gap2 modules; 63 v04 tests pass incl. draft_block_cached == draft_block + incremental-ctx-kv equivalence.

Evidence: results/research/k3_e2e_gpu_bench_incremental.json, k3_specdecode_fused_stable.json, k3_dflash_accept_{baseline,noscale,code,humaneval}.json.