K3 GPU beta: f_θ+S5 K/V-Restoration verifier — incremental decode (=AR) + DFlash fused spec-decode (>AR) on Gemma 4 26B-A4B#107

Merged

FluffyAIcode merged 84 commits into

mainfrom

AgentMemory/v04-k3-gap12-restored-spec-decode-server-2815

Jun 11, 2026

Commits on Jun 9, 2026

K3 Block B + C: f_theta projection + cross-model DLMRestoredVerifier (P0)

cursoragent
and
FluffyAIcode
committed

Commits on Jun 10, 2026

K3 P0 critical fixes + vast reviewer aids + integrated NIAH eval

cursoragent
and
FluffyAIcode
committed
K3: support Gemma4 multimodal nested config/decoder in f_theta train + cross-model verifier

cursoragent
and
FluffyAIcode
committed
K3: capture V from k_proj output for Gemma4 v_proj-None (KV-sharing) layers

cursoragent
and
FluffyAIcode
committed
K3: heterogeneous per-layer verifier KV heads in f_theta + per-layer capture/loss for Gemma4

cursoragent
and
FluffyAIcode
committed
K3: Gemma4-faithful cross-model restore forward (per-layer KV, v_norm, RoPE unsqueeze_dim=2, v_proj-None, evicted slicing) + gemma4 helpers import + tests

cursoragent
and
FluffyAIcode
committed
K3: cast f_theta input to encoder weight dtype (fp32 f_theta vs bf16 drafter K/V)

cursoragent
and
FluffyAIcode
committed
K3: fix integrated NIAH eval to use real niah_eval API (chat-template encode, aggregate_recall, v04_dlm_restored window)

cursoragent
and
FluffyAIcode
committed
K3: handle BatchEncoding return from Gemma4 apply_chat_template in integrated NIAH eval

cursoragent
and
FluffyAIcode
committed
K3: per-layer verifier head_dim in f_theta (Gemma4 full layers use global_head_dim=512, 2 KV heads)

cursoragent
and
FluffyAIcode
committed
K3: add identity-restore diagnostic (inject verifier's own K/V) to isolate restore machinery from f_theta accuracy

cursoragent
and
FluffyAIcode
committed
K3 f_theta v1 trained checkpoint (Gemma4 26B-A4B verifier, per-layer KV; loss 50.8->3.70, 13.74x)

cursoragent
and
FluffyAIcode
committed
K3 integrated NIAH gate evidence: arch_correct=1.0 PASS, recall gate FAIL (f_theta v1), identity-restore recall=1.0 (machinery validated)

cursoragent
and
FluffyAIcode
committed
K3 f_θ trainer v2 — fix recall=0 (cosine+mag loss + NIAH data + cosine LR + 5× longer)

cursoragent
and
FluffyAIcode
committed
K3 f_θ trainer v3 — one-shot attention-output distillation (skip v2 intermediate)

cursoragent
and
FluffyAIcode
committed
K3 S6: --mix-alpha-sweep fidelity->recall diagnostic (interpolate evicted K/V between f_theta and true; map recall vs residual rel_mse)

cursoragent
and
FluffyAIcode
committed
K3 attn_distill v3 evidence: train reduction 21.47x (attn-output rel-err 1.0->~0.20), but integrated NIAH recall still 0/10 both rungs (arch gate PASS)

cursoragent
and
FluffyAIcode
committed
K3 S6 alpha-sweep on attn_distill v3: recall 0 for all alpha<1.0 (degenerate — attn_distill K/V are ~135x off-scale; k_norm/v_norm normalize scale away, so raw-space mix is confounded)

cursoragent
and
FluffyAIcode
committed
K3 S6 alpha-sweep on scale-matched relmse v3: recall knee in (0,0.5]; full-attn rel_mse 0.36 -> recall 1.0, 1.44 -> 0; eval-domain err (1.44) >> in-domain (0.58) = distribution shift

cursoragent
and
FluffyAIcode
committed
K3 f_θ trainer v4: attn_distill_hybrid loss — fix the f_θ collapse exposed by alpha-sweep

cursoragent
and
FluffyAIcode
committed
K3 S6 knee refinement (relmse v3): recall transition alpha 0.3->0.4->0.5 = full-attn rel_mse 0.71(0/10)->0.52(6/10)->0.36(10/10)

cursoragent
and
FluffyAIcode
committed
K3 trainer aid: forward NIAH_MIN_LINES/NIAH_MAX_LINES env to --niah-{min,max}-lines (was ignored)

cursoragent
and
FluffyAIcode
committed
K3 fix: import apply_rotary_pos_emb for attn_distill_hybrid too (was only attn_distill -> hybrid crashed)

cursoragent
and
FluffyAIcode
committed
K3 v4a warm-start hybrid checkpoint (rank256, init relmse v3, attn_distill_hybrid, gen1024, niah140, 10k): reduction 3.42x, attn-output ratio ~0.24

cursoragent
and
FluffyAIcode
committed
K3 v4b fresh hybrid checkpoint (rank768, 128 NIAH, gen1024, niah140, 20k): reduction 8.01x, attn-output ratio ~0.21

cursoragent
and
FluffyAIcode
committed
K3 v4a/v4b hybrid integrated NIAH evidence: both recall 0/10 both rungs (arch PASS) despite scale-matched hybrid + NIAH data + bigger/longer/warm-start

cursoragent
and
FluffyAIcode
committed
K3 fidelity probe v4a/v4b: eval full-attn rel_mse 1.42/1.52 (== relmse v3's 1.44) — full-attn K/V fidelity floor independent of loss/rank/data; blend to 0.36 -> recall 1.0 (threshold confirmed)

cursoragent
and
FluffyAIcode
committed
K3 v4a/v4b canonical NIAH + alpha-sweep artifacts: NIAH 0/10 both; sweep recall flips 0->1 between alpha 0.25 (full-attn ~0.8) and 0.5 (~0.37), identical for both

cursoragent
and
FluffyAIcode
committed

Commits on Jun 11, 2026