Kakeya Inference Engine for Mac — MLX speculative-decode beta (consolidated → main)

[FluffyAIcode]

What

Consolidated Mac MLX beta for merge to main + tagging. Brings the full all-MLX speculative-decode stack onto main, purely additive on ce911bb (the CUDA #107 beta). 39 files, +10.7k, recall 1.0 throughout.

Ready for you to merge + tag. I cannot merge or push tags from the agent (read-only gh, no merge action). After merging, tag the merge commit, e.g.:

git tag -a kakeya-inference-engine-for-mac-beta <merge-sha> -m "Kakeya Inference Engine for Mac — MLX spec-decode beta"
git push origin kakeya-inference-engine-for-mac-beta

What's included (consolidated)

• All-MLX DFlash drafter + fused spec-decode engine (b876 PR Step-2 rescue: all-MLX DFlash drafter — parity-proven, 17× over the hybrid fused path (0.476× AR) #112): native MLX drafter (zero per-block MLX↔PyTorch bridge), fused_specdecode_generate_mlx, the v3 rollback, parity/kv-quant evals.
• Gap-A incremental restored decode + MLXRestoredIncrementalVerifier (my MLX port of #107: incremental decode (Step 1) + fused DFlash spec-decode engine (Step 2) #109): prefill→native cache→generate_step.
• CUDA-DynamicCache-parity rollback (my CUDA-parity rollback for the all-MLX fused loop (keep accepted K/V, trim only rejected) — +33% on code, ~AR parity #115): make_full_kv_prompt_cache + fused_specdecode_generate_mlx_trim (keep accepted K/V, trim only rejected) — +33% over v3, ~AR parity on block-8 long-code.
• Mac bridge (git-bus + self-hosted kakeya-mac-m4 runner) + evidence gate (k3_report_gate.py) + presets (k3-fused-allmlx-code[-trim], -natural, -singlefused-probe, …).
• README baseline: the 0.09× → ~1.0× AR journey (companion to PR README: Kakeya Inference Engine for Mac — MLX spec-decode port journey (K3 beta baseline) #116, included here so the beta is self-contained).

Deliberately EXCLUDED

• native_restored_cache.py (MLX native restored-cache primitive — systemic fix for the Mac throughput collapse #110, the native bypass) — per the 2026-06-13 directive it's a product bypass that doesn't exercise the architecture; left out of the beta. (Its PR MLX native restored-cache primitive — systemic fix for the Mac throughput collapse #110 stays open, labelled forbidden-for-validation.)

Evidence (real Mac, all-MLX fused, recall 1.0)

• CUDA-trim block-4 0.68× AR (+33% vs v3 0.51×); block-8 ~1.0–1.05× (AR parity, best long-code); output bit-identical to v3.
• Acceptance/throughput investigation closed: the binding constraint is the 26B verify(L) compute per block — not rollback, sync count, acceptance, quantization, or context length. >AR remains CUDA-favoured (H200 1.27×). Remaining lever: the 4.5→7.7 drafter accept-len gap.
• Bounded S5 KV (~133 MB vs ~1309 MB naïve @ 5.8k ctx; ~48 MB affine-4).

Testing

• ✅ Linux: pytest tests/inference_engine/bridge/ tests/inference_engine/bench/ → 92 passed; MLX backend tests pass except 4 pre-existing b876 test_fused_specdecode.py fixture failures (present on PR Step-2 rescue: all-MLX DFlash drafter — parity-proven, 17× over the hybrid fused path (0.476× AR) #112's base, stash-verified — not introduced here; flagged for a fixture refresh).
• ✅ Real Mac (bridge, from this branch): mlx-env-probe success (branch is bridge-runnable); the engine paths validated in PRs MLX port of #107: incremental decode (Step 1) + fused DFlash spec-decode engine (Step 2) #109/MLX native restored-cache primitive — systemic fix for the Mac throughput collapse #110/CUDA-parity rollback for the all-MLX fused loop (keep accepted K/V, trim only rejected) — +33% on code, ~AR parity #115.

Notes for the merge owner

• Supersedes/absorbs the README PR README: Kakeya Inference Engine for Mac — MLX spec-decode port journey (K3 beta baseline) #116 (included here) — close README: Kakeya Inference Engine for Mac — MLX spec-decode port journey (K3 beta baseline) #116 or merge this instead.
• The b876 chain (Step-2 rescue: all-MLX DFlash drafter — parity-proven, 17× over the hybrid fused path (0.476× AR) #112/Mac bridge M1: cloud-agent access to the self-hosted kakeya-mac-m4 (git-bus) + distributed-inference integration evaluation #111/v0.5-M1 milestone: agent capability exchange + distributed spec decode on multi-host fleets (ADR 0009) #105) and my MLX port of #107: incremental decode (Step 1) + fused DFlash spec-decode engine (Step 2) #109/CUDA-parity rollback for the all-MLX fused loop (keep accepted K/V, trim only rejected) — +33% on code, ~AR parity #115 are the sources; this is their integration onto main.
• Pre-existing follow-up: the all-MLX fused is 6/8 byte-exact vs greedy AR (bf16 drift) — worth fp32 verify accumulation before promoting beyond beta.