Context: we ran what we believe is the first independent benchmark of Ternary Bonsai 27B on consumer hardware (RTX 5060 Ti 16 GB, fork commit b9591, CUDA build for sm_120, Linux). Overall results were good for the fork: ternary +24-31% decode vs a best-engine IQ2_XXS baseline, 262k context fits a 16 GB card with q4_0 KV, and the DSpark lossless-precision claim verified byte-identical Q4_1 vs bf16 (5/5 prompts at temp 0). Raw data and scripts: https://github.com/Astezelex/bonsai-27b-16gb-bench (also PrismML-Eng/Bonsai-demo#98). This report is friction we hit on the way.
Same GGUF (Qwen3.6-27B UD-IQ2_XXS), same card (5060 Ti), same flags (-ngl 999, llama-bench tg128, 10 reps, first dropped): fork 34.23 t/s vs mainline 35.78 t/s (+4.5% for mainline), pp512 within noise. Mainline commit for the comparison: 12127de.
Why it matters: A/B comparisons "ternary vs X" run entirely inside the fork will overstate ternary's relative advantage unless the baseline runs on mainline. A note in the README benchmarking guidance would prevent that (we report fork-vs-mainline as "best engine each": +24%; the same-engine number would read +30%).
Context: we ran what we believe is the first independent benchmark of Ternary Bonsai 27B on consumer hardware (RTX 5060 Ti 16 GB, fork commit b9591, CUDA build for sm_120, Linux). Overall results were good for the fork: ternary +24-31% decode vs a best-engine IQ2_XXS baseline, 262k context fits a 16 GB card with q4_0 KV, and the DSpark lossless-precision claim verified byte-identical Q4_1 vs bf16 (5/5 prompts at temp 0). Raw data and scripts: https://github.com/Astezelex/bonsai-27b-16gb-bench (also PrismML-Eng/Bonsai-demo#98). This report is friction we hit on the way.
Same GGUF (Qwen3.6-27B UD-IQ2_XXS), same card (5060 Ti), same flags (
-ngl 999, llama-bench tg128, 10 reps, first dropped): fork 34.23 t/s vs mainline 35.78 t/s (+4.5% for mainline), pp512 within noise. Mainline commit for the comparison:12127de.Why it matters: A/B comparisons "ternary vs X" run entirely inside the fork will overstate ternary's relative advantage unless the baseline runs on mainline. A note in the README benchmarking guidance would prevent that (we report fork-vs-mainline as "best engine each": +24%; the same-engine number would read +30%).