Bonsai 1-bit MLX editions: benchmark tour
Benchmark at a glance
M4 Mac mini 路 16 GB memory 路 short prompt 路 model already loaded.
| Version | Output tokens/sec | Speed vs original | First token | Task checks |
|---|---|---|---|---|
| Original 1-bit Bonsai, stock-style settings | 19.7 | 1.00脳 | 1.62 s | Not separately checked |
| Optimization | 19.8 | 1.00脳; no meaningful gain | 1.60 s | 15/18 |
| Fast | 22.0 | 1.12脳 | 1.36 s | 11/18 |
| UltraFastUnStable, revised | 24.6 | 1.25脳 | 1.35 s | 11/18, earlier same-candidate check |
Read this before comparing: these are three-run medians for the same 81-token input and 128 output tokens, with thinking off and a tiny warm-up. Loading time is excluded. The revised UltraFastUnStable was measured later with a different memory ceiling, so its ratio is a comparison of recorded results鈥攏ot a fresh same-session, controlled speedup. Longer inputs and tool schemas take longer. A higher rate does not mean better or correct answers; task counts are not percentages of intelligence retained.
The retired 48-block experiment reached 79.5 tokens/sec / 0.427 s TTFT, but produced nonsense or blank lines. It is unusable, is not the current UltraFastUnStable download, and is kept only in its historical revision tag. Measurement details and limitations.
Exact recorded numbers
| Version | Decode median (tokens/sec) | TTFT median (s) | Peak MLX memory (GB) | Original full blocks retained |
|---|---|---|---|---|
| Original, stock-style settings | 19.672 | 1.615 | Not shown here | 64 |
| Optimization | 19.764 | 1.599 | 4.709 | 64 |
| Fast | 22.029 | 1.364 | 4.061 | 54 |
| Revised UltraFastUnStable | 24.557 | 1.353 | 4.063 | 54, plus small affine replacements |
| Archived unusable 48-block experiment | 79.532 | 0.427 | 1.568 | 16 |
Speed ratios divide recorded decode medians by the original 19.672 tokens/sec: Optimization 1.0047脳, Fast 1.1198脳, revised UltraFastUnStable 1.2483脳. Optimization's tiny difference is not a meaningful demonstrated speed gain. The archived experiment's approximately 4.04脳 rate is nonsense generation, not usable work, and must not be advertised as a usable assistant speedup.
Test conditions and evidence
All headline runs used an M4 Mac mini with 16 GB unified memory, an 81-token summarization prompt, 128 generated tokens, greedy decoding, thinking disabled, and a two-token warm-up. Loading was excluded from TTFT. Each run was a separate process. This measures output tokens鈥攏ot hidden reasoning, end-to-end tool execution, or the time to a correct completed answer.
The original used stock-style mlx-lm generation options on the same original 1-bit weights. It is not a higher-precision, 2-bit, GGUF, or cloud baseline. Optimization and Fast used 512-token prefill chunks, 4-bit KV conversion after 256 tokens, a 300 MB allocator cache, and a 9.5 GB wired ceiling. The revised UltraFastUnStable used the same generation tuning but a 6.5 GB wired ceiling in a later session. Model outputs differ. The 81-token case does not establish whether KV conversion improves speed on longer inputs.
Optimization decode runs: 20.658, 19.764, 19.205; TTFT: 1.578, 1.599, 1.611. Fast decode runs: 22.029, 23.569, 20.481; TTFT: 1.368, 1.349, 1.364. Revised UltraFastUnStable decode runs: 24.557, 24.857, 24.400; TTFT: 1.347, 1.361, 1.353. Its summary was coherent but made unsupported assumptions and hit the output cap. Short math returned 391 for 17 脳 23.
The fresh revised-edition quality sequence stopped after swap grew; there was no fresh complete 18-task evaluation of that packaged file. Its 11/18 figure comes from the same earlier in-memory pruning/correction candidate. Fresh summary and arithmetic outputs matched that candidate exactly. This is regression evidence, not a replacement for a complete reevaluation.
Optimization's 15/18 and Fast's 11/18 are earlier checks with strict requested content and formatting. Counts are not intelligence-retention percentages. Tool calls were inspected as text; no real searches or image generation were executed. The original stock-style settings were not separately quality-tested. Both pruned editions have serious weaknesses; Optimization is the quality-first choice within this three-edition set, not a guarantee of perfect answers.
Detailed evidence:
- Optimization and Fast measurements and task rubric.
- Revised UltraFastUnStable measurements, failures, and safety stop.
Installation tour
- Pick the edition: Optimization for original weights; Fast for modest pruning; UltraFastUnStable for corrected pruning with a little more recorded speed.
- Download the entire repository from Files and versions or use the model card's hf download command. Weights alone omit the runtime/setup and, for pruned editions, required custom architecture code.
- Install uv and Apple's Metal toolchain. Run the included installer to build Prism's pinned 1-bit MLX fork. Building takes extra disk space and time; temporary build files are removed afterward. No compiled runtime is shipped.
- Run a short capped prompt before interactive use. Model loading contributes additional startup latency. Your app must supply, validate, and execute tools.
All editions need an Apple Silicon Mac. They are not universally compatible with every app's Download MLX button. Pruning/corrections are embedded in the pruned checkpoints; supporting code and runtime settings are not all encoded in a single model.safetensors file.