On September 17, 2026, AI startup PrismML released Bonsai 2 27B, a version of the open Qwen 3.8 27B language model compressed to run in roughly 5.9 GB of memory. The full-precision original needs about 54 GB. The company says the compressed model keeps 98.2% of the original’s benchmark performance while running on consumer GPUs, laptops, phones, and even inside a web browser.
The claim matters because the parent model is no toy. Qwen 3.8 27B trades benchmark blows with far larger proprietary systems, and the gap between “runs on a phone” and “rivals Opus-class models” has effectively closed. Here is what Bonsai 2 27B actually is, what the numbers show, where it falls short, and why this release may say more about the future of AI hardware than about any single model.
What Is Bonsai 2 27B?
Bonsai 2 27B is a ternary-weight language and vision model released under an Apache 2.0 license, with weights free to download on the official Hugging Face model card. It is not a new model trained from scratch. PrismML took Qwen 3.8 27B, an open-weight model that we have shown rivals GPT-5.6 and Claude Opus-class systems on coding and computer-use tasks, and converted its weights to a three-value format.
The release is a follow-up to PrismML’s first Bonsai model from July 2026, which we covered in our analysis of 1-bit LLMs bringing 27B-class AI to laptops and phones. The first generation claimed around 95% retention of its parent model. Bonsai 2 claims 98.2% against a newer, stronger parent.
The key specifications, as reported by PrismML:
- 27.36 billion total parameters: a 24.35B language backbone, 2.54B embeddings and output head, and a 0.47B vision tower
- 5.93 GB text weights in the PTQ1_0 format, versus 53.80 GB in FP16, roughly 9x smaller
- 98.2% benchmark retention across PrismML’s evaluation suites, vendor-reported and not yet independently reproduced
- 262,144-token context window, with text, image, and video input
- Apache 2.0 license, with builds for NVIDIA GPUs (CUDA), Apple silicon (MLX), and a web browser demo (WebGPU)
How Ternary Weights Shrink 54 GB Into 5.9 GB
A conventional AI model stores each weight as a 16-bit or 32-bit number, millions of possible values per parameter. A ternary model throws almost all of that away. After training, every weight in the language model is rounded to one of just three values: minus one, zero, or plus one. Information-theoretically, three states need only about 1.585 bits to encode, and PrismML reports an effective 1.72 bits per weight once shared scaling factors and high-precision remainder tensors are counted.
Two details keep the compression from destroying accuracy. First, a high-precision scale factor is stored for every group of 128 weights, so the model can still express large and small values at the group level. Second, a Hadamard rotation (a mathematical reshuffling of weight matrices before rounding) spreads outlier values across the matrix so that naive rounding does not erase critical information. About 26.2 million parameters, just 0.098% of the total, stay in higher precision because they are most sensitive to compression.
PrismML ships two GGUF weight packings, and the choice is not cosmetic:
- PTQ1_0 packs the trits densely at roughly 1.75 bits per weight (5.95 GB). This is the variant to use when memory is the bottleneck, and it performs best on Ada, L4, and similar GPU generations.
- PQ2_0 stores each trit in a 2-bit slot at roughly 2.13 bits per weight (7.21 GB). It is larger on disk but cheaper for the hardware to unpack, so it wins when compute is the bottleneck, and it performs best on Blackwell, Hopper, Ampere, and Apple silicon.
A separate 0.63 GB vision tower file loads only when the model processes images. For Apple users, an MLX build runs natively on Mac, iPhone, and iPad. One practical catch: the custom GGUF formats require PrismML’s fork of the llama.cpp runtime. Incompatible runtimes may reject the files or, worse, misread the weights silently, so users cannot simply point stock tooling at the download.
The Benchmarks: What 98.2% Retention Actually Looks Like
PrismML evaluated Bonsai 2 27B in “thinking mode” using EvalScope and vLLM on NVIDIA H100 hardware. The headline 98.2% figure comes from a 14-benchmark suite across six skill categories, where Bonsai 2 averaged 84.78 against the FP16 parent’s 86.32. The company’s press materials also cite a broader 20-benchmark suite (83.9 versus 85.4), which produces the same 98.2% headline. All results are vendor-reported.
The category-level breakdown from the 14-benchmark suite is more revealing than the average:
| Capability | Qwen 3.8 27B FP16 | Bonsai 2 27B | Retention |
|---|---|---|---|
| Knowledge and reasoning | 85.55 | 79.86 | 93.4% |
| Math (GSM8K, MATH-500, AIME) | 97.06 | 96.57 | 99.5% |
| Coding | 89.07 | 89.42 | 100.4% |
| Instruction following | 81.25 | 82.66 | 101.7% |
| Agentic and tool calling | 76.74 | 74.92 | 97.6% |
| Vision | 71.36 | 66.19 | 92.8% |
| Overall average | 86.32 | 84.78 | 98.2% |
[interactive_chart]{“type”:”bar”,”themeMode”:”light”,”background”:”#f8fafc”,”title”:”Benchmark Retention by Capability”,”subtitle”:”Bonsai 2 27B score as a percentage of Qwen 3.8 27B FP16″,”source”:”PrismML Hugging Face model card”,”sourceUrl”:”https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf”,”unitSuffix”:”%”,”dataLabels”:true,”height”:440,”categories”:[“Knowledge and reasoning”,”Math”,”Coding”,”Instruction following”,”Agentic and tool calling”,”Vision”],”series”:[{“name”:”Retention vs FP16″,”data”:[93.4,99.5,100.4,101.7,97.6,92.8]}],”colors”:[“#0e7490″],”allowDownload”:true}[/interactive_chart]
Three patterns stand out. Math and coding are essentially at parity, and instruction following actually exceeds the full-precision parent. The compression cost is concentrated in knowledge recall and vision, the categories most dependent on precisely stored factual detail. And some individual results go beyond parity: on the AIME 2026 competition math benchmark, Bonsai 2 scored 95.83 against the FP16 model’s 94.58.
The average also hides where the model genuinely struggles. Long-horizon software engineering tasks fall well outside the headline: PrismML’s whitepaper reports 60.8 on SWE-bench Verified against the parent’s 80.6, and 52.8 on Terminal-Bench 2.1 against 69.7. In practice, that means Bonsai 2 handles isolated coding problems nearly as well as the original but loses significant ground on multi-step agentic work that chains dozens of actions. Anyone expecting a free Opus-class coding agent will be disappointed; anyone wanting a fast, private, local assistant for everyday reasoning and code will not.
Ternary Weights vs. 2-Bit Quantization: It Is Not Even Close
The most important comparison in PrismML’s release is not against FP16. It is against conventional quantization, the technique everyone already uses to squeeze large models onto consumer hardware. Quantization takes a full-precision model after training and rounds its weights to fewer bits. Ternary conversion does something related but more radical, and the results diverge sharply at low bit widths.
| Variant of Qwen 3.8 27B | True bits/weight | Footprint | Thinking average | vs FP16 |
|---|---|---|---|---|
| FP16 (reference) | 16.0 | 54 GB | 86.32 | 100% |
| UD-Q4_K_XL (“4-bit”) | 5.2 | 17.6 GB | 85.18 | 98.7% |
| Bonsai 2 27B (ternary) | 1.72 | 5.9 GB | 84.78 | 98.2% |
| IQ2_XXS (“2-bit”) | 2.8 | 9.4 GB | 72.59 | 84.1% |
Read that table twice. The conventional 2-bit build is larger than Bonsai 2 at 9.4 GB, yet it loses almost 16% of the parent model’s capability while the ternary model loses 1.8%. The 4-bit build needs 17.6 GB, three times Bonsai 2’s footprint, to edge it by half a point.
[interactive_chart]{“type”:”column”,”themeMode”:”light”,”background”:”#f8fafc”,”title”:”Average Benchmark Score vs. Model Footprint”,”subtitle”:”Thinking-mode average, 14 benchmarks (PrismML evaluation)”,”source”:”PrismML Hugging Face model card”,”sourceUrl”:”https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf”,”dataLabels”:true,”height”:420,”rotateLabels”:-20,”categories”:[“FP16 (54 GB)”,”UD-Q4_K_XL 4-bit (17.6 GB)”,”Bonsai 2 ternary (5.9 GB)”,”IQ2_XXS 2-bit (9.4 GB)”],”series”:[{“name”:”Benchmark average”,”data”:[86.32,85.18,84.78,72.59]}],”colors”:[“#0e7490″],”yAxis”:{“title”:”Average score”},”allowDownload”:true}[/interactive_chart]
The collapse of conventional 2-bit quantization is not uniform, which is the tell that something structural is going on. On multiple-choice knowledge tests like MMLU-Redux, the IQ2_XXS build holds up fine (88.93 versus 91.46). On multi-step reasoning it disintegrates: AIME 2026 drops from 94.58 to 57.50, and LiveCodeBench collapses from 90.05 to 56.40. Bonsai 2, meanwhile, stays at 95.83 and 90.07 on those same two benchmarks. Rounding a trained FP16 model down to 2 bits destroys the long reasoning chains; a model designed around three-value weights preserves them.
This is the strongest evidence in the release that post-hoc quantization and ternary conversion are fundamentally different operations, not points on the same scale. It also explains why “parameter count” and “precision” may need to be rethought as competing resources, a point we return to below.
Speed: 143 Tokens per Second on a GPU, Usable Almost Anywhere Else
Memory savings only matter if the compressed model is also fast, and here ternary weights have a structural advantage: arithmetic on minus one, zero, and plus one can be executed as additions and subtractions rather than full multiplications. PrismML reports batch-1 decode speeds measured on September 16, 2026, using custom low-bit kernels:
| Hardware | Reported speed | Notes |
|---|---|---|
| NVIDIA RTX 5090 | ~143 tokens/second | 0.582 mWh per token |
| NVIDIA RTX 4090 | ~97 tokens/second | PTQ1_0 packing |
| NVIDIA L4 (72 W) | ~32 tokens/second | Data-center low-power card |
| Apple M5 Max | ~47 tokens/second | MLX build, laptop-class hardware |
| Apple M5 Pro | ~28 tokens/second | MLX build |
[interactive_chart]{“type”:”column”,”themeMode”:”light”,”background”:”#f8fafc”,”title”:”Reported Decode Speed by Hardware”,”subtitle”:”Tokens per second, batch-1 decode (PrismML, September 16, 2026)”,”source”:”PrismML Hugging Face model card”,”sourceUrl”:”https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf”,”dataLabels”:true,”height”:400,”categories”:[“RTX 5090″,”RTX 4090″,”Apple M5 Max”,”NVIDIA L4 (72 W)”,”Apple M5 Pro”],”series”:[{“name”:”Tokens per second”,”data”:[142.5,96.7,46.8,32.1,27.7]}],”colors”:[“#155e75″],”yAxis”:{“title”:”Tokens per second”},”allowDownload”:true}[/interactive_chart]
PrismML claims roughly 40% better energy efficiency per token than a full-precision 8B model, which would make a 27B-capable model cheaper to run locally than a model less than a third its size. On the mobile side, the earlier Bonsai generation was reported at about 11 tokens per second on an iPhone 17 Pro Max, fast enough for interactive use, and Bonsai 2 ships MLX builds for iPhone and iPad plus a WebGPU demo that runs the model inside a browser at under 6 GB of memory.
These are all vendor-reported figures from a single evaluation pipeline, and community reproductions are still catching up. The honest summary for now: for any device with 8 GB of free memory, this class of model has moved from “technically possible” to “genuinely usable,” with CPU-only inference, consumer GPUs, Apple silicon, and phones all in play.
Does Parameter Count Matter More Than Precision?
Bonsai 2 27B quietly makes an argument that the industry has been circling for years: the number of parameters a model has may matter more than how precisely each one is stored. The 27-billion-parameter ternary model retains 98.2% of the full-precision original’s benchmark performance. The conventional 2-bit quantization of that same model, storing each weight with more distinct values than ternary has, retains just 84.1%.
If we take the 1.8% average loss at face value, a straightforward extrapolation follows. A ternary model with 30 to 35 billion parameters would need only about 6.4 to 7.5 GB at 1.72 bits per weight, still smaller than a 2-bit quantized 27B model, and would need to gain just 2 to 8% in raw capability over the 27B version to surpass the FP16 original on benchmarks. In other words, trading precision for parameters appears to be a winning exchange rate, at least in this regime. This is informed speculation, not a PrismML claim: no 30 to 35B ternary model has been trained or benchmarked yet, and retention could degrade less gracefully at larger scales or on tasks not captured in the average.
There is also a subtle selection effect to keep in mind. Ternary conversion is not free-form rounding; it involves rotations, group-wise scaling, and recovery fine-tuning tuned precisely to preserve benchmark performance. The comparison shows that this compression recipe beats naive low-bit quantization, which is impressive, but it does not yet prove that any arbitrary 35B ternary model would beat any 27B FP16 model. The next year of releases will test the thesis directly.
Training Is Still Expensive, but the Hardware Is Catching Up
One caveat gets lost in the excitement: Bonsai 2 27B did not get cheaper to train. Like most ternary releases, it was trained in full 16-bit or 32-bit precision by Alibaba, then converted to ternary weights afterward by PrismML. The compression saves inference cost, memory, and energy; it does nothing for training compute.
Two developments could change that arithmetic. The first is native low-precision training. Research on ternary architectures such as 1.58-bit BitNet models trained entirely in three-value precision has shown that models can learn from scratch in low-bit formats, which would cut training cost rather than only inference cost. The second is hardware support. Nvidia’s Blackwell generation introduced native 4-bit tensor cores, and the company’s low-precision training work shows FP8, MXFP8, and NVFP4 training runs matching BF16 training accuracy on downstream benchmarks, with an important caveat: fully NVFP4-trained models diverged, so stable training required keeping select layers in BF16. On the inference side, Nvidia’s NVFP4 quantization work reports about 1% or less accuracy loss on large models like DeepSeek-R1 when converted from FP8.
Why does 4-bit hardware matter for a ternary story? Because the weakest link in today’s pipeline is the conversion step: compressing a full-precision model down to three values costs a measurable amount of benchmark performance, the 1.8% we keep referencing. If future models are trained natively in low precision, that conversion gap could shrink further, and training itself could get cheaper. It is speculative but directionally clear: the hardware ecosystem is converging on low-precision arithmetic at every stage of the pipeline.
Two Ends of the Same Spectrum
The same week Bonsai 2 27B appeared, the AI industry’s other end was also in the news. We covered the Qwen 3.8 Max update, now China’s most expensive frontier AI, a model whose per-token costs, particularly on verbose outputs, sit at the top of the Chinese market. The contrast could not be sharper: one release maximizes capability regardless of price, the other maximizes capability per gigabyte of memory.
Until recently, the path to near-frontier local AI ran through hardware purchases. Enthusiasts bought AI mini PCs marketed as budget alternatives to DGX-class machines, and even specialized silicon entered the picture, such as Alibaba’s RISC-V chip running a 27B model without a GPU at all. Those products solved a real problem: full-precision 27B models need 32 to 64 GB of fast memory, which ordinary computers do not have.
Bonsai 2 reframes the problem. If a 5.9 GB ternary file delivers 98% of a model that itself rivals Opus-class systems, the hardware requirement for serious local AI drops from “specialized mini PC” to “any laptop or phone made in the last few years.” That does not make AI PCs or accelerators obsolete: the ternary model still loses badly on agentic coding tasks, larger ternary models will need memory again, and running multiple models or long contexts shifts the bottleneck back to hardware. But it does challenge the assumption that hundreds of gigabytes of RAM and expensive graphics cards are the only route to useful private AI.
The bigger implication is economic and environmental. Local inference means no per-token API charges, no data leaving the device, and, if the efficiency claims hold, far less energy per query than sending requests to data centers. As low-precision formats spread through training hardware and inference runtimes, the class of models that fit on everyday devices will keep growing. The question is no longer whether near-frontier models can run on a phone. It is how soon models that beat today’s frontier will.
Frequently Asked Questions
How big is Bonsai 2 27B, and can my device run it?
The text model is 5.93 GB in the memory-optimized PTQ1_0 format (7.21 GB in the compute-optimized PQ2_0 format), plus an optional 0.63 GB vision file. Any computer with 8 GB of available memory can run it, including recent laptops, mini PCs, and high-end phones. GPU acceleration requires an NVIDIA card via CUDA or Apple silicon via MLX; CPU-only inference is possible but slower.
Is Bonsai 2 27B as good as the original Qwen 3.8 27B?
On PrismML’s vendor-reported benchmarks, it averages 98.2% of the parent model’s scores, at parity on math and coding and slightly better on instruction following. It is meaningfully weaker on knowledge recall, vision, and especially long agentic tasks such as SWE-bench Verified (60.8 versus 80.6). Independent third-party evaluations have not yet been published.
What does “ternary” mean for an AI model?
Every weight in the model takes one of three values: minus one, zero, or plus one. Three states encode in about 1.585 bits each, versus 16 bits for a standard model, which is how 27 billion parameters fit in under 6 GB. High-precision scale factors, stored per group of 128 weights, preserve the model’s ability to express different magnitudes.
Does Bonsai 2 27B run on phones?
Yes. PrismML ships MLX builds for iPhone and iPad, and reports that the earlier Bonsai generation reached about 11 tokens per second on an iPhone 17 Pro Max. A WebGPU demo also runs the model in a browser. Community benchmarks for Bonsai 2 on specific handsets are still emerging.
Can I run it with standard llama.cpp?
Not with the main GGUF files. The PTQ1_0 and PQ2_0 formats use custom packing that requires PrismML’s fork of the llama.cpp runtime; incompatible runtimes may reject the files or misinterpret the weights. Apple users can sidestep this with the official MLX builds.
Was the model trained in ternary precision?
No. The underlying Qwen 3.8 27B was trained in full precision by Alibaba, and PrismML converted the weights to ternary afterward. Training cost was not reduced. Research on natively trained ternary models and low-precision training hardware could change that in future releases.
The Bottom Line
Bonsai 2 27B is not the most capable AI model released this month, and its vendor-reported benchmarks deserve healthy skepticism until independent evaluations land. What makes it important is the direction it points. A model within striking distance of systems that once required data center hardware now fits in the memory budget of a mid-range phone, and the evidence suggests the compression cost of that shrinkage is small and getting smaller.
If the emerging lesson holds that ternary parameter count matters more than per-parameter precision, the industry’s optimization target shifts: stop paying for bits and start paying for parameters. Combined with training hardware that is rapidly learning to operate below 16-bit precision, the practical ceiling for AI that runs privately, cheaply, and locally is rising fast. The expensive frontier and the phone-sized frontier are diverging products, and for a growing share of real tasks, the phone-sized one is already enough.
