Flint vs. Bonsai 2 27B: The Honest Local Coding LLM Comparison

Two open-weight models aimed at local coding landed five weeks apart, and they represent two completely different bets on how to squeeze near-frontier intelligence into personal hardware. PrismML’s Bonsai 2 27B, released September 17, 2026, takes a dense 27-billion-parameter model and compresses it down to 5.9 GB with ternary weights. Recursive’s Flint, released October 21, 2026, spreads 40 billion parameters across a mixture-of-experts architecture and activates only 2.5 billion per token. Both are Apache 2.0, both run offline, and both claim class-leading coding performance. Only one of them, in most people’s hands, actually delivers interactive speed.

In one direct test on a single GPU, Bonsai fit entirely in video memory and flew; Flint did not fit and crawled, because part of its 24 GB of quantized weights had to route through slow system memory. That experience is not a quirk. It is the central trade-off between these two designs, and it explains almost everything about when each model is the right pick. Here is how they compare on benchmarks, speed, memory, and practical daily use.

Why These Two Models Demand a Head-to-Head

The comparison matters because the two models overlap in exactly the slot they target: a private, permissive-licensed coding assistant for hardware you already own. They also landed within five weeks of each other, and both vendors publish results on shared benchmarks, which makes a genuine head-to-head possible, something you cannot do with most local releases.

One caution up front: every number in this article is vendor-reported. Recursive ran its own harness; PrismML ran its own. Neither set has been independently reproduced. Read every table below as directional, not definitive.

Two Philosophies of Efficiency

Bonsai 2 27B is not a new model. PrismML took Qwen 3.8 27B, the open dense model that rivals GPT-5.6 and Claude Opus-class systems on coding tasks, and rounded every language-model weight to one of three values: minus one, zero, or plus one. Three states need only about 1.585 bits, and with group-wise FP16 scales the shipped file lands at 1.76 bits per weight, or 5.93 GB. All 27.36 billion parameters stay active for every token; the trick is that ternary arithmetic runs as additions and subtractions instead of full multiplications, and each token moves roughly a ninth of the memory traffic an FP16 model would. This is the second generation of a recipe that retained about 95% of its parent the first time around, as covered when 1-bit LLMs brought 27B-class AI to laptops and phones.

Flint is the opposite bet. Recursive, a lab founded by former Google, Meta, and OpenAI researchers and led by Richard Socher, trained a 40-billion-parameter mixture-of-experts model from scratch in under four months, with heavy help from its automated research system, the Eureka Machine. Each of its 256-expert layers activates six experts plus one shared expert for any given token, so only about 2.5 billion parameters do arithmetic per token. The full 40 billion still have to sit somewhere in memory, which is where the size difference bites.

The spec sheet side by side:

The Benchmark Head-to-Head

This is where the comparison gets genuinely interesting, because both vendors measured their models on overlapping agentic-coding tests. The parent model, Qwen 3.8 27B at full precision, is included as the reference point both releases measure themselves against.

On long-horizon agentic coding, the workload that chains dozens of tool calls through a real repository, Flint is the clear winner: 13.4 points ahead on SWE-bench Verified and 2.1 ahead on Terminal-Bench 2.1, despite carrying four times the memory footprint. Recursive’s scaffolds also gave agents up to five hours per task across at least five attempts, a demanding protocol. On single-shot coding, the picture inverts completely: Bonsai 2 posts 90.07 on LiveCodeBench v6, essentially matching its full-precision parent, while Flint scores 80.3. Flint also edges instruction following (95.1 on IFEval) and comfortably wins the competition-math check where Bonsai 2 fights back (95.83 on AIME 2026, actually above the parent’s 94.58).

The pattern is coherent. A dense model with all parameters active, even aggressively compressed, holds onto the focused reasoning that single-shot coding requires. A sparse model built and trained for agentic loops, with a 512K context window and purpose-built kernels, wins when the job is long, multi-step, and tool-heavy. Neither is “best” at everything, and the honest answer to which is best on benchmarks is: Flint for agentic work, Bonsai 2 for everything that fits in a single prompt.

Speed on Real Hardware

Raw token-generation numbers, batch-1 decode, as published by each vendor:

Two rows deserve attention. On Apple’s M4 Pro, Flint’s optimized rec engine, with its Metal megakernel fusing the entire decode pass, hits more than 125 tokens per second where Bonsai’s llama.cpp Metal path manages 18. Different inference stacks complicate the comparison, but the active-parameter design is doing real work there.

On the other hand, Recursive’s own charts already acknowledge this: they report average tokens per second across a full 128K-token session rather than the short-context peak, and coding agents are precisely the workload that fills context. Bonsai 2’s predominantly linear-attention backbone is built for exactly that sustained long-context case.

Then there is the fit problem, which firsthand testing brought home. With Bonsai 2 fully resident on the GPU, tokens streamed at full speed. Flint, with over 21 GB of 4-bit weights, spilled past the card’s memory, and generation dropped to a crawl as system RAM took over the spillover. Same machine, same day. The lesson is not that one architecture is faster; it is that a model you cannot fit never gets to show you its speed. Fully fitted on a 32 GB card, Flint’s low active-parameter count gives it a per-token traffic advantage and speeds in the same 100 to 150 tok/s class as Bonsai 2.

If you want to reproduce the comparison on your own machine, the test setup matters as much as the hardware:

• Measure free video memory first, then pick the file that fits with 2 to 4 GB left for the KV cache.
• Match runtimes to packings: upstream llama.cpp b11555 or newer for Bonsai 2’s Q2_0 file, PrismML’s fork for the denser packings, any recent build for Flint.
• Use each vendor’s recommended thinking-mode sampling (temperature 1.0, top-p 0.95, top-k 20) so runs are comparable.
• Set a generous output budget; both models think before answering, and small caps cut them off mid-reasoning.
• Benchmark short prompts and a filled context separately, because Flint’s speed gap between the two is roughly 14x.
• Score prompt processing on its own; reading a codebase is compute-bound and behaves differently from decode.

Memory, Hardware Fit and the 24 GB Question

The footprint chart is the whole story for most buyers. Bonsai 2 runs from 5.93 GB in its densest packing to 8.49 GB in the Apple MLX build, which puts serious local coding inside any recent laptop with 8 to 16 GB of free memory, and it ships MLX builds for iPhone and iPad plus a WebGPU demo that runs in a browser. The full-precision parent needs 54 GB; the standard 4-bit quantization of that parent needs 17.6 GB; and conventional 2-bit quantization of the same model collapses to a 75.2 average while Bonsai 2 holds 83.9, which is why the ternary route exists at all.

Flint’s usable builds start at 21.28 GB. On a 24 GB card that leaves almost nothing for the KV cache, so expect layer splitting and a real speed penalty. A 32 GB card takes the default file with short to medium context, and a 64 GB RTX Spark is Recursive’s reference machine. Recursive’s own engine targets M4 Pro and M5 Pro/Max Macs with at least 48 GB of unified memory. Below the 4-bit tier, Unsloth’s ladder down to 1-bit (11.4 GB) exists, but quality degrades sharply below 3-bit, and on a 16 GB machine a smaller model at 4-bit is usually the better buy than Flint at 1-bit.

Memory capacity, not raw chip speed, decides which of these models you can actually use. That is also why the enthusiasts building dedicated local AI servers at home are suddenly reconsidering: a 5.9 GB ternary file removes most of the reason for the mini server rack.

Ecosystem and Practical Differences

Flint runs on stock llama.cpp (its architecture landed upstream in October 2026), plus Ollama, LM Studio, vLLM, SGLang, Unsloth, or Recursive’s lightweight rec engine. That is the most frictionless onboarding in local AI right now: download, run, done. Bonsai 2’s two flagship GGUF packings require PrismML’s llama.cpp fork, though a third packing (Q2_0, 7.63 GB) runs on upstream llama.cpp build b11555 or newer, and Apple users can sidestep the whole question with native MLX builds. PrismML also ships an OpenAI-compatible server with vision and tool calling wired in.

Both are reasoning models that write a visible thinking block before answering, so both want generous output budgets and llama.cpp builds recent enough to know their architectures. Both recommend near-identical thinking-mode sampling (temperature 1.0, top-p 0.95, top-k 20). Bonsai 2 defaults to xhigh reasoning effort, thinks noticeably less at medium, and does not support low; Flint advertises a 512K context but recommends staying under 256K. For multimodal work there is no contest: Bonsai 2 takes images through a separate vision tower, while Flint is text-only. Both licenses are Apache 2.0, so either can be commercialized without fees, and both keep every byte of code on your machine, which is the entire point of running a local model in the first place.

Which Model Should You Run?

The decision tree is short, and VRAM draws the branches.

1. 8 to 16 GB of RAM or VRAM, including laptops and phones: Bonsai 2 27B, full stop. It is the only near-frontier-class model at this footprint, and Flint’s 1-bit fallback is not competitive.
2. 24 GB card: still Bonsai 2. Flint fits only with painful layer splitting, and the speed tax for the spill is exactly what a single-GPU test showed.
3. 32 to 64 GB: now Flint earns serious consideration. Fully resident, it leads on agentic coding, offers 512K context, and runs on stock tooling. If your card is 48 GB or larger, you can even keep both resident and route single-shot work to Bonsai 2 and long agentic sessions to Flint.

If your work is short-prompt coding help, explanations, refactors, and tests, Bonsai 2 27B is the better daily driver at every hardware tier, and its energy cost is remarkable (0.582 mWh per token on an RTX 5090, with PrismML claiming roughly 40% better efficiency than a full-precision 8B model). If you are running an autonomous coding agent that spends hours inside a codebase and you own memory-rich hardware, Flint’s benchmark lead and ecosystem support make it the stronger tool.

What the Numbers Still Cannot Tell You

Three reasons to hold all of this loosely. First, everything above is vendor-reported: Recursive scores Flint on its own harness, and PrismML scores Bonsai 2 on EvalScope with vLLM. The scaffolds differ too: Terminus 2 with internet access for Flint’s Terminal-Bench run, single-attempt Terminus-2 JSON for PrismML’s.

Second, the vendors disagree about the shared reference point: PrismML’s whitepaper scores Qwen 3.8 27B at 69.7 on Terminal-Bench 2.1 while Recursive’s chart shows the same model at 73.0. When two labs cannot agree on the baseline by three points, treat five-point gaps between models as noise until independent runs exist.

Third, quantization changes each model in ways benchmarks under-report: a quantized build can behave differently on your specific codebase than any published chart suggests, which is one more reason single leaderboard numbers, like MMLU scores, rarely predict real task performance.

There is also the gap between the compressed models and their parent. Both lose ground on long agentic tasks relative to full-precision Qwen 3.8 27B (80.6 on SWE-bench Verified), PrismML openly frames Bonsai 2’s agentic performance as its next roadmap step, and Recursive itself says Flint needs complex work broken into smaller subtasks with explicit instructions. Local, private, and near-frontier is real; local, private, and effortless is not quite here.

Frequently Asked Questions

Which model wins on benchmarks, Flint or Bonsai 2 27B?

Split decision, based on vendor-reported numbers. Flint leads on long-horizon agentic coding: 74.2 versus 60.8 on SWE-bench Verified and 54.9 versus 52.8 on Terminal-Bench 2.1. Bonsai 2 27B leads on single-shot coding, 90.07 versus 80.3 on LiveCodeBench v6, and its parent Qwen 3.8 27B beats both on the agentic tests. Because the two vendors use different harnesses, treat the gaps as directional.

Why might Bonsai run faster than Flint on some GPUs?

Almost certainly memory fit. Bonsai 2’s weights are 5.9 to 7.6 GB, so they fit entirely in your GPU’s fast memory, and ternary decoding is cheap arithmetic. Flint’s 4-bit builds start at 21.28 GB, so a card that cannot hold them spills into system RAM, and decode speed collapses to the bandwidth of your slowest memory. On a 32 GB or larger card, fully fitted, Flint’s speed is in the same class.

Can both models run on the same machine?

Yes, on roughly 48 GB of VRAM or unified memory, since 5.95 plus 21.3 GB of weights plus cache headroom fits comfortably. On smaller machines you must pick one, and the footprint column decides.

Do they work with Ollama and LM Studio?

Flint does, natively, along with stock llama.cpp and vLLM. Bonsai 2’s main packings need PrismML’s llama.cpp fork; its Q2_0 packing runs on upstream llama.cpp b11555+, and its MLX builds run in mlx-vlm 0.7.8 or newer.

Can Flint be squeezed onto smaller GPUs with lower-bit builds?

Partly. Community builds go down to a 1-bit file at 11.4 GB, which fits a 16 GB machine, but quality falls sharply below the 3-bit tier, and by 1-bit the model codes considerably worse. On a tight machine, a smaller model at 4-bit is usually the better buy than Flint at 1-bit.

Is either model actually free?

Both release weights under Apache 2.0, so personal and commercial use carry no per-token fees. Your costs are a download (6 to 25 GB), hardware, and electricity.

Which one is better for privacy and energy use?

Both keep code on your machine and work offline. Bonsai 2 has the edge on energy per token at 0.582 mWh on an RTX 5090 and runs on devices drawing laptop-class power, while Flint needs a bigger memory footprint and therefore bigger hardware.

The Bottom Line

Flint and Bonsai 2 27B are not really competitors so much as two answers to the same question, tuned for different checkbooks of memory. Bonsai 2 compresses a proven dense 27B model into a file that fits nearly anything, keeps almost all of its single-shot coding quality, and pays for that with a real drop on long agentic work. Flint trains a purpose-built sparse model that wins the agentic benchmarks and the context-length game but demands three to four times the memory, with headline speeds that only exist on well-fitted hardware and short contexts.

If your GPU has 24 GB or less, the decision makes itself: Bonsai 2 27B is the model, and the speed difference you noticed is the ternary bet paying off. If you are sitting on 32 to 64 GB and your workload is hours-long agentic coding, Flint is the better instrument and, five weeks newer, the one with more momentum behind it. Full details live in each release’s own documentation: Recursive’s Flint announcement, the Flint weights on Hugging Face, PrismML’s Bonsai 2 documentation, and the Bonsai 2 model card.

And when both vendors agree on nothing except the license, the most honest local coding LLM comparison ends where it started: run the one that fits, then test it on your own code. For deeper dives on each contender, read our full Flint analysis and our Bonsai 2 27B breakdown.