Flint: The Fastest Local Coding Model Is a 40B MoE Built Mostly by AI

On October 21, a startup called Recursive released Flint, a 40-billion-parameter coding model you can download and run on hardware you already own. It is neither the biggest model of the year nor the most capable. It may be the most interesting, for how it was built and for what it says about where a local coding model can go.

Flint sends each token down a narrow path through the network, so only about 2.5 billion of its 40 billion parameters do the work of producing it. The rest stay loaded in memory, idle until a different token calls for them. Less arithmetic per token is what lets an ordinary laptop keep up, reaching more than 125 tokens a second on an M4 Pro MacBook Pro, though the compressed weights and custom kernels described below contribute nearly as much.

The other half of the story is stranger. Recursive says an internal system called the Eureka Machine helped design nearly the whole thing: the architecture, the compression recipe, and the low-level code that makes it run fast. Here is what actually shipped, what the speed numbers mean in practice, and where the claims run ahead of the evidence.

What Recursive Actually Released

Flint is the first model from Recursive, a lab founded by former Google, Meta, and OpenAI researchers and led by Richard Socher. The weights were published under the Apache 2.0 license, which means anyone can download, run, modify, and even commercialize the model without paying per token.

The spec sheet, drawn from Recursive’s launch post and the model card, is compact enough to hold in your head:

• 40 billion total parameters, about 2.5 billion active per token
• Each mixture-of-experts layer holds 256 experts, though only six, plus one shared expert, apply to any given token
• Up to 512,000 tokens of context, though Recursive recommends staying under 256,000
• Trained from scratch in under four months by a small team of people and agents
• Runs through llama.cpp, Ollama, LM Studio, vLLM, SGLang, Unsloth, or Recursive’s own inference engine

It is a reasoning model rather than a plain chat model. It writes a visible thinking block before it answers, and its recommended sampling settings (temperature 1.0, top-p 0.95, top-k 20) are tuned for that behavior rather than for polite conversation.

Why 2.5B Active Parameters Makes Flint Fast

A mixture-of-experts model is built from specialized sub-networks, called experts, and a router that picks a small handful of them for each token. Six of 256 experts handle this word, a different six the next one, and the remaining 250 do nothing for either. Each token touches about 2% of the experts in a layer, and Flint routes 47 of its 49 layers this way, skipping only the first two.

This is the same dense-versus-sparse trade-off that now shapes most efficient model design, and the consequences reach past any single benchmark. A sparse model can match the coding quality of a dense model with 25 to 40 billion parameters while doing a fraction of the arithmetic per word.

Recursive claims Flint runs more than five times faster than those dense models, a gap that widens further on hardware with limited memory bandwidth. That is the part hardware makers care about, because it turns a model that needed a server rack into one that fits on a desk.

The size of that shift shows up best next to Meta’s Muse Spark Open, another open model aimed at Macs. It carries 816 billion parameters and needs at least a terabyte of memory to run, which puts it in a different category of machine entirely. Flint does comparable work in its own class from 40 billion parameters and roughly 24 GB on disk.

What the Eureka Machine Actually Did

Recursive is not claiming that an AI built a model by itself. What it claims is narrower, and more interesting: the search over architecture, compression, and compute kernels was automated, and the gains compounded layer by layer.

People stayed in the loop throughout. A small team of researchers and agents set the targets, decided which experimental directions to fund with compute, reviewed what came back, and did the engineering that turned a promising design into a working release. The distinction matters. “Automated research” describes where the searching happens. It does not mean nobody was steering.

Nor is Flint an isolated experiment. Anthropic and OpenAI already use AI to automate large parts of their own model development, so the idea is less novel than it sounds. A study of more than 700 task logs found that agents now handle much of the work while humans still make most of the critical research calls.

Recursive is trying to automate that final judgment, and Flint has not shown that it can. Neither has anyone else.

Three examples from the launch post show what that looked like in practice.

• Architecture. The team started from a baseline of 35 billion total parameters with 3 billion active. The Eureka Machine explored different expert configurations and widths and landed on the 40 billion / 2.5 billion design as the best projected balance of speed and coding ability.
• Compression. Against common practice, it quantized the output layer to 4 bits. Because only 2.5 billion parameters are active, that output layer is a large share of everything read per token, so shrinking it pays off directly. To recover the accuracy lost, it added a distillation stage where the compressed model trains on its own pre-sampled answers, picked from a coreset it discovered itself, with the 16-bit original as teacher. Recursive says this nearly halved the number of training steps versus its baseline.
• Kernels. It wrote specialized code for low-bit projections, expert routing, attention, and prompt processing. On M4 and M5 chips it produced a Metal “megakernel” that fuses the entire decoding pass into one operation, so the model is not just smaller on paper but faster in practice.

What humans still did gets overlooked. They set the benchmarks the system optimized against, decided where to spend compute, and judged what came back. The Eureka Machine widened the search. It did not decide what success looked like.

The money behind that claim belongs in the same conversation. Recursive came out of stealth in May 2026 with $650 million at a $4.65 billion valuation, backed by GV, Nvidia, and AMD Ventures, at a point when it had no published technical results to show. Flint is the first evidence that the approach produces anything at all. That clears a very low bar.

The kernel work is the most concrete result. NVIDIA’s own numbers show a 55% speed gain on its RTX Spark hardware from code alone, with the weights untouched. Recursive’s wager is that a system that improves its research process will produce the next model faster than a bigger team could. Flint is a first data point, not the proof.

What “Runs on a CPU” Really Means

This is where explanations usually get vague, so it pays to use numbers. Flint can run without a graphics card, but only on a machine with an unusually large amount of fast memory. The weights have to fit somewhere, and the model is 40 billion parameters whether or not you own a GPU.

The uncompressed model is about 79.5 GB. No consumer machine runs that comfortably. The usable versions are 4-bit quantizations, and a community port has published measured quality and speed figures for each one at the flint-GGUF repository.

Those are one volunteer’s files. Unsloth’s Dynamic 3.0 builds cover a much wider ladder, all the way down to 1-bit:

The 1-bit file is the interesting one, because it puts a 40-billion-parameter model inside a machine that would choke on the 24 GB default. It is also where the trade turns bad. Quality degrades noticeably from the 3-bit tier downward, and by 1-bit the model is a different proposition: it fits almost anywhere and codes considerably worse. On a 16 GB laptop, a smaller model at 4-bit is usually the better buy than Flint at 1-bit.

Treat those numbers as a memory floor. A 24 GB graphics card needs careful layer splitting to hold the default file, while a 32 GB card takes it with short to medium context.

On a Mac or a CPU-only box, the same 4-bit file needs roughly 20 to 25 GB of usable RAM just for the weights, plus room for the context window, which grows with every token you feed in.

Running the full checkpoint on a Mac is a heavier ask. Recursive’s own engine targets M4 Pro, M5 Pro, and M5 Max machines with at least 48 GB of unified memory and about 30 GB of free disk, which is a real laptop but a well-specified one, not a five-year-old ultrabook.

CPU inference is genuine rather than theoretical, because the 2.5 billion active parameters keep the arithmetic per token modest and llama.cpp runs the same file on CPU, Metal, CUDA, ROCm, or Vulkan. What you will not get is that speed. Processing on a CPU is bound by memory bandwidth, which is the same reason aggressive compression is what finally pushed larger models onto laptops and phones in the first place.

Measured Speed, Not Marketing Speed

The 125 and 150 tokens per second figures are real, but they come with a condition buried in Recursive’s own announcement: short contexts. For scale, comparable models, including Google’s Gemma 4 and Nvidia’s Nemotron 3.5 Lightning, managed 58 to 69 tokens per second on the same Mac in Recursive’s own tests. Speed also falls as the context fills, and coding agents are exactly the workload that fills context.

The third row is the one to trust, because it comes from an independent benchmark rather than a launch chart. It shows what a well-matched 32 GB workstation card does with the standard quantized file: fast enough to feel interactive, and roughly half the laptop figure. The hardware counts as much as the model.

Prompt processing, the phase where the model reads your codebase, ran at 4,450 to 4,800 tokens per second across the three quantization files on the same card. Reading is much cheaper than writing, which is why a large context window is survivable at all.

Where Flint Falls Short

Before you download it, six things are worth knowing.

• The benchmarks are vendor-run and not independently verified. Recursive’s 74.2% on SWE-bench Verified, 89.7% on SWE-Bench Pro v2, and 54.9% on Terminal-Bench 2.1 come from its own harness. No outside lab has reproduced them.
• The top speed only applies to short contexts, and it collapses as they grow. One reviewer measured roughly 9 tokens per second with the context filled to about 115,000 tokens, against 125 tokens per second at short contexts. Coding agents are precisely the workload that fills a context window.
• It does not win every test. Flint leads all six coding and instruction-following benchmarks among sparse MoE models under 50 billion parameters, including Qwen3.6-35B-A3B and Laguna S 2.1. Dense models such as Qwen 3.8 still score higher on some of those same tests, so the class-leading claim is scoped to its weight class rather than to coding in general.
• It needs more hand-holding than the big paid assistants. Recursive says so itself, advising users to break complex work into smaller subtasks with explicit instructions. That is not fatal for an offline model, but it narrows what you can hand it.
• The license is permissive; the safety guidance is thin. The model card warns that Flint can produce buggy code and unsafe tool calls and that any agent built on top must enforce its own permissions. Running locally does not make generated code safe to execute blindly.
• Quantization changes the model. The perplexity penalty in the table above is small but not zero, and it concentrates in rare experts and unusual tokens. A quantized build can behave differently on your specific codebase than any benchmark suggests.

Flint also arrives into a crowded open-weight market, where a 27B open model already trades blows with far larger systems on coding tasks, so speed rather than raw capability is the honest reason to pick it.

Not everything cuts against it, though. One test on record is genuinely independent: Wired’s Will Knight built several small programs on a Mac with 48 GB of memory and reported that Flint handled his coding requests reliably, where smaller models had often stumbled.

Why a Model That Fits on Your Desk Matters

Most arguments about AI come down to benchmark scores. The shift happening here is less about capability and more about who can run the model at all, and what it costs to do so.

A local coding model changes three things for anyone weighing the environmental and practical cost of AI.

It removes the data center from the loop for that workload, so there is no server sitting idle between your requests. It removes the meter, because your marginal cost per token becomes electricity rather than a per-token fee. And it removes the network, which means code never leaves the machine, and the tool keeps working when the connection does not.

None of that makes local inference automatically greener. A laptop drawing 60 watts for an hour is not free, and its embodied carbon is already spent.

But a model that activates 2.5 billion parameters instead of 40 billion does less work for the same output, and that ratio is the most direct lever anyone has on energy per token. Sparse, aggressively compressed models are how that lever gets pulled, and the fact that one now runs at interactive speed on a well-specced laptop is the development to watch.

How to Try Flint on Your Own Machine

If you want to test it, the path is short, but the order matters.

1. Check your memory first. You need roughly 20 to 25 GB free for a 4-bit build, plus headroom for context. Below that, look at a smaller model instead.
2. Install a recent build of llama.cpp. The flint architecture only landed upstream in October 2026, so older builds fail with an unknown architecture error.
3. Download the quantization that matches your hardware. Q4_K_M if you have the room, IQ4_NL or IQ4_XS if you are tight, or Unsloth’s UD builds if you need to go smaller. Do not go below 3-bit unless getting it to fit counts for more than output quality.
4. Give it a generous output budget. Flint thinks before it answers, so a small token limit returns reasoning without a conclusion.
5. Start with a small, well-defined task, then work up. This is the workflow Recursive recommends, and it is the one that matches what a 2.5-billion-active model does well.

The original weights are published on Hugging Face under Apache 2.0, and Recursive’s engine is aimed at recent Apple silicon with 48 GB or more. Both deserve a read before you commit a 24 GB download to a metered connection.

Frequently Asked Questions

Is Flint free to use?

Yes. The weights are released under the Apache 2.0 license, which permits personal and commercial use without per-token fees. Your costs are hardware and electricity.

Can Flint run on a CPU without a graphics card?

It can, through llama.cpp, which supports CPU-only inference. The limit is memory rather than compute: a 4-bit build needs roughly 20 to 25 GB of RAM, plus space for context. Expect much slower generation than the GPU figures.

How much memory do I need for Flint?

Plan on 21.28 GB for the smallest 4-bit file, 22.51 GB for the faster one, or 24.12 GB for the best-quality one, then add room for the KV cache. A 32 GB machine is the realistic minimum for comfortable use. Unsloth’s ladder goes as low as 11.4 GB at 1-bit and 18.5 GB at 3-bit, but quality falls sharply below 3-bit, so treat those as a last resort rather than a way to save disk space.

Was Flint really built by AI?

Partly, though “built by AI” needs unpacking. A small team of people working with AI agents built it, while an automated research system proposed ideas, implemented them, ran experiments, and chose what to try next. Recursive describes that as automation under human supervision, not an autonomous model that built itself.

Is Flint better than Qwen or Gemma for coding?

On Recursive’s own benchmarks, it leads mixture-of-experts models under 50 billion parameters, and it is substantially faster than dense models of similar ability. Independent testing is still thin, so judge it on your own tasks rather than on a vendor’s chart.

Does Flint work offline?

Yes. Once the weights are downloaded, no network connection is required, and no code leaves your machine unless you build an agent that sends it somewhere.

The Bottom Line

Flint is a good model with an unusually clear thesis behind it. The efficient part, a 40-billion-parameter mixture-of-experts model that activates only about 2.5 billion parameters per token, is a proven recipe executed well. The AI-designed part is a plausible early result rather than a demonstration, and Recursive’s own notes about hand-holding and unverified benchmarks are the right ones to keep in mind.

The reason to care is practical. A capable local coding model that runs on a laptop, offline, with no meter running, is a different kind of tool from one that rents you intelligence by the token. Flint is not the last word on that idea, but it is the most convincing version of it so far.