News · 2026-07-30

A 26-billion-parameter model runs in 2GB of RAM by streaming experts off the SSD

An open-source project called TurboFieldfare runs Gemma 4's 26-billion-parameter model on an 8GB MacBook Air using about 2GB of memory, by keeping a small shared core resident and streaming the rest off the SSD one token at a time. The repository, from developer drumih, has gathered over 2,000 stars since mid-July and is Apache 2.0 licensed. Its own README states the trick without embellishment: it "streams only the experts needed for each token from SSD."

Key facts

• Runs Gemma 4 26B-A4B with ~2GB of weights and key-value cache in memory; the installed model is 14.3GB on disk.
• Measured decode speed: 5.1-6.3 tokens/sec on an 8GB M2 MacBook Air, 31-35 tokens/sec on a 24GB M5 Pro.
• Written in Swift and Metal, model-specific rather than a wrapper around MLX or llama.cpp; Apache 2.0, 2,000+ GitHub stars.
• Primary source: the TurboFieldfare repository.

The reason this works is the shape of the model rather than any compression miracle. Gemma 4 26B-A4B is a mixture of experts: it holds 26 billion parameters but activates only about 3.88 billion of them for any given token, routing each word to a small subset of specialised blocks. Standard runtimes load all of it into memory anyway, because you cannot predict in advance which experts a token will need and fetching them late is slow.

TurboFieldfare takes the opposite bet. It keeps the always-needed parts resident - a 1.35GB shared core, the router, and the key-value cache in 16-bit precision - and fetches the routed experts from the SSD per token. The weights are stored in 4-bit precision with an 8-bit router, so each fetch is small. Modern Apple Silicon SSDs are fast enough that this loses less than the intuition suggests.

A useful way to picture it: rather than moving an entire reference library into a small office, you keep the index and the few volumes you consult constantly on the desk, and walk to the stacks for anything else. It works because you only ever need a handful of volumes per question, and the stacks are close.

The honest number is the throughput. On the 8GB M2 MacBook Air - the machine the headline is really about - the project measures 5.1 to 6.3 tokens per second. That is roughly the pace of a person reading aloud: perfectly usable for a chat, slow for anything agentic that generates thousands of tokens. On a 24GB M5 Pro, where more of the model can stay resident, it reaches 31 to 35 tokens per second. The gap between those two figures is the cost of the trick, stated plainly, which is more than most projects do.

What is notable is that the README does not oversell it. It says the model runs "without loading the entire 14.3 GB model into memory," gives the storage requirement up front, notes that the installer repacks about 15GB, and publishes 103 measured results across kernels, caching, I/O, prefill and decode in a documented experiment record. The distinction between "runs in 2GB of RAM" and "needs only 2GB of RAM resident at a time" is one that headlines lose and the repository keeps.

This is the third distinct route to the same destination in as many months. Quantization shrinks the weights; sparse activation shrinks the compute; expert streaming shrinks what has to be resident. Ground Truth has covered three separate tricks dropping the local inference floor in one day and a 28.9-million-parameter model running on an $8 microcontroller. The pattern is consistent: the constraint that actually binds local inference is memory bandwidth and capacity, not arithmetic, which is why inference is memory-bound and why every meaningful optimisation is about moving fewer bytes.

The caveats are real and the project names most of them. It is model-specific rather than general - it runs Gemma 4 26B-A4B and not whatever you would prefer. It requires macOS 26, Metal 4 and Swift 6.2, so it is Apple Silicon only. Throughput depends on prompt length, generated length and the state of the operating system's page cache, which means the published benchmarks are a reference point rather than a guarantee. And streaming from SSD means sustained heavy use writes and reads more than a resident model would. None of that undercuts the result. A 26-billion-parameter model answering questions on the cheapest MacBook Apple sells is a genuinely different world from the one where that required a workstation.

Primary source, verified: read the paper →

Ground Truth. (2026, July 30). A 26-billion-parameter model runs in 2GB of RAM by streaming experts off the SSD. Ground Truth. https://groundtruth.day/news/a-26-billion-parameter-model-in-2gb-by-streaming-experts-off-the-ssd.html

@misc{groundtruth:a-26-billion-parameter-model-in-2gb-by-streaming-experts-off-the-ssd,
  title  = {A 26-billion-parameter model runs in 2GB of RAM by streaming experts off the SSD},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/a-26-billion-parameter-model-in-2gb-by-streaming-experts-off-the-ssd.html}
}

Topics: local-inference · open-source · apple-silicon · mixture-of-experts · quantization · tools