apple-silicon
A 284-billion-parameter model with a 3-gigabyte working set, and a 96-gigabyte disk bill News
An open-source engine called Mference runs DeepSeek V4 Flash on a 24 GB Mac with an effective memory footprint of about 3 gigabytes by streaming each token's experts off the SSD, but the checkpoint still occupies 90 to 98 gigabytes of disk and the test ran at a 4,000-token context.
A 26-billion-parameter model runs in 2GB of RAM by streaming experts off the SSD News
TurboFieldfare, an open-source Swift and Metal runtime, runs Gemma 4's 26-billion-parameter model on an 8GB MacBook Air by keeping only a 1.35GB core in memory and pulling each token's experts from disk as it needs them.
Ollama nearly doubles Gemma's speed on Macs by guessing ahead News
A free local-AI tool now runs Google's Gemma model far faster on Apple computers using a trick where a small model drafts words and the big one checks them in bulk.
TurboQuant-MLX Tool
Quantization tooling for MLX with published size and speed measurements, including a 3-bit path that takes a 120-billion-parameter model from about 63GB down to 48GB on consumer Macs.
TurboFieldfare Tool
A Swift and Metal runtime that runs Gemma 4's 26B model on an 8GB MacBook Air by keeping a 1.35GB core resident and streaming the rest of the experts off the SSD. Ships as a Mac app, a CLI and an OpenAI-compatible local server.
SwiftLM Tool
An MLX-based runtime for Apple silicon whose --stream-experts mode reads mixture-of-experts weights straight off an NVMe SSD, letting a machine run models several times larger than its RAM.
Ollama 0.31 Tool
Run open models on your own computer; the new version nearly doubles Gemma's speed on Apple Silicon using multi-token prediction, on by default.
Mference Tool
Runs DeepSeek V4 Flash on Apple silicon by keeping the shared core, attention and cache resident while streaming each token's routed experts off the SSD. Publishes an unusually honest memory budget: about 3 GB working set, 90 to 98 GB on disk, tested at a 4,000-token context on a 24 GB Mac, with no quality parity test yet.
Maple-Preview (2-bit MLX build) Tool
DeepGrove's 20B mixture-of-experts model with about 1B active parameters per token, packaged for Apple Silicon at roughly 5.3GB. The build uses affine two-bit group quantisation with four-bit embeddings and output head, and its loader packs ternary values into two-bit codes. Note that the native BF16 repository is about 40.4GB, and DeepGrove publishes no ternary training recipe or independent evaluation.