News · 2026-09-22

Apple's M5 Ultra makes large local AI practical by capacity, not by beating an RTX 5090

A 256 GB M5 Ultra Mac Studio can run large sparse models and long local-agent contexts that overflow a 32 GB GPU, according to a detailed hands-on review. It does not beat an RTX 5090 on speed when the same workload fits in VRAM; its real advantage is keeping model weights and context in one large unified-memory pool.

Key facts

• Federico Viticci's MacStories review tested a 256 GB M5 Ultra Mac Studio with MLX and oMLX.
• Its low-bit Qwen3.8-Flash-Next path crossed roughly 100 tokens per second on short prompts.
• On a matched Qwen3.8 27B short prompt, an RTX 5090 produced 59 tokens/sec versus the Mac's 48.
• Apple says an M5 Ultra Mac Studio starts at $5,499.

The review is an important corrective to two common slogans: “frontier models now run on normal local hardware,” and “Apple Silicon beats GPUs for AI.” Neither is the right conclusion. The test machine was a top-end M5 Ultra with 256 GB of unified memory, running macOS 27 and the MLX/oMLX stack. That is a premium workstation configuration, not the ordinary laptop that most people mean by local AI.

Viticci tested Qwen3.8-Flash-Next in several quantizations, GLM-5.3-Flash, Qwen3.8 27B, local image generation, and agent workflows. In the 4-bit/5-bit Flash-Next path, short prompts exceeded 100 tokens per second. After very long contexts, generation remained around 75–85 tokens per second in the described use. Three concurrent Flash-Next requests completed in 22.1 seconds at 81.5 combined output tokens per second. These are one reviewer's disclosed software-and-hardware measurements, not a general benchmark or a guarantee for every model.

The decisive comparison is memory behavior. A 5090 was faster on a matched Qwen3.8 27B short prompt, 59 tokens/sec against 48 on the Mac, and still faster at 256K context when using an 8-bit attention cache, 30 against 24. Its Tensor Cores and bandwidth win when model and context fit inside 32 GB of VRAM. The M5's value appears after capacity is exhausted. When the 5090 path borrowed ordinary system RAM across PCIe, the review reports 4.6, 2.9, and 1.5 tokens/sec at 64K, 128K, and 256K context. The Mac retained a usable path because the CPU, GPU, model, and cache all shared the unified pool.

Think of VRAM as a fast workbench and unified memory as a much larger combined workshop. A task that fits on the workbench can be completed faster there. Once it spills into a distant storage room, every tool fetch becomes costly. The Mac's 256 GB workshop is slower at the bench but avoids that particular walk. This is why memory capacity, memory bandwidth, and compute throughput should not be collapsed into a single “can it run?” question.

The review also documents the limits. Qwen3.8-Flash-Next has a large N-gram embedding component; its 4-bit and 5-bit variants fit fully in the test machine's memory, while 6-bit and 8-bit variants held tables on SSD and fetched entries on demand. One load hit macOS memory pressure, another was refused by the serving software's projected limit, and a repeated 256K GLM retrieval test ran out of memory. SSD offload is a capacity workaround, not free memory: it trades latency and storage traffic for an otherwise impossible load.

Apple's starting price is $5,499, and the exact price of the reviewed 256 GB setup was not stated in the checked source. That price reframes the use case. A high-memory Mac can be a quiet private agent server for people who need large sparse models, long sessions, data locality, and predictable availability. It is not automatically cheaper than cloud inference, easier than a hosted model, or faster than a high-end GPU for a model that already fits.

The strongest counterargument is methodological. MLX/oMLX and the 5090's GGUF or alternative stack use different quantizations, attention caches, and implementations, so this is not a clean chip-only race. The reviewer also says he is not an AI developer. Those caveats improve rather than erase the result: it is a concrete workflow report. The local-AI market is bifurcating into 12–16 GB users who need quantization and careful model choice, and expensive unified-memory owners who can trade price and setup friction for capacity and privacy.

The useful buying question is not whether a model technically launches, but whether it delivers acceptable first-token delay, sustained generation, concurrent-user behavior, context length, and reliability at the precision the intended work actually needs.

Primary source, verified: read the paper →

Ground Truth. (2026, September 22). Apple's M5 Ultra makes large local AI practical by capacity, not by beating an RTX 5090. Ground Truth. https://groundtruth.day/news/m5-ultra-local-ai-capacity-not-speed.html

@misc{groundtruth:m5-ultra-local-ai-capacity-not-speed,
  title  = {Apple's M5 Ultra makes large local AI practical by capacity, not by beating an RTX 5090},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/m5-ultra-local-ai-capacity-not-speed.html}
}

Topics: local-ai · hardware · apple-silicon · inference · open-weights