News · 2026-10-17

Apple ran Muse Spark Open at 36.73 tokens a second across four Mac Studios

Apple published a distributed inference recipe that runs Meta's Muse Spark Open at 36.73 generated tokens per second across four M5 Ultra Mac Studios. The result uses a compressed checkpoint and an 8,192-token context. Two-machine configurations also work, but they are separate measurements and do not produce the quoted four-machine speed.

Key facts

• Apple published the distributed MLX recipe October 15, 2026.
• Its 36.73-token-per-second result uses four Macs, tensor parallelism and the NVFP4 checkpoint.
• Meta documents either two machines with 512 GB unified memory each or four with 256 GB each.
• Primary sources are Apple's benchmark and Meta's deployment guide.

The important advance is a documented way to use several local machines as one inference system. A large downloadable model does not become practical merely because its files can be obtained. The weights must fit across available memory, and the machines must exchange intermediate results quickly enough that communication does not dominate generation.

Muse Spark Open has 816 billion total parameters, with 42 billion active for an individual token, according to the vendors' documentation. The active count helps explain its computation pattern but does not mean that only those parameters need to be stored somewhere. Apple's setup distributes the model across multiple high-memory systems.

Apple compares two ways of sharing the work. Pipeline parallelism divides the model's depth, assigning different layers to different machines. Tensor parallelism divides work within layers, requiring the machines to cooperate repeatedly during token generation. One resembles passing a document through different departments; the other resembles several people working on different parts of each page and comparing results before turning it.

The latter arrangement can improve speed, but only when the comparisons are cheap enough. Apple's recipe uses Thunderbolt 5 connections and remote direct memory access, which allows efficient exchanges between machines. The benchmark describes a mesh with two peer-to-peer cables per host pair. The communication design is part of the result, rather than a peripheral accessory.

In the published table, the two-machine pipeline configuration produces 15.74 tokens per second, and two-machine tensor parallelism reaches 24.99. Four-machine tensor parallelism produces the headline 36.73. These are vendor measurements at the stated context and precision, with a short measured generation, rather than a promise about every conversation length.

There is a subtle baseline issue. Apple's article says this model does not fit on a single Studio, so the baseline for this comparison is already a two-machine pipeline run. Reading a speedup chart as an improvement over one ordinary desktop would misstate both the memory requirement and the performance comparison.

Long context introduces another tradeoff. At 65,536 tokens, Apple finds that two-machine tensor parallelism can lag pipeline parallelism during prompt processing for this model. Generating the next token and processing a long initial prompt are different workloads. A system that feels quick after the first token can still take longer to produce that first token, as the prefill and decode lesson explains.

The memory description must also stay precise. These Macs use unified memory shared by processors, rather than a separate conventional graphics-memory pool. Meta's documented configurations supply a combined terabyte of unified memory. That is a stated deployment arrangement, not proof that a smaller pool cannot work under different settings. The dossier does not verify the compressed checkpoint's download size, and an earlier approximate file-size claim was explicitly cut.

The model card supplies additional access and licensing context. Download access involves a form, and the model's custom license includes conditions affecting some commercial model-serving businesses. Downloadable weights therefore do not mean unrestricted, anonymous access or identical rights under every business model. Ground Truth's previous release coverage provides the broader release context.

Community discussion on Hacker News challenges the everyday meaning of running this model at home. The objection is fair: two extremely high-memory machines or four workstations are a substantial local installation. That does not erase the engineering result. It clarifies who can use it and what local ownership actually costs in equipment and complexity.

Apple titles the work “Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio.” Distributed is the operative word. The result expands the documented deployment options for institutions and well-equipped local users; it does not turn an enormous model into a routine single-computer download. Independent reruns, long-context latency and sustained multi-user serving remain the useful next checks.

Primary source, verified: read the paper →

Ground Truth. (2026, October 17). Apple ran Muse Spark Open at 36.73 tokens a second across four Mac Studios. Ground Truth. https://groundtruth.day/news/apple-mlx-muse-spark-four-mac-benchmark.html

@misc{groundtruth:apple-mlx-muse-spark-four-mac-benchmark,
  title  = {Apple ran Muse Spark Open at 36.73 tokens a second across four Mac Studios},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/news/apple-mlx-muse-spark-four-mac-benchmark.html}
}

Topics: local-ai · inference · hardware · open-weights