sglang
Kimi K3 is downloadable, but the floor to run it is eight datacenter GPUs News
Kimi K3's 1.56-terabyte checkpoint needs a single eight-GPU B300 or MI355X node as its practical minimum, and no version of llama.cpp can load it today, so open weights currently mean operator-scale rather than local.
SGLang K2 Horizon Cookbook Tool
IFM's validated serving configuration for the K2 Horizon family, with measured H200 latency and throughput for every model size. Covers the tensor-parallel setup and the router numerics override that preserves checkpoint behaviour -- the difference between the model running and the model running correctly.
Ling-3.0-flash Tool
inclusionAI's 124B mixture-of-experts model with about 5.1B parameters activated per token. Sparse routing genuinely cuts per-token compute, but this is a server-class artifact, not a laptop one: the BF16 repository is roughly 255GB and the official serving path calls for custom SGLang or vLLM forks with tensor parallelism across four GPUs.