experimental
Everything on Ground Truth tagged “experimental” — 1 item.
WASTE Tool
A C inference engine that streams only the experts a mixture-of-experts model actually activates directly off NVMe, using spare RAM as an expert cache. Its author reports running the full 2.78-trillion-parameter Kimi K3 on a 64 GB laptop at about half a token per second. An existence proof, not a chat app.