local-llm
The open video model tops out at fifteen seconds, not twenty-six News
MiniMax's H3 weights have drawn 2,900 stars in four days, but the company's own repository caps a single generation at fifteen seconds and says the hosted component it left out is critical to output quality.
The harness, not the model, moved DeepSeek's score by twenty tasks News
Independent benchmarker Harrison 'sentdex' Kinsley re-ran DeepSeek V4 Flash 0731 on the same 89-task terminal benchmark under a different agent harness and watched it go from 44 solved to 64 solved, without changing the model.
The Full 2.8-Trillion-Parameter Kimi K3 Now Runs on Sixteen Desktop Boxes News
An operator has the complete Kimi K3 checkpoint running across sixteen GB10 mini-workstations wired through a single 400G switch, producing roughly 21 to 25 tokens per second for one user, on hardware with a verifiable floor around $57,200.
The 'Ternary' 20B Model Everyone Downloaded Today Ships as a Two-Bit Package News
DeepGrove's Maple-Preview is being shared as a 20-billion-parameter ternary model that fits in about 5 gigabytes, but its native release is a 40GB BF16 repository and the small footprint belongs to a separate two-bit MLX build with no published ternary training recipe.
A llama.cpp Patch Learns Which Experts to Keep in VRAM While You Type News
An open pull request to llama.cpp tracks which mixture-of-experts submodels get used most during inference and promotes them to GPU memory on the fly, roughly doubling decode speed on an 8GB card in the author's own tests - while slowing other models down.
The Cheap 284B Rig Is Really 768GB of Server Memory News
A builder running DeepSeek V4-Flash at 33 tokens a second on two RTX 3090s is holding about 6.6GB of weights per card and roughly 170GB per instance in system memory on a four-socket enterprise server, which is where the model actually lives.
Qwen3.8-Max Shipped as a Paid API, Not as Open Weights News
Alibaba put Qwen3.8-Max live as a hosted API at $2 per million input tokens and $6 per million output tokens, a fifth cheaper than the model it replaces, while the open weights it promised for Max and a 27B sibling have not shipped.
Kimi K3 is downloadable, but the floor to run it is eight datacenter GPUs News
Kimi K3's 1.56-terabyte checkpoint needs a single eight-GPU B300 or MI355X node as its practical minimum, and no version of llama.cpp can load it today, so open weights currently mean operator-scale rather than local.
How a model is stored: safetensors, GGUF, and why one model arrives in 96 files Lesson
A trained model is just a large dictionary of numbered arrays saved to disk, and the file format that holds them determines whether the model loads safely, loads fast, and loads at all on your hardware.
AMD Absorbs FastFlowLM Team to Build GPU-Free NPU Inference News
AMD announced on July 17, 2026 that the FastFlowLM team has joined its Artificial Intelligence Group to build out an NPU-first, GPU-free local inference stack for Ryzen AI laptops.
Bonsai puts a 27B model on your phone, and shows what breaks News
PrismML shrank a 27-billion-parameter model from 54 gigabytes to 3.9 by storing each weight as a single bit, and published the benchmark table showing tool calling and vision falling off a cliff.
Mesh LLM lets you run models too big for any single machine by splitting them across peers News
Mesh LLM, the top project on Hacker News this week, runs models larger than any one machine can hold by partitioning them across networked peers -- layers 0-15 on one node, 16-31 on another -- over a serverless peer-to-peer transport, exposing a standard OpenAI-compatible API on localhost.
Ternary-Bonsai-27B (GGUF) Tool
PrismML's ternary-weight 27B model in GGUF at ~7.2 GB deployed, with custom CUDA/Metal/CPU kernels. Expands local hardware reach, though agentic reliability is still limited per early tests.
Oh My Pi Tool
Full-featured terminal agent harness that an independent benchmarker measured lifting DeepSeek V4 Flash 0731 from 44 to 64 solved tasks on an 89-task terminal benchmark, with no change to the model. It costs several times the tokens per solve.
NInfer Tool
A focused inference engine that runs Qwen 3.6 models on one RTX 5090 with a 262,000-token context using an INT8 key-value cache, reporting roughly 188 tokens per second at 250,000 tokens of context. Methodology, seeds and limits are published openly.
Mesh LLM Tool
Runs models too big for one machine by splitting them across networked peers over serverless peer-to-peer transport; ~18 MB install, 40+ models up to 235B, OpenAI-compatible API on localhost.
Maple-Preview (2-bit MLX build) Tool
DeepGrove's 20B mixture-of-experts model with about 1B active parameters per token, packaged for Apple Silicon at roughly 5.3GB. The build uses affine two-bit group quantisation with four-bit embeddings and output head, and its loader packs ternary values into two-bit codes. Note that the native BF16 repository is about 40.4GB, and DeepGrove publishes no ternary training recipe or independent evaluation.
LvLLM Tool
A community inference runtime specialised in hybrid CPU and GPU execution of mixture-of-experts models, with NUMA-aware scheduling, expert weight management, and MXFP4 quantization kernels. Its DeepSeek V4 build publishes a working dual-RTX-3090 launch configuration at 22K context.
Kimi-K3-DSpark Tool
Inferact's draft model for Kimi K3. It proposes seven tokens at a time for K3 to verify and accept or discard, and its block-diffusion backbone shares K3's attention-cache layout so no second cache format is needed. This is the component behind the 21-25 tokens per second measured on a sixteen-node GB10 cluster running the full K3 checkpoint.
Google Gemma (open weights) Tool
Google's open-weight model family, light enough that developers are now embedding it directly into interactive apps - including a demo running Gemma inside the Godot game engine via Vulkan compute shaders, no Python server required.
Gemma Gem Tool
A Chrome extension that runs Gemma 4 E2B locally through WebGPU using an ONNX build with 4-bit weights, and gives the resulting agent page-reading, clicking, typing, screenshot and JavaScript tools. Worth knowing before you install: the widely quoted ~500MB is the cached download on disk, and the project's own estimates for GPU and system memory during inference are substantially higher and not benchmarked on real devices.
FastFlowLM Tool
An NPU-first, GPU-free inference runtime built exclusively for AMD Ryzen AI (XDNA) NPUs, targeting long-context local LLMs at low power on laptop-class hardware; the team just joined AMD, with open install guides for Ubuntu, Arch, and more.
Bonsai 27B Tool
PrismML's 1-bit and 1.58-bit builds of Qwen3.6 27B, compressing a 54 GB model to 3.9 GB (binary) or 5.9 GB (ternary) and running at roughly 11 tokens per second on an iPhone 17 Pro. The release ships an honest benchmark table showing the cost: instruction following, tool calling, and vision all degrade sharply, and the vendor states agentic coding is not a strong target of this release.