mixture-of-experts
DeepSeek's new open model is 1.6 trillion parameters and runs 49 billion of them per token News
DeepSeek published DeepSeek-V4-Pro on Hugging Face with 1.6 trillion total parameters, 49 billion activated per token, and a one-million-token context window, making it the largest openly downloadable model of the current frontier wave.
NVIDIA built a 30B model for the boring half of agent work News
NVIDIA released Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model that activates only 3 billion parameters per token and is aimed at the tool calls and validation steps that consume most of a long-running agent's time.
Ant published Ling-3.0-flash's weights under plain MIT, with no rider attached News
Ant Group's InclusionAI lab published the full weights for its 124-billion-parameter Ling-3.0-flash model on Hugging Face this week under an unmodified MIT license, with no acceptable-use policy, revenue threshold, or branding requirement anywhere in the release.
The Cheap 284B Rig Is Really 768GB of Server Memory News
A builder running DeepSeek V4-Flash at 33 tokens a second on two RTX 3090s is holding about 6.6GB of weights per card and roughly 170GB per instance in system memory on a four-socket enterprise server, which is where the model actually lives.
Offloading and streaming: running a model bigger than your memory Lesson
Offloading and streaming let a machine run a model far larger than its memory by keeping only the parts needed right now in fast memory and fetching the rest from system RAM or disk on demand, trading speed for capacity.
Kimi K3 runs in 8 gigabytes of RAM, at 33 seconds per token News
A hand-written C engine generates text with Moonshot's 2.8-trillion-parameter Kimi K3 using a peak of 8.24 gigabytes of RAM and no GPU, by reading the model's four-bit experts directly off disk, at a rate of roughly one token every 33 seconds.
A 284-billion-parameter model with a 3-gigabyte working set, and a 96-gigabyte disk bill News
An open-source engine called Mference runs DeepSeek V4 Flash on a 24 GB Mac with an effective memory footprint of about 3 gigabytes by streaming each token's experts off the SSD, but the checkpoint still occupies 90 to 98 gigabytes of disk and the test ran at a 4,000-token context.
DeepSeek re-trained V4 Flash without touching the architecture and its coding-agent score went from 7 to 54 News
DeepSeek published new MIT-licensed weights for V4 Flash on July 31 that change only the post-training, lifting the model's score on a real-world software-engineering agent test from 7.3 to 54.4 out of 100.
Thinking Machines ships Inkling-Small's open weights - all 532 gigabytes of them News
Thinking Machines has published the full weights for Inkling-Small, a 276-billion-parameter sparse model that activates only 12 billion parameters per token and accepts text, images and audio, under an Apache 2.0 licence with a separate use policy attached.
LG shipped a 750-billion-parameter model and quietly dropped its restrictive licence News
LG AI Research released K-EXAONE 2.0, a 750-billion-parameter sparse model with 37 billion active, under Apache 2.0 - a break from the custom EXAONE licence that governed its previous releases.
A 26-billion-parameter model runs in 2GB of RAM by streaming experts off the SSD News
TurboFieldfare, an open-source Swift and Metal runtime, runs Gemma 4's 26-billion-parameter model on an 8GB MacBook Air by keeping only a 1.35GB core in memory and pulling each token's experts from disk as it needs them.
Kimi K3 topped a fullstack coding board at maximum effort News
Moonshot's open-weight Kimi K3, served at its highest reasoning setting, took first place on Code Arena's July 23 WebDev snapshot over Claude Fable 5 and GPT-5.6 Sol, though the live board has since moved it to second.
DeepSeek V4 Flash hits 32 tokens a second on one desktop News
A published benchmark shows DeepSeek's 284-billion-parameter V4 Flash generating 32 tokens per second entirely on one AMD Strix Halo machine, using aggressive quantization, speculative decoding and reduced expert routing.
Moonshot releases Kimi K3: a 2.8-trillion-parameter open-weight model, 1.56 terabytes on disk News
Moonshot AI published the full downloadable weights for Kimi K3, a 2.8-trillion-parameter model that uses only 104 billion parameters per token, handles text and images, and reads just over a million tokens of context.
Upstage Ships Solar Open 2, a 250-Billion-Parameter Model That Runs on Two GPUs News
Korea's Upstage released open weights for a 250-billion-parameter agent model that activates only 15 billion parameters per token and fits on two NVIDIA H200s when quantized, under a custom license with attribution requirements.
Three separate tricks dropped the hardware floor for local AI in one day News
A 26-billion-parameter Gemma model ran on an iPhone by streaming expert weights from storage, AMD shipped a sparse mixture-of-experts model activating 2.8 billion parameters per token, and a llama.cpp fork began saving conversation caches to disk - three unrelated attacks on three different bottlenecks.
Giving experts no memory cuts optimizer state from 50 gigabytes to 1.3 News
A single-author preprint shows that a mixture-of-experts model can drop momentum entirely for its expert layers, shrinking persistent optimizer state from 50.55 gigabytes to 1.29 with almost no effect on final quality.
Distributed training: how one model gets split across thousands of chips Lesson
No single chip can hold a frontier model, so training is split across thousands of them in four distinct ways -- by data, by layer, by tensor, and by expert -- and choosing the right mix is what separates a cluster running at a third of its potential from one running at a tenth.
Ant's Ling-3.0-flash goes live free: 124 billion parameters, 5 billion doing the work News
Ant released Ling-3.0-flash on July 23, a 124-billion-parameter model that activates only about 4% of itself per token, with a 256,000-token context and free access on OpenRouter and Vercel's AI Gateway.
Alibaba Ships Qwen3.6 as Open Weights, Betting on Efficiency Over Size News
Alibaba released its Qwen3.6 line under Apache 2.0, led by a 35-billion-parameter mixture-of-experts model that activates only about 3 billion parameters per token and targets agentic coding.
Basalt Labs' 'Best AI Model' Claim Collapses: Its Own Repo Admits Monolith-1.0 Was a Relabeled 7B Model News
Basalt Labs claimed its Monolith-1.0 model scored 99.4% on Humanity's Last Exam and was a 1.57-trillion-parameter system, but its own Hugging Face model card now says the publicly released model was an inflated version of the much smaller Qwen 2.5 7B Instruct, and the weights have been pulled.
Thinking Machines releases Inkling, now the top-ranked US open-weights model News
Thinking Machines Lab released Inkling, a 975-billion-parameter open-weights model under Apache 2.0 that Artificial Analysis ranks as the strongest open-weights model from any US lab, scoring 41 on its Intelligence Index.
What Ring-2.6-1T's model card actually says News
Ant Group's openly downloadable trillion-parameter model is real and MIT-licensed, but its benchmark claims are vendor-supplied and measured against a previous generation of rivals -- not the current frontier.
Tencent open-sources Hy3, a lean mixture-of-experts model that punches above its weight News
Tencent released Hy3 under the permissive Apache 2.0 license: a mixture-of-experts model with 295 billion total but only 21 billion active parameters and a 256K context window, which the company says competes with models five times its size.
China's GLM-5.2 Ships as the Top Open-Weight Model, Under MIT License News
Z.ai released GLM-5.2, a 753-billion-parameter model, as open weights under an MIT license, and an independent index ranks it the strongest open-weight model available, close behind the leading closed models at a fraction of the price.
Mistral releases a lean, open model built for formal math proofs News
Leanstral 1.5 is a free, open model specialized for writing machine-checked mathematical proofs, using a design that keeps only a small slice of itself active at a time.
A 35-billion-parameter agent that punches like a trillion-parameter model News
Shanghai AI Lab argues you can reach giant-model performance on long tasks not by adding parameters, but by training on much longer chains of real work.
Mixture of Experts: The Committee Inside a Giant Model Lesson
Why the biggest AI models are not really one big brain but a large team of specialists, only a few of whom wake up for any given word -- the trick that lets a model be huge and fast at the same time.
A small but elegant idea: putting 'experts' inside the attention layer News
Grouped Query Experts brings the mixture-of-experts trick into attention, activating only half a model's query heads per token while matching the full version -- at least at small scale.
A Classic Efficiency Trick Just Moved Into a New Part of the AI News
For years, the committee-of-specialists design that keeps big models fast lived in one layer of the network. A clean new result shows it works in the attention layer too, halving some of the work for free.
moondream 3.1 (9B-A2B) Tool
An open-weight vision-language model with 9B total but only 2B active parameters, offering native object detection, pointing, captioning, and segmentation at roughly the speed of a 2B dense model.
Xiaomi MiMo-V2.5-DFlash Tool
Xiaomi's official DFlash release on Hugging Face -- a 1-trillion-parameter mixture-of-experts model (42B active) under an MIT license, with FP4 quantization and parallel decoding for high inference throughput.
WASTE Tool
A C inference engine that streams only the experts a mixture-of-experts model actually activates directly off NVMe, using spare RAM as an expert cache. Its author reports running the full 2.78-trillion-parameter Kimi K3 on a 64 GB laptop at about half a token per second. An existence proof, not a chat app.
TurboFieldfare Tool
A Swift and Metal runtime that runs Gemma 4's 26B model on an 8GB MacBook Air by keeping a 1.35GB core resident and streaming the rest of the experts off the SSD. Ships as a Mac app, a CLI and an OpenAI-compatible local server.
SwiftLM Tool
An MLX-based runtime for Apple silicon whose --stream-experts mode reads mixture-of-experts weights straight off an NVMe SSD, letting a machine run models several times larger than its RAM.
Solar Open 2 Tool
Upstage's 250-billion-parameter mixture-of-experts model activates only 15 billion parameters per token and runs on two NVIDIA H200 GPUs once quantized, with a one-million-token context aimed at long multi-step agent work. Weights and a full technical report are public under a custom license requiring Solar-prefixed derivative names and Built with Solar attribution.
Ring-2.6-1T Tool
Ant Group's trillion-parameter mixture-of-experts reasoning model, activating roughly 63 billion parameters per token, with 128K context extendable to 256K. All checkpoints openly downloadable under the MIT license, with high and xhigh reasoning-effort settings that trade depth against speed and cost. Benchmark claims are vendor-supplied and measured against a previous generation of rivals.
Qwen3.6-35B-A3B Tool
Alibaba's open-weight agentic-coding model (35B total / 3B active, Apache 2.0) with 262K native context extensible toward 1M tokens, hybrid Gated-DeltaNet + MoE attention, thinking preservation across turns, and built-in tool use. Downloadable and self-hostable on common open serving stacks.
Noema Overfit Tool
Repackages compatible mixture-of-experts model files so shared weights stay resident in memory while expert weights stream from local storage on demand, letting phones load models far larger than their RAM. Experimental, and slower than a smaller fully-resident model on short prompts.
NVIDIA Nemotron 3.5 Lightning Tool
A 30B mixture-of-experts model with only 3B parameters active per token, trained for the high-volume half of agent work: tool calls, result validation and subagent delegation. NVFP4 and BF16 checkpoints, with weights, training data and recipes released under a permissive licence.
Mference Tool
Runs DeepSeek V4 Flash on Apple silicon by keeping the shared core, attention and cache resident while streaming each token's routed experts off the SSD. Publishes an unusually honest memory budget: about 3 GB working set, 90 to 98 GB on disk, tested at a 4,000-token context on a 24 GB Mac, with no quality parity test yet.
LvLLM Tool
A community inference runtime specialised in hybrid CPU and GPU execution of mixture-of-experts models, with NUMA-aware scheduling, expert weight management, and MXFP4 quantization kernels. Its DeepSeek V4 build publishes a working dual-RTX-3090 launch configuration at 22K context.
Ling-3.0-flash (free API) Tool
Ant's 124B-parameter mixture-of-experts model that activates only 5.1B parameters per token, with a 256K context and OpenAI- and Anthropic-compatible endpoints. Currently free on OpenRouter as inclusionai/ling-3.0-flash:free; aimed at long-horizon agent workflows and tool calling.
Inkling Tool
Thinking Machines Lab's 975B-parameter mixture-of-experts model, released July 15 under Apache 2.0. Only ~41B parameters activate per token, it accepts text, image and audio input, and it handles up to 1M tokens of context. Artificial Analysis ranks it the top US open-weights model. Free to download, modify and use commercially -- but you will need serious hardware to run it.
Gemma 4 26B A4B Tool
Google's compute-efficient multimodal model with 25.2 billion total parameters but only 3.8 billion active per token, aimed at running usefully on hardware that cannot host a dense model of comparable capability.
DeepSeek-V4-Pro Tool
A downloadable 1.6-trillion-parameter mixture-of-experts model that activates 49 billion parameters per token, with a one-million-token context window under an MIT license. Serious server hardware required, but the weights are yours.
DeepSeek V4 Flash 0731 Tool
The current V4 Flash checkpoint, with weights, the DSpark draft head embedded, and the encoder file that reveals the reasoning-effort labels are prompt prefixes rather than a compute dial. The card also specifies the intended FP8 key-value cache and FP4 indexer cache serving recipe.