mixture-of-experts
Looping half a model's layers twice beat making the model bigger News
A paper posted September 1, 2026 ran the first compute-matched test of looped mixture-of-experts transformers and found that re-running the middle half of the layers a second time saves compute at the frontier, with savings growing as budgets grow.
A 104 GB model now runs on a 48 GB Mac by streaming experts off the SSD News
slotstream, a single Swift binary released as a Show HN on September 1, 2026, runs the 104 GB Qwen3.8-Flash-Next mixture-of-experts model on Macs with a fraction of that memory by keeping a small trunk resident and reading expert weights off the SSD on demand -- about 12 tokens per second on a 48 GB machine.
Qwen put a 20-million-entry n-gram table inside a model News
Alibaba's Qwen released Qwen3.8-Flash-Next, a preview of the architecture behind Qwen4, whose headline idea is scaling parameters through a 20-million-entry table of word pairs and triples that can be offloaded off the GPU -- 51 billion parameters that never need to be computed, only looked up.
GLM-5.3-Flash was Ox Alpha, and it ran on Chinese chips News
Z.ai released GLM-5.3-Flash under an MIT licence and confirmed it is the anonymous \u201cOx Alpha\u201d model that topped OpenRouter for a week -- served, the company says, entirely on a cluster of Chinese AI accelerators at per-token cost comparable to NVIDIA hardware.
A 22-gigabyte local coder matched Opus on a 25-problem slice News
TielCoder, a 4-bit re-quantization of the Ornith-1.5 mixture-of-experts model, fits in 22.4 GB and fixed 12 of 25 recent real-world coding issues in its maintainer's tests, the same count as Claude Opus 4.6 at medium effort.
Ling-3.0 ships a hybrid-linear mixture of experts under a plain MIT license News
inclusionAI released the Ling-3.0 family on Hugging Face with a permissive MIT license, mixing linear and full attention in a fixed three-to-one ratio, and the smallest variant activates only 1.3 billion of its 7.9 billion parameters per token.
DeepSeek is selling a checkpoint it has not published News
DeepSeek's API now serves a model version named DeepSeek-V4-Pro-0813 and at least five commercial hosts resell it by that exact name, but the company has not published a matching dated weights page, and none of the resellers undercuts DeepSeek's own price.
A 753 billion parameter model ran on a single workstation GPU News
A serving system called FreeToken reports running frontier-scale sparse models on ordinary personal hardware, including a 753 billion parameter model on one workstation GPU and a 35 billion parameter model on a laptop with 8 gigabytes of video memory.
DeepSeek's new open model is 1.6 trillion parameters and runs 49 billion of them per token News
DeepSeek published DeepSeek-V4-Pro on Hugging Face with 1.6 trillion total parameters, 49 billion activated per token, and a one-million-token context window, making it the largest openly downloadable model of the current frontier wave.
NVIDIA built a 30B model for the boring half of agent work News
NVIDIA released Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model that activates only 3 billion parameters per token and is aimed at the tool calls and validation steps that consume most of a long-running agent's time.
Ant published Ling-3.0-flash's weights under plain MIT, with no rider attached News
Ant Group's InclusionAI lab published the full weights for its 124-billion-parameter Ling-3.0-flash model on Hugging Face this week under an unmodified MIT license, with no acceptable-use policy, revenue threshold, or branding requirement anywhere in the release.
The Cheap 284B Rig Is Really 768GB of Server Memory News
A builder running DeepSeek V4-Flash at 33 tokens a second on two RTX 3090s is holding about 6.6GB of weights per card and roughly 170GB per instance in system memory on a four-socket enterprise server, which is where the model actually lives.
Offloading and streaming: running a model bigger than your memory Lesson
Offloading and streaming let a machine run a model far larger than its memory by keeping only the parts needed right now in fast memory and fetching the rest from system RAM or disk on demand, trading speed for capacity.
Kimi K3 runs in 8 gigabytes of RAM, at 33 seconds per token News
A hand-written C engine generates text with Moonshot's 2.8-trillion-parameter Kimi K3 using a peak of 8.24 gigabytes of RAM and no GPU, by reading the model's four-bit experts directly off disk, at a rate of roughly one token every 33 seconds.
A 284-billion-parameter model with a 3-gigabyte working set, and a 96-gigabyte disk bill News
An open-source engine called Mference runs DeepSeek V4 Flash on a 24 GB Mac with an effective memory footprint of about 3 gigabytes by streaming each token's experts off the SSD, but the checkpoint still occupies 90 to 98 gigabytes of disk and the test ran at a 4,000-token context.
DeepSeek re-trained V4 Flash without touching the architecture and its coding-agent score went from 7 to 54 News
DeepSeek published new MIT-licensed weights for V4 Flash on July 31 that change only the post-training, lifting the model's score on a real-world software-engineering agent test from 7.3 to 54.4 out of 100.
Thinking Machines ships Inkling-Small's open weights - all 532 gigabytes of them News
Thinking Machines has published the full weights for Inkling-Small, a 276-billion-parameter sparse model that activates only 12 billion parameters per token and accepts text, images and audio, under an Apache 2.0 licence with a separate use policy attached.
LG shipped a 750-billion-parameter model and quietly dropped its restrictive licence News
LG AI Research released K-EXAONE 2.0, a 750-billion-parameter sparse model with 37 billion active, under Apache 2.0 - a break from the custom EXAONE licence that governed its previous releases.
A 26-billion-parameter model runs in 2GB of RAM by streaming experts off the SSD News
TurboFieldfare, an open-source Swift and Metal runtime, runs Gemma 4's 26-billion-parameter model on an 8GB MacBook Air by keeping only a 1.35GB core in memory and pulling each token's experts from disk as it needs them.
Kimi K3 topped a fullstack coding board at maximum effort News
Moonshot's open-weight Kimi K3, served at its highest reasoning setting, took first place on Code Arena's July 23 WebDev snapshot over Claude Fable 5 and GPT-5.6 Sol, though the live board has since moved it to second.
DeepSeek V4 Flash hits 32 tokens a second on one desktop News
A published benchmark shows DeepSeek's 284-billion-parameter V4 Flash generating 32 tokens per second entirely on one AMD Strix Halo machine, using aggressive quantization, speculative decoding and reduced expert routing.
Moonshot releases Kimi K3: a 2.8-trillion-parameter open-weight model, 1.56 terabytes on disk News
Moonshot AI published the full downloadable weights for Kimi K3, a 2.8-trillion-parameter model that uses only 104 billion parameters per token, handles text and images, and reads just over a million tokens of context.
Upstage Ships Solar Open 2, a 250-Billion-Parameter Model That Runs on Two GPUs News
Korea's Upstage released open weights for a 250-billion-parameter agent model that activates only 15 billion parameters per token and fits on two NVIDIA H200s when quantized, under a custom license with attribution requirements.
Three separate tricks dropped the hardware floor for local AI in one day News
A 26-billion-parameter Gemma model ran on an iPhone by streaming expert weights from storage, AMD shipped a sparse mixture-of-experts model activating 2.8 billion parameters per token, and a llama.cpp fork began saving conversation caches to disk - three unrelated attacks on three different bottlenecks.
Giving experts no memory cuts optimizer state from 50 gigabytes to 1.3 News
A single-author preprint shows that a mixture-of-experts model can drop momentum entirely for its expert layers, shrinking persistent optimizer state from 50.55 gigabytes to 1.29 with almost no effect on final quality.
Distributed training: how one model gets split across thousands of chips Lesson
No single chip can hold a frontier model, so training is split across thousands of them in four distinct ways -- by data, by layer, by tensor, and by expert -- and choosing the right mix is what separates a cluster running at a third of its potential from one running at a tenth.
Ant's Ling-3.0-flash goes live free: 124 billion parameters, 5 billion doing the work News
Ant released Ling-3.0-flash on July 23, a 124-billion-parameter model that activates only about 4% of itself per token, with a 256,000-token context and free access on OpenRouter and Vercel's AI Gateway.
Alibaba Ships Qwen3.6 as Open Weights, Betting on Efficiency Over Size News
Alibaba released its Qwen3.6 line under Apache 2.0, led by a 35-billion-parameter mixture-of-experts model that activates only about 3 billion parameters per token and targets agentic coding.
Basalt Labs' 'Best AI Model' Claim Collapses: Its Own Repo Admits Monolith-1.0 Was a Relabeled 7B Model News
Basalt Labs claimed its Monolith-1.0 model scored 99.4% on Humanity's Last Exam and was a 1.57-trillion-parameter system, but its own Hugging Face model card now says the publicly released model was an inflated version of the much smaller Qwen 2.5 7B Instruct, and the weights have been pulled.
Thinking Machines releases Inkling, now the top-ranked US open-weights model News
Thinking Machines Lab released Inkling, a 975-billion-parameter open-weights model under Apache 2.0 that Artificial Analysis ranks as the strongest open-weights model from any US lab, scoring 41 on its Intelligence Index.
What Ring-2.6-1T's model card actually says News
Ant Group's openly downloadable trillion-parameter model is real and MIT-licensed, but its benchmark claims are vendor-supplied and measured against a previous generation of rivals -- not the current frontier.
Tencent open-sources Hy3, a lean mixture-of-experts model that punches above its weight News
Tencent released Hy3 under the permissive Apache 2.0 license: a mixture-of-experts model with 295 billion total but only 21 billion active parameters and a 256K context window, which the company says competes with models five times its size.
China's GLM-5.2 Ships as the Top Open-Weight Model, Under MIT License News
Z.ai released GLM-5.2, a 753-billion-parameter model, as open weights under an MIT license, and an independent index ranks it the strongest open-weight model available, close behind the leading closed models at a fraction of the price.
Mistral releases a lean, open model built for formal math proofs News
Leanstral 1.5 is a free, open model specialized for writing machine-checked mathematical proofs, using a design that keeps only a small slice of itself active at a time.
A 35-billion-parameter agent that punches like a trillion-parameter model News
Shanghai AI Lab argues you can reach giant-model performance on long tasks not by adding parameters, but by training on much longer chains of real work.
Mixture of Experts: The Committee Inside a Giant Model Lesson
Why the biggest AI models are not really one big brain but a large team of specialists, only a few of whom wake up for any given word -- the trick that lets a model be huge and fast at the same time.
A small but elegant idea: putting 'experts' inside the attention layer News
Grouped Query Experts brings the mixture-of-experts trick into attention, activating only half a model's query heads per token while matching the full version -- at least at small scale.
A Classic Efficiency Trick Just Moved Into a New Part of the AI News
For years, the committee-of-specialists design that keeps big models fast lived in one layer of the network. A clean new result shows it works in the attention layer too, halving some of the work for free.
slotstream Tool
A single Swift binary that runs the 104 GB Qwen3.8-Flash-Next mixture-of-experts model on Apple Silicon Macs with far less memory, by streaming expert weights off the SSD. Speaks the Ollama and OpenAI chat APIs, so existing tools work unchanged. About 12 tokens per second on a 48 GB Mac; needs roughly 110 GB of free disk.
moondream 3.1 (9B-A2B) Tool
An open-weight vision-language model with 9B total but only 2B active parameters, offering native object detection, pointing, captioning, and segmentation at roughly the speed of a 2B dense model.
Xiaomi MiMo-V2.5-DFlash Tool
Xiaomi's official DFlash release on Hugging Face -- a 1-trillion-parameter mixture-of-experts model (42B active) under an MIT license, with FP4 quantization and parallel decoding for high inference throughput.
WASTE Tool
A C inference engine that streams only the experts a mixture-of-experts model actually activates directly off NVMe, using spare RAM as an expert cache. Its author reports running the full 2.78-trillion-parameter Kimi K3 on a 64 GB laptop at about half a token per second. An existence proof, not a chat app.
TurboFieldfare Tool
A Swift and Metal runtime that runs Gemma 4's 26B model on an 8GB MacBook Air by keeping a 1.35GB core resident and streaming the rest of the experts off the SSD. Ships as a Mac app, a CLI and an OpenAI-compatible local server.
SwiftLM Tool
An MLX-based runtime for Apple silicon whose --stream-experts mode reads mixture-of-experts weights straight off an NVMe SSD, letting a machine run models several times larger than its RAM.
Solar Open 2 Tool
Upstage's 250-billion-parameter mixture-of-experts model activates only 15 billion parameters per token and runs on two NVIDIA H200 GPUs once quantized, with a one-million-token context aimed at long multi-step agent work. Weights and a full technical report are public under a custom license requiring Solar-prefixed derivative names and Built with Solar attribution.
Ring-2.6-1T Tool
Ant Group's trillion-parameter mixture-of-experts reasoning model, activating roughly 63 billion parameters per token, with 128K context extendable to 256K. All checkpoints openly downloadable under the MIT license, with high and xhigh reasoning-effort settings that trade depth against speed and cost. Benchmark claims are vendor-supplied and measured against a previous generation of rivals.
Qwen3.6-35B-A3B Tool
Alibaba's open-weight agentic-coding model (35B total / 3B active, Apache 2.0) with 262K native context extensible toward 1M tokens, hybrid Gated-DeltaNet + MoE attention, thinking preservation across turns, and built-in tool use. Downloadable and self-hostable on common open serving stacks.
Ornith-1.5-35B-A3B Tool
The tool-using, agentic-coding mixture-of-experts base model behind TielCoder, with long context and a vision tower. Its multi-token-prediction head was re-uploaded in trained form on August 23, 2026.
Ornith-1.5 Tool
Three open coding and agentic models -- 397B and 35B mixture-of-experts plus a 9B dense model with a quantized Mobile build for phones. The 9B is single-GPU at roughly 19 GB with a 262,144-token context and OpenAI-compatible tool calling; the flagship matches Claude Opus 4.8 on terminal-coding benchmarks.
Noema Overfit Tool
Repackages compatible mixture-of-experts model files so shared weights stay resident in memory while expert weights stream from local storage on demand, letting phones load models far larger than their RAM. Experimental, and slower than a smaller fully-resident model on short prompts.
NVIDIA Nemotron 3.5 Lightning Tool
A 30B mixture-of-experts model with only 3B parameters active per token, trained for the high-volume half of agent work: tool calls, result validation and subagent delegation. NVFP4 and BF16 checkpoints, with weights, training data and recipes released under a permissive licence.
Mference Tool
Runs DeepSeek V4 Flash on Apple silicon by keeping the shared core, attention and cache resident while streaming each token's routed experts off the SSD. Publishes an unusually honest memory budget: about 3 GB working set, 90 to 98 GB on disk, tested at a 4,000-token context on a 24 GB Mac, with no quality parity test yet.
LvLLM Tool
A community inference runtime specialised in hybrid CPU and GPU execution of mixture-of-experts models, with NUMA-aware scheduling, expert weight management, and MXFP4 quantization kernels. Its DeepSeek V4 build publishes a working dual-RTX-3090 launch configuration at 22K context.
Ling-3.0-flash (free API) Tool
Ant's 124B-parameter mixture-of-experts model that activates only 5.1B parameters per token, with a 256K context and OpenAI- and Anthropic-compatible endpoints. Currently free on OpenRouter as inclusionai/ling-3.0-flash:free; aimed at long-horizon agent workflows and tool calling.
Ling-3.0 Tool
inclusionAI's hybrid-linear mixture-of-experts family under a plain MIT license, mixing three linear-attention blocks per full-attention block across 128 routed experts. The tiny variant holds 7.9B parameters and activates 1.3B per token; native BF16, FP8 and INT4 support is declared on the card.
Inkling Tool
Thinking Machines Lab's 975B-parameter mixture-of-experts model, released July 15 under Apache 2.0. Only ~41B parameters activate per token, it accepts text, image and audio input, and it handles up to 1M tokens of context. Artificial Analysis ranks it the top US open-weights model. Free to download, modify and use commercially -- but you will need serious hardware to run it.
Gemma 4 26B A4B Tool
Google's compute-efficient multimodal model with 25.2 billion total parameters but only 3.8 billion active per token, aimed at running usefully on hardware that cannot host a dense model of comparable capability.
DeepSeek-V4-Pro Tool
A downloadable 1.6-trillion-parameter mixture-of-experts model that activates 49 billion parameters per token, with a one-million-token context window under an MIT license. Serious server hardware required, but the weights are yours.
DeepSeek V4 Flash 0731 Tool
The current V4 Flash checkpoint, with weights, the DSpark draft head embedded, and the encoder file that reveals the reasoning-effort labels are prompt prefixes rather than a compute dial. The card also specifies the intended FP8 key-value cache and FP4 indexer cache serving recipe.