linear-attention
Ling-3.0 ships a hybrid-linear mixture of experts under a plain MIT license News
inclusionAI released the Ling-3.0 family on Hugging Face with a permissive MIT license, mixing linear and full attention in a fixed three-to-one ratio, and the smallest variant activates only 1.3 billion of its 7.9 billion parameters per token.
Ling-3.0 Tool
inclusionAI's hybrid-linear mixture-of-experts family under a plain MIT license, mixing three linear-attention blocks per full-attention block across 128 routed experts. The tiny variant holds 7.9B parameters and activates 1.3B per token; native BF16, FP8 and INT4 support is declared on the card.
FlashKDA Tool
Moonshot AI's MIT-licensed kernel implementation of Kimi Delta Attention, the linear-attention mechanism underneath Kimi K3, published ahead of the model weights themselves. Useful today for anyone building or serving bounded-state attention rather than a growing key-value cache.