News · 2026-09-03

An open lab shipped six models at once, and released the checkpoints and data recipes too

IFM released K2 Horizon on September 3, 2026 as six models rather than one -- 375B-A23B, 36B-A4B, 32B, 7B, 3.7B and 0.9B -- all under Apache 2.0, all sharing the same core architecture, vocabulary, training methodology and evaluation infrastructure. Alongside the weights, IFM published intermediate checkpoints, data-construction recipes, training code, configurations and fine-grained training logs, making this one of the most reproducible releases at this scale.

Key facts

• Six models from 375 billion down to 0.9 billion parameters, all Apache 2.0 for models and code, spanning edge devices to enterprise deployment.
• The 36B-A4B uses Mixture-of-Value Attention: 36 billion total parameters, about 4 billion active per token, with native 524,288-token context.
• Download sizes from the repositories' own file listings: 74.9 GB for the bf16 36B-A4B, 48.4 GB for the FP8 variant. IFM's serving recipe is validated on 2x H200 (141 GB cards).
• Primary source: IFM's K2 Horizon launch post.

Most open releases hand you a finished object and nothing else. You get weights, a license, a benchmark table, and no way to know why any decision was made. K2 Horizon is structured as the opposite argument. IFM describes the six models as a connected development tree rather than isolated drops: they share interfaces, deployment tooling and evaluation infrastructure, with a smaller vocabulary used only for the 0.9B model. Post-training is described as a lineage, not a set of independent runs.

The architectural novelty has a name: MoVA, or Mixture-of-Value Attention. Standard mixture-of-experts models put sparsity in the feed-forward layers -- of many parallel sub-networks, only a few fire per token. MoVA pushes that same idea into the attention mechanism. The result on the 36B-A4B is 36 billion parameters stored but roughly 4 billion doing work on any given token. The practical translation: you pay 36 billion parameters' worth of memory and roughly 4 billion parameters' worth of compute per token. It is the difference between owning a full toolbox and carrying three tools up the ladder.

Context length is the other headline. The FP8 repository states native 524,288-token context from mid-training onward -- not a post-hoc extension bolted on at the end, which is how many long-context claims are manufactured. That 512K figure also holds on the 375B-A23B, 32B, 7B and 3.7B models; the 0.9B is the exception at 128K. Our explainer on context windows covers why "trained with it" and "extended to it" produce very different behaviour at the far end of the window.

For anyone planning to actually download this, the numbers from the repositories' own file listings: the bf16 36B-A4B is 74.9 GB spread across 48 safetensors shards. The FP8 variant is 48.4 GB. The GGUF repository currently ships a single 74.9 GB bf16 file. On hardware, IFM does not publish a minimum GPU memory requirement, but it does state that its serving recipe is validated on two H200 GPUs -- 141 GB of memory each -- using tensor parallelism across both. Of the FP8 build, the repository says: "The FP8 model performs closely in line with the original BF16 model on our evaluations, while reducing memory footprint and enabling faster inference on FP8-capable hardware." That is a vendor evaluating its own quantization, but it is at least a stated claim rather than an implied one.

The release contents are what distinguishes this from a weight drop. Intermediate checkpoints let researchers study how capabilities emerge during training rather than inspecting only the finished model -- the difference between a photograph and a time-lapse. Data-construction recipes and mixture compositions let someone contest the training choices. Fine-grained logs let someone diagnose them. Very few labs at this scale publish any of the three.

The honest caveat, and it is a significant one: you probably cannot run this locally yet. IFM's own GGUF repository states that K2 Horizon support in upstream llama.cpp is still in progress and directs users to the project's fork. The FP8, 32B, 7B and 0.9B model pages show no hosted inference provider. An Apache 2.0 fleet that requires a forked runtime is a promise with a dependency attached, and the history of new architectures reaching mainstream local tooling is measured in weeks or months, not days -- as our explainer on model file formats explains, a new attention variant means real work in every downstream runtime.

Why it matters: the open-model conversation has been stuck on parameter counts and benchmark tables for two years. A release that ships the training trajectory, the data recipes and six sizes cut from the same tree is an argument that reproducibility is the thing worth competing on. Whether MoVA generalises is a question the field can now actually investigate, because IFM published enough for someone else to check.

Primary source, verified: read the paper →

Ground Truth. (2026, September 3). An open lab shipped six models at once, and released the checkpoints and data recipes too. Ground Truth. https://groundtruth.day/news/an-open-lab-shipped-six-models-that-share-one-training-tree.html

@misc{groundtruth:an-open-lab-shipped-six-models-that-share-one-training-tree,
  title  = {An open lab shipped six models at once, and released the checkpoints and data recipes too},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {sep},
  url    = {https://groundtruth.day/news/an-open-lab-shipped-six-models-that-share-one-training-tree.html}
}

Topics: open-weight-models · ifm · k2-horizon · attention · reproducibility · local-llm