distributed
Everything on Ground Truth tagged “distributed” — 2 items.
Distributed training: how one model gets split across thousands of chips Lesson
No single chip can hold a frontier model, so training is split across thousands of them in four distinct ways -- by data, by layer, by tensor, and by expert -- and choosing the right mix is what separates a cluster running at a third of its potential from one running at a tenth.
MindSpeed-LLM Tool
Huawei's official large-model training toolkit for Ascend NPUs, covering distributed layouts, checkpoint conversion and supported model families. Worth reading its support table honestly -- DeepSeekV4-Flash is currently marked Prototype, its label for not-fully-validated features.