Learn · Beginner

How a model is stored: safetensors, GGUF, and why one model arrives in 96 files

A trained AI model, stripped of all mystique, is a very large dictionary: thousands of named arrays of numbers saved to disk. The file format wrapped around that dictionary decides three practical things - whether loading the file can execute malicious code, how fast the weights reach your GPU, and whether your runtime can reconstruct the model at all. Two formats dominate today: safetensors for training and datacenter serving, and GGUF for running models on ordinary hardware.

This sounds like plumbing. It is plumbing. It is also the reason a model can be fully public, fully downloaded, and completely unrunnable.

What is actually in the file

Every layer of a neural network holds parameters - the numbers adjusted during training. A transformer layer has attention projection matrices, feed-forward matrices, normalization scales. Each gets a name like model.layers.42.mlp.down_proj.weight, and each is a rectangular block of numbers with a known shape and numeric type.

A checkpoint file is that whole name-to-array mapping, serialized. Nothing more. There is no code in it, no logic, no architecture - just labelled numbers. The architecture lives separately, in a small configuration file and in the runtime's source code, which knows what to do with an array called down_proj.

Why safetensors exists

The original way to save these dictionaries in PyTorch used Python's pickle module, which serializes arbitrary Python objects - and reconstructing a pickled object means executing code embedded in the file. Downloading a stranger's model checkpoint was therefore equivalent to running a stranger's program. Several proof-of-concept malicious checkpoints demonstrated the risk.

Safetensors, developed at Hugging Face, fixes this by being deliberately stupid. The file is a small JSON header describing every array's name, data type, shape and byte offsets, followed by one contiguous block of raw bytes. There is nothing to execute. A loader reads the header, then maps the bytes straight into memory.

The security fix bought a speed win too. Because every array's exact byte range is known upfront, the operating system can memory-map the file and hand pages to the GPU without parsing or copying. That is why safetensors was a named contribution in NVIDIA's Open Secure AI Alliance launch: a safe model format is genuine security infrastructure, not just convenience.

Why models come in 96 pieces

Kimi K3's release is 1.56 terabytes across 96 shard files plus an index. That sharding is not arbitrary. A single terabyte-scale file cannot be resumed reliably after a dropped connection, cannot be verified piecewise, and cannot be uploaded by most tooling. Sharding turns one fragile transfer into 96 recoverable ones.

The index file is the important half. It maps every array name to the shard containing it, so a loader can open the index, decide which shards it needs, and fetch only those - useful when you are loading a mixture-of-experts model and want to stream expert weights on demand. It also makes completeness checkable: when the K3 weights landed, an independent developer verified all 2,760 tensors were present and correctly mapped, which is only possible because the index says what should be there. A previous release that shipped with 54 tensors missing is why anyone bothers to check.

Why GGUF is different

GGUF, from the ggml project behind llama.cpp, solves a different problem. Safetensors assumes the runtime already knows the architecture. GGUF assumes it does not, and packs the metadata into the file: vocabulary, tokenizer settings, architecture name, layer counts, rotary embedding parameters, and the quantization scheme used for each array.

That self-contained design is what makes single-file local inference work. You download one file, point a program at it, and it runs. GGUF also natively supports mixed quantization - keeping sensitive layers at higher precision while compressing the bulk - which is how a model trained in 16-bit precision fits into a fraction of the memory on a laptop.

The trap worth understanding

Here is the thing that catches people, and it played out publicly with Kimi K3.

Converting weights to GGUF is a numeric operation. Teaching a runtime to use those weights is a code operation. The two are completely separate, and only the first is easy.

Community conversions of K3 appeared within hours, shrinking the checkpoint from 1,453.8 GiB to 938.6 GiB. The converter's own notes state the result is not loadable: llama.cpp has no registered K3 architecture, its expert-count limit sits below K3's 896 experts, and the model's novel attention-residual and latent-routing components have no implementation. The file is valid. The reader does not exist.

So "a GGUF exists" never means "you can run it." That distinction is the single most useful thing to take from this lesson, and it explains why the practical hardware floor for K3 remains eight datacenter GPUs despite the weights being free.

The one-line version

Safetensors is a safe, fast box of numbers for people with a rack. GGUF is a self-describing box of compressed numbers for people with a laptop. Sharding makes terabyte models transferable. And none of it matters until the runtime knows what the numbers mean - which is why open weights and open access are not the same thing.

Ground Truth. (2026, July 27). How a model is stored: safetensors, GGUF, and why one model arrives in 96 files. Ground Truth. https://groundtruth.day/learn/model-file-formats-safetensors-and-gguf.html

@misc{groundtruth:model-file-formats-safetensors-and-gguf,
  title  = {How a model is stored: safetensors, GGUF, and why one model arrives in 96 files},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/learn/model-file-formats-safetensors-and-gguf.html}
}

Topics: fundamentals · model-formats · safetensors · gguf · quantization · local-llm · infrastructure