gguf
How a model is stored: safetensors, GGUF, and why one model arrives in 96 files Lesson
A trained model is just a large dictionary of numbered arrays saved to disk, and the file format that holds them determines whether the model loads safely, loads fast, and loads at all on your hardware.
Unsloth DeepSeek-V4-Flash-0731 GGUF Tool
Published quantisations of DeepSeek's 671-billion-parameter Flash model, ranging from roughly 91 GB at two bits to 162 GB at eight. The card is also the clearest available statement of what hardware each tier actually needs.
Ternary-Bonsai-27B (GGUF) Tool
PrismML's ternary-weight 27B model in GGUF at ~7.2 GB deployed, with custom CUDA/Metal/CPU kernels. Expands local hardware reach, though agentic reliability is still limited per early tests.
Muse Glimmer 30B GGUF (Unsloth) Tool
Community-packaged quantized builds of Muse Glimmer that fit under 20 GB, with setup instructions for llama.cpp, Ollama, vLLM, and SGLang. This is the practical path if you want the model running on a single 24 GB consumer graphics card rather than compiling the full-precision weights yourself.
KoboldCpp v1.118 Tool
Single-binary local model server that shipped its own fix for multi-turn DeepSeek V4 Flash prompt-processing problems on the same day as the llama.cpp fix. Useful if you want a working DS4 setup without building anything.
Inkling-Small GGUF Tool
Quantized builds of Thinking Machines' newly released 276B/12B multimodal open-weight model, packaged for llama.cpp, LM Studio and Ollama so you do not have to download the 532GB original.
Gemma-4 12B Coder (GGUF) Tool
A fine-tuned, locally-runnable version of Google's Gemma-4 model specialized for programming tasks, packaged in a format that runs efficiently on everyday consumer hardware.
DeepSeek-V4-Flash-0731 GGUF (Unsloth) Tool
Community quantizations of the new MIT-licensed DeepSeek weights in GGUF form, running from roughly 83GB at aggressive low precision to about 162GB at 8-bit. Usable on high-memory workstations and multi-GPU rigs, not on a laptop.