on-device
Liquid Shipped a 2.6B Tool-Calling Model and Told You Not to Code With It News
Liquid AI released LFM2.5-2.6B, a small model whose stack is mostly cheap convolution layers with attention rationed to eight of thirty, built for tool use and long context - and whose own model card says not to use it for agentic coding or knowledge-heavy work.
An $8 Microcontroller Now Runs a 28.9-Million-Parameter Language Model News
A developer fit a 28.9-million-parameter model onto an ESP32-S3 with 512KB of fast memory by keeping most of the model in flash storage and reading only about 450 bytes of it per word generated.
A complete text-to-speech system now fits in 9.4 million parameters News
Inflect-Micro-v2 packs an entire English speech synthesis stack, including the waveform decoder, into 9,356,513 parameters that run locally with no external vocoder or API.
Three separate tricks dropped the hardware floor for local AI in one day News
A 26-billion-parameter Gemma model ran on an iPhone by streaming expert weights from storage, AMD shipped a sparse mixture-of-experts model activating 2.8 billion parameters per token, and a llama.cpp fork began saving conversation caches to disk - three unrelated attacks on three different bottlenecks.
Bonsai puts a 27B model on your phone, and shows what breaks News
PrismML shrank a 27-billion-parameter model from 54 gigabytes to 3.9 by storing each weight as a single bit, and published the benchmark table showing tool calling and vision falling off a cliff.
A 4B model on your device nearly matched a 72B one - by copying its memories News
Researchers distilled a large AI agent's memory skills onto a compact 4-billion-parameter model, raising its success rate on a household-task benchmark from 4% to 78% while running three times faster than its 72B teacher.
Program-as-Weights compiles a plain-English spec into a tiny model you run on a laptop News
A new method called Program-as-Weights uses a 4-billion-parameter 'compiler' to turn a natural-language task description into a small weight file that a frozen 0.6B model runs, matching a 32B model's quality while using about one-fiftieth the memory and running at 30 tokens a second on a MacBook.
Samsung's Trick Makes a Tiny 4B Agent Nearly Match a Model 18 Times Bigger News
Samsung R&D UK and Queen Mary University of London published DuoMem, a distillation method that took a 4-billion-parameter agent from a 4.3 percent task-success rate to 77.9 percent, nearly matching a 72-billion-parameter teacher model's 87.1 percent.
A tiny image-editing AI now runs entirely inside your web browser News
Moebius is a small inpainting model claiming far-larger-model quality, and a developer ported it to run on your own machine in a browser tab -- no server, no upload.
Ternary-Bonsai-27B (GGUF) Tool
PrismML's ternary-weight 27B model in GGUF at ~7.2 GB deployed, with custom CUDA/Metal/CPU kernels. Expands local hardware reach, though agentic reliability is still limited per early tests.
Program-as-Weights demo Tool
A public demo and code for compiling natural-language task specs into tiny neural artifacts that run locally on a frozen small model, matching much larger models on narrow fuzzy tasks.
Program-as-Weights Tool
Turns a plain-English task spec into a small weight file that a frozen 0.6B model runs locally -- matching a 32B model's quality at roughly one-fiftieth the memory and about 30 tokens/sec on a MacBook M3. Open repo and site for compiling cheap, offline 'fuzzy' text programs.
Noema Overfit Tool
Repackages compatible mixture-of-experts model files so shared weights stay resident in memory while expert weights stream from local storage on demand, letting phones load models far larger than their RAM. Experimental, and slower than a smaller fully-resident model on short prompts.
MiniCPM5-1B Tool
OpenBMB's dense 1B local model with Think and No-Think modes, trained with SFT, RL, and on-policy distillation. Designed for on-device and edge deployment.
LFM2.5-2.6B Tool
Liquid AI's 2.7B tool-calling model with a 128k context, built as 22 short-convolution layers plus 8 grouped-query-attention layers so most token mixing stays local and cache-friendly. Post-trained inside real agent harnesses for tool use, extraction, retrieval and long-context workflows. The model card explicitly recommends against agentic coding and knowledge-heavy tasks, and it always enters a reasoning mode before answering.
Inflect-Micro-v2 Tool
A complete English speech synthesis stack in 9,356,513 parameters, waveform decoder included, producing 24 kHz mono audio locally with no external vocoder or API. One fixed synthetic male voice, no cloning, flatter prosody than large systems - but it runs anywhere.
Gemma-4 WebGPU Kernels Tool
A demo running Google's Gemma-4 model directly inside a web browser using your device's graphics hardware — private, on-device AI with no server and no data leaving your machine.
Cactus Hybrid (Gemma-4 E2B) Tool
A phone-sized Gemma-4 checkpoint with an attached error probe that scores how likely each answer is wrong and routes low-confidence queries to a cloud model; weights and runtime are public (set CACTUS_CLOUD_STRICT_SSL before using the cloud path).
Bonsai 27B (GGUF) Tool
PrismML's roughly 27.8-billion-parameter Qwen-derived model trained with 1-bit binary or 1.58-bit ternary weights end to end, which the company says fits in about 4 GB and runs on phone-class hardware. Performance figures are vendor-reported and not independently replicated.
Bonsai 27B Tool
PrismML's 1-bit and 1.58-bit builds of Qwen3.6 27B, compressing a 54 GB model to 3.9 GB (binary) or 5.9 GB (ternary) and running at roughly 11 tokens per second on an iPhone 17 Pro. The release ships an honest benchmark table showing the cost: instruction following, tool calling, and vision all degrade sharply, and the vendor states agentic coding is not a strong target of this release.