on-device-ai
Google releases ML Drift for graphics-processor inference on devices News
Google’s Apache-2.0 ML Drift engine supports four graphics interfaces for on-device inference and can run standalone or accelerate LiteRT.
Google releases EmbeddingGemma 2 for local search across words, pictures, and sound News
Google’s EmbeddingGemma 2 maps text and media into a shared search space, with a quantized full configuration measured at about 567 MB of active phone RAM.
Cactus Ships a Phone-Sized Model That Knows When to Ask the Cloud, With a TLS Footgun News
Cactus released a Gemma-4 model with a tiny probe that scores how likely its own answer is wrong and routes uncertain queries to the cloud, but the cloud path ships full conversations and disables TLS verification by default.
Apple's on-device SpeechAnalyzer beats Whisper Small using about a third of the compute News
Apple's new on-device SpeechAnalyzer API cut errors roughly fourfold versus its legacy recognizer and outperformed OpenAI's Whisper Small while using about a third of the compute, in an English-only single-vendor benchmark.
This method compiles plain English into a tiny model that rivals a 32B giant News
A new paper, Program-as-Weights, uses a 4B compiler to turn natural-language specs into small neural adapters that let a 0.6B model match a 32B model on fuzzy tasks, running at about 30 tokens per second on a laptop.
ML Drift Tool
Google’s Apache-2.0 graphics-compute engine for on-device inference, available standalone or as LiteRT’s accelerator.
EmbeddingGemma 2 Tool
Released modular embedding model for local retrieval across text, code, images, audio, and sampled video; applications supply the index and interface.
Apple SpeechAnalyzer Tool
Apple's on-device speech-to-text API that cut errors roughly fourfold over the legacy recognizer and beat Whisper Small using about a third of the compute - private, local transcription with no cloud round-trip.