multimodal
The open video model tops out at fifteen seconds, not twenty-six News
MiniMax's H3 weights have drawn 2,900 stars in four days, but the company's own repository caps a single generation at fifteen seconds and says the hosted component it left out is critical to output quality.
Mistral Shipped an Open-Weight Safety Judge That Takes Its Policy as a Question News
Mistral released Shieldstral 1.0 3B, an Apache-2.0 multimodal moderation model that reads a plain-language yes/no policy question at inference time instead of a fixed harm taxonomy baked into its weights, and runs on a single 16GB GPU.
MiniMax Shipped H3's Weights and Kept the Best Part Hosted News
MiniMax released the weights for its H3 video-and-audio generation model, and its own model card says the input-processing stage that is critical to output quality is not included in the release and the 2K output stage is not open-sourced at all.
Vision Transformers: what happens when you feed a picture to a language architecture Lesson
A Vision Transformer chops an image into a grid of small patches, treats each patch as a word, and runs the exact same Transformer machinery that powers language models over the resulting sequence. Google Research showed in 2020 that this beats purpose-built image networks once you train it on enough data, and it is why today's image, video and robot models all share one architecture.
Thinking Machines ships Inkling-Small's open weights - all 532 gigabytes of them News
Thinking Machines has published the full weights for Inkling-Small, a 276-billion-parameter sparse model that activates only 12 billion parameters per token and accepts text, images and audio, under an Apache 2.0 licence with a separate use policy attached.
Microsoft lets the video codec pick which pixels the model sees News
Microsoft's Mage-VL reuses a video file's own compression decisions to choose which image patches a vision model processes, cutting visual tokens by over 75% and reporting up to a 3.5x speedup over uniform frame sampling.
Humans score 96% on a new visual exam. The best model gets one in ten. News
A new benchmark called ActiveVision asks models to keep re-examining a picture while reasoning through it, and GPT-5.5 solved 9 of 85 problems at its highest reasoning setting while unaided humans averaged 81.7.
TimeLens2 teaches video AI to point to the exact seconds that answer a question News
Researchers released TimeLens2, an open-weight video model fine-tuned to answer a text query by returning the exact timestamp intervals in a video that contain the evidence, using a new distance-sensitive reward and a carefully curated dataset, with small versions reported to beat much larger open models on temporal grounding.
VideoChat3 Halves Video-Model Latency by Compressing Space and Time First News
VideoChat3, an open 4-billion-parameter video model, compresses frames across space and time before the language model, roughly halving latency versus a comparable model.
Boogu-Image-0.1: a fully open image model that claims to close in on closed systems for about $400K News
Boogu-Image-0.1 is a fully open-source unified image generation and editing model family whose researchers say a base model reaching near-frontier quality cost roughly $400,000 to train, arguing the closed-open gap is closing through data and pipeline quality rather than raw compute scale.
Thinking Machines releases Inkling, now the top-ranked US open-weights model News
Thinking Machines Lab released Inkling, a 975-billion-parameter open-weights model under Apache 2.0 that Artificial Analysis ranks as the strongest open-weights model from any US lab, scoring 41 on its Intelligence Index.
A benchmark audit finds most video-understanding tests can be aced without watching the video News
Video-Oasis audited video-understanding benchmarks and found about 55% of samples are solvable with no visual input at all - models exploit linguistic priors instead of watching motion, and once the shortcuts are removed, state-of-the-art systems barely beat random guessing.
How AI Turns Speech Into Text Lesson
Automatic speech recognition (ASR) converts spoken audio into written text by breaking sound into tiny slices, encoding them into features a model understands, and decoding those into words -- and its accuracy is measured by word error rate, the fraction of words it gets wrong.
Vidu S1 generates video you can steer with your voice in real time News
A new paper introduces Vidu S1, a video model that generates interactive 540p video at up to 42 frames per second on consumer GPUs and lets users reshape the scene on the fly with voice commands, without the drift that usually breaks long AI video.
Google's Gemma 4 is a small open multimodal family that skips the image encoder News
Google released Gemma 4, an open-weight model family from 2.3 to 31 billion parameters that natively handles vision and audio, including a 12-billion-parameter variant that ingests raw image and audio patches with no separate encoder.
Why AI Vision Benchmarks Reward Getting Close Instead of Getting It Right News
A new evaluation method argues standard image benchmarks hide model failures by averaging all details equally, and it exposes an 8-point perception gap between open and proprietary models that looser scoring conceals.
What Are Vision-Language-Action Models? Lesson
A vision-language-action (VLA) model is a single neural network that takes in camera images and a plain-language instruction and outputs the actual motor commands to carry it out, letting one model both understand a scene and physically act on it.
Orca proposes a single 'world latent space' to replace next-token, next-frame, and next-action prediction News
Researchers introduced Orca, a world foundation model that learns one unified latent space from multimodal signals and predicts the next world state rather than the next token or frame, outperforming similar-sized specialists on text, image, and action tasks.
Image generators can't plan. This one bolts on a brain that can. News
Qwen-Image-Agent wraps planning, reasoning, and memory around a text-to-image model so it can break a hard request into steps - and the local-AI crowd immediately asked whether it runs on a gaming GPU.
One model that listens, sees, and talks back in real time News
Wan-Streamer collapses the usual chain of separate speech and video tools into a single model built for live, two-way conversation.
This model's job is to make better training data for other models News
DataClaw0 turns the grind of cleaning and labeling training data into a learned skill -- a small model that refines raw, messy multimodal streams into dense, purpose-built lessons.
Wan Streamer v0.3 Tool
Streaming audio-visual interaction model that treats a video as a persistent world plus a time-varying event stream, running full-duplex real-time conversation at 640x368 / 25fps with roughly 550ms total interaction latency.
VideoChat3-4B Tool
A fully open 4B-parameter video multimodal model for general, long-form, and streaming video understanding, released with weights, training code, training strategy, and datasets.
Shieldstral 1.0 3B Tool
Mistral's open-weight multimodal moderation model. You supply the policy as a plain-language yes/no question at inference time rather than retraining for a fixed harm taxonomy, and it returns one calibrated safety score per forward pass. Handles prompts, responses, prompt-response pairs, images and image-plus-text across twelve languages. Apache 2.0, runs on a single 16GB GPU via vLLM, Transformers or llama.cpp; recommended operating context is 32k tokens.
Seed2.0 (ByteDance Seed) Tool
ByteDance's Seed2.0 family (Pro, Lite, Mini) of closed, API-hosted models aimed at long-tail knowledge and complex instruction-following, accessed through ByteDance's Volcano Engine (Ark) platform. Not open weights despite the academic-style model card.
Qwen3.8-Max Tool
Alibaba's new flagship multimodal model, live today as a paid API at $2 per million input tokens and $6 per million output tokens, with a one-million-token context, function calling, structured output, and prompt caching that drops repeated input to $0.25 per million. Weights are promised but not published.
Qwen-Image-2.0-Pro Tool
Alibaba's latest open image-generation model in the Qwen family, downloadable and runnable locally, part of a broad open-weight release wave that also refreshed the Qwen3.6 chat models.
Muse Image Tool
Meta's agentic image model, free for everyday creation inside Meta AI, Instagram Stories (US), and WhatsApp; it can search, write code, and self-refine rather than mapping a prompt straight to pixels, and stamps outputs with an invisible Content Seal watermark.
MiniMax-M3 Tool
A natively multimodal open model trained on text, image, and video from the first step, with a million-token context and a sparse-attention design built for speed; downloadable for self-hosting and also offered through MiniMax's own API and agent platform.
Meta Model API (Muse Spark 1.1) Tool
Meta's first paid, hosted model API, built around the Muse Spark 1.1 multimodal reasoning model -- a million-token context window with active context compaction, zero-shot tool and MCP support, and an OpenAI-compatible interface so existing code drops in with little more than an endpoint change.
Mage-VL Tool
Microsoft's codec-native multimodal model that reuses a video file's own bit allocation to pick visual tokens, reporting over 75% fewer tokens and up to 3.5x faster inference than uniform frame sampling. Works with H.264, HEVC and DCVC-RT.
Inkling-Small GGUF Tool
Quantized builds of Thinking Machines' newly released 276B/12B multimodal open-weight model, packaged for llama.cpp, LM Studio and Ollama so you do not have to download the 532GB original.
Inkling Tool
Thinking Machines Lab's 975B-parameter mixture-of-experts model, released July 15 under Apache 2.0. Only ~41B parameters activate per token, it accepts text, image and audio input, and it handles up to 1M tokens of context. Artificial Analysis ranks it the top US open-weights model. Free to download, modify and use commercially -- but you will need serious hardware to run it.
Gemma 4 26B A4B Tool
Google's compute-efficient multimodal model with 25.2 billion total parameters but only 3.8 billion active per token, aimed at running usefully on hardware that cannot host a dense model of comparable capability.
Gemma 4 Tool
Google's downloadable model family (2.3B-31B, dense and MoE) that natively handles text, vision, and audio, including a 12B encoder-free variant and a thinking mode.
Fara 1.5-27B Tool
Microsoft's MIT-licensed 27B multimodal agent that operates web browsers from screenshots alone, emitting clicks, typing, scrolling, and navigation, and trained to pause on ambiguity or unauthorized irreversible actions.
Boogu-Image 0.1 Tool
An open-source unified image understanding and generation model family (Base, Turbo, Edit, Edit-Turbo) with instruction-based editing and bilingual Chinese-English text rendering, trained for roughly $400K. Apache 2.0.