moderation
Mistral Shipped an Open-Weight Safety Judge That Takes Its Policy as a Question News
Mistral released Shieldstral 1.0 3B, an Apache-2.0 multimodal moderation model that reads a plain-language yes/no policy question at inference time instead of a fixed harm taxonomy baked into its weights, and runs on a single 16GB GPU.
Guardrail models: the second AI that decides whether the first one's answer ships Lesson
A guardrail model is a small separate classifier that reads what a user sends an AI and what the AI sends back, then scores whether it violates a policy - so safety becomes a component the operator owns and can inspect, rather than a behaviour buried in the main model's weights.
Alibaba's Qwen3Guard flags unsafe AI output token-by-token as it's being generated News
Alibaba released Qwen3Guard, its first open safety-guardrail model, whose streaming variant classifies an AI response for safety as each token is generated rather than after the fact -- and adds a 'Controversial' tier between Safe and Unsafe that apps can tune stricter or looser.
Shieldstral 1.0 3B Tool
Mistral's open-weight multimodal moderation model. You supply the policy as a plain-language yes/no question at inference time rather than retraining for a fixed harm taxonomy, and it returns one calibrated safety score per forward pass. Handles prompts, responses, prompt-response pairs, images and image-plus-text across twelve languages. Apache 2.0, runs on a single 16GB GPU via vLLM, Transformers or llama.cpp; recommended operating context is 32k tokens.
Qwen3Guard Tool
Alibaba's first open-weights safety-filter model, released under Apache 2.0 in three sizes, covering 119 languages, with a streaming variant that can flag unsafe text token by token as it is generated.