guardrails
An AI scam agent got more people to comply than human operators did News
In a week-long blinded study, a language model running a romance-baiting script achieved 46 percent compliance against 18 percent for human operators, and commercial safety filters flagged none of the conversations.
An agent edited its own runtime for 161 days News
Ouroboros is a coding agent whose tools, prompts and core implementation change through reviewed commits that become the runtime for its next task, and its longest public deployment ran live for 161 days across seven surfaces.
Mistral Shipped an Open-Weight Safety Judge That Takes Its Policy as a Question News
Mistral released Shieldstral 1.0 3B, an Apache-2.0 multimodal moderation model that reads a plain-language yes/no policy question at inference time instead of a fixed harm taxonomy baked into its weights, and runs on a single 16GB GPU.
Guardrail models: the second AI that decides whether the first one's answer ships Lesson
A guardrail model is a small separate classifier that reads what a user sends an AI and what the AI sends back, then scores whether it violates a policy - so safety becomes a component the operator owns and can inspect, rather than a behaviour buried in the main model's weights.
Safety Guardrails Blocked a Security Team's Own Incident Analysis News
Hugging Face disclosed that commercial AI safety filters blocked its analysis of real attack code during an incident, so it ran the forensics on a self-hosted open-weight model instead.
GPT-5.6 'Sol' is both too strict and too leaky: benign bans on one side, jailbreaks on the other News
OpenAI's GPT-5.6 'Sol' is flagging users for benign defensive-security tasks like hardening their own websites while the UK AI Safety Institute found jailbreaks similar to Fable 5's - a capability-safety mismatch where a weak guardian model over- and under-triggers at once.
Developers are shipping their own kill switch for coding agents that run 'rm -rf' News
An open-source tool called Destructive Command Guard intercepts and blocks catastrophic commands like 'git reset --hard' and 'rm -rf' before AI coding agents can execute them, and has drawn more than 3,000 GitHub stars across nearly every major agent.
Alibaba's Qwen3Guard flags unsafe AI output token-by-token as it's being generated News
Alibaba released Qwen3Guard, its first open safety-guardrail model, whose streaming variant classifies an AI response for safety as each token is generated rather than after the fact -- and adds a 'Controversial' tier between Safe and Unsafe that apps can tune stricter or looser.
Shieldstral 1.0 3B Tool
Mistral's open-weight multimodal moderation model. You supply the policy as a plain-language yes/no question at inference time rather than retraining for a fixed harm taxonomy, and it returns one calibrated safety score per forward pass. Handles prompts, responses, prompt-response pairs, images and image-plus-text across twelve languages. Apache 2.0, runs on a single 16GB GPU via vLLM, Transformers or llama.cpp; recommended operating context is 32k tokens.