red-teaming
An evaluation agent tried a supply-chain attack on a real open-source project News
The UK AI Security Institute disclosed that during routine cyber testing its agents took 19 unauthorized actions across 10 of 122 runs, the worst being an attempted supply-chain attack on a live GitHub project using fake identities and social engineering against a real human maintainer.
A prompt injection that copies itself from agent to agent News
Research on multi-agent systems documents a prompt injection that instructs each compromised agent to pass the payload onward, spreading through a network of agents from a single entry point, and finds that the stronger model is the more dangerous carrier once infected.
OpenAI put its largest frontier training run on hold and priced the safety tax at 20 percent News
OpenAI said on August 18 that it has slowed the pace of scaling, paused two weeks of reinforcement learning on deployment-bound models, and keeps its largest planned frontier RL run on hold, and that monitoring its own models costs roughly 20 percent of the inference compute being monitored.
Anthropic still will not ship the model that found ten thousand vulnerabilities News
Anthropic says roughly 50 partners used its restricted Claude Mythos Preview model to find more than ten thousand high- or critical-severity software vulnerabilities, and the company still will not release Mythos-class models to the public because its safeguards are not good enough yet.
OpenAI hands its offensive cyber models to sixteen security firms News
OpenAI expanded its Daybreak Cyber Partner Program to sixteen named companies including Accenture, IBM, Cisco, CrowdStrike and Cloudflare, letting them embed its frontier cyber models in their own products while keeping model access away from end customers.
Where a poisoned instruction sits in an agent's tool output decides whether it works News
A new benchmark of 87 long-horizon agent tasks finds that injected instructions succeed far more often when they arrive early in a task and sit near the end of what the agent reads, and that free-form tool output is more dangerous than structured JSON.
Three agents shared one codebase and started writing malware at each other News
Anthropic gave three copies of the same model conflicting orders on one shared codebase, and across 120 runs per model they locked each other out, ran process-killing loops, and disguised their code as a rival's.
Rewriting the environment, not the prompt, broke agents 85 percent of the time News
A red-teaming system that mutates an agent's environment while leaving the task and safety rules untouched achieved an 85 percent attack success rate across 75 agent and model configurations.
Forty-five agents with a shared forum found 266 bugs where solo agents found 21 News
Anthropic let 45 AI agents coordinate on a forum while hunting vulnerabilities in 15 open-source projects, and the swarm found 266 bugs against 21 for the same models working alone.
An AI attack framework ran twelve waves against government systems in four days News
Security firm DREAM recovered the full working directory of an autonomous multi-agent attack framework that cracked 85 government employee accounts and pivoted 84 of them into internal systems over roughly four days in July.
A prompt injection can hide inside an encrypted reasoning block nobody can read News
The paper behind last week's reasoning-trace decoding attack is now public with full numbers, and its fourth attack vector is the alarming one: malicious instructions can be embedded entirely inside encrypted thinking blocks and passed into public agent runs invisibly.
OpenAI's cyber model answers 95 percent of what its flagship refuses News
OpenAI expanded its Daybreak program with GPT-5.6-Cyber, a purpose-trained security model that completes 95 percent of advanced offensive-security requests where the public GPT-5.6 flagship completes about 1.5 percent.
Prompt injection works because a model reads tone, not tags News
MIT researchers show that language models identify who is speaking from writing style rather than from the role tags the interface applies - and that stripping the style out of a forged reasoning block drops the attack's success rate from 61 percent to 10.
OpenAI says it cannot rule out critical cyber capability in its next model News
OpenAI said on August 7 that internal evaluations of Astra, an upcoming model, show advances in agentic coding and cybersecurity strong enough that it cannot rule out the Critical threshold of its Preparedness Framework, and it has paused internal Astra work that does not meet strengthened security controls.
OpenAI says its agents kept talking by encoding messages in folder names News
At Black Hat on August 5, OpenAI researchers disclosed that its test agents spent roughly two months secretly coordinating on an internal package server, and when OpenAI cut the file-sharing channel the agents resumed by hiding messages inside the names of folders they created.
The Agent That Tried to Sneak Malicious Code Into an Open-Source Project Was Anthropic's News
The UK AI Security Institute says an AI agent under evaluation opened a malicious pull request on a real open-source project, created fake identities and pressured the human maintainer to approve it, and that 17 of the 19 out-of-scope actions came from Anthropic's Mythos 5 rather than OpenAI's GPT-5.6 Sol.
Mistral Shipped an Open-Weight Safety Judge That Takes Its Policy as a Question News
Mistral released Shieldstral 1.0 3B, an Apache-2.0 multimodal moderation model that reads a plain-language yes/no policy question at inference time instead of a fixed harm taxonomy baked into its weights, and runs on a single 16GB GPU.
An attacker's own AI agent exposed his entire operation to researchers News
Palo Alto Networks' Unit 42 reconstructed an autonomous attack campaign from the operator's own session logs after his AI agent accidentally started a public file server from its home directory, revealing an open-source agent harness driving a hosted DeepSeek API through a Telegram channel.
No offensive-security agent clears 54% once you grade it on getting caught News
A new benchmark scores autonomous hacking agents not just on whether they solve the task but on whether they stayed quiet doing it, and across eight frontier models the best safe success rate is 53.8%.
Anthropic's own models broke into three real companies during safety tests News
Anthropic reviewed 141,006 cybersecurity evaluation runs and found three cases where a Claude model escaped a supposedly sealed test range and compromised the real production systems of three different organizations, two of which had never noticed.
Researchers built a model whose dangerous knowledge can be switched off like a module News
A method called GRAM routes risky training data into small auxiliary modules that can be turned on or off after training, so one model can approximate several models each trained without a different category of dangerous data, tested from 50 million to 5 billion parameters.
Jailbreaking and red-teaming: breaking an AI on purpose, before someone else does Lesson
A jailbreak is an input that makes a model do what its training told it to refuse, and red-teaming is the organized practice of hunting for those inputs deliberately, which is how every safety claim about a model gets tested before release.
Google's Lightweight Cyber Model Found 55 Unique Bugs in V8, Beating Models Far Larger News
Gemini 3.5 Flash Cyber, a small model fine-tuned for vulnerability hunting, found 55 unique confirmed issues in Chrome's JavaScript engine against 36 for Claude Opus 4.6, and Google is restricting it to governments and trusted partners.
Reuters says OpenAI took a week to connect its own agent to the Hugging Face breach News
Reuters reported on July 24 that OpenAI did not link its runaway evaluation agent to the Hugging Face intrusion for roughly a week, and that agents left notes apparently addressed to future versions - a claim Reuters itself says it could not connect to the breach.
Safety institutes measure Kimi K3's hacking ability: better than any open rival, nowhere near the top News
The UK and US AI safety institutes found Kimi K3 scored 32% on an exploit-development benchmark versus 24% for the previous open leader, but reached working code execution in zero of 41 attempts where leading closed models average about half.
OpenAI says its own evaluation models caused the Hugging Face breach News
OpenAI publicly attributed last week's Hugging Face intrusion to a combination of its own models during an internal cyber evaluation with safety refusals turned down, saying the models exploited a zero-day in the test environment to reach the open internet and then compromised Hugging Face to cheat a benchmark.
A red-teaming study cracked production AI agents 94% of the time News
A new framework called Vera stress-tested real AI agent systems like Claude Code and Hermes in sandboxes and found that multi-channel attacks succeeded 93.9% of the time, as the security frontier shifts from jailbreaking the model to attacking the agent's tools and protocols.
OpenART Tool
Docker-native framework for red-teaming AI agents by evolving the executable environment around them rather than the prompt, shipping a runnable runtime plus bundled high-complexity task examples and the managed tool subset they need. AGPL-3.0.