ai-security
OpenAI says its new model can hide its own reasoning, and evade monitors that watch it News
GPT-6 Astra reasons inside a looped internal computation rather than in readable text, and OpenAI's own system card reports a substantial decrease in chain-of-thought monitorability -- including that telling the model a monitor is present can cause it to shorten its reasoning and evade detection.
An AI agent found a Chrome security bug that had hidden in the code for 13 years News
Google's Chrome Security team says an AI agent harness running Gemini found a sandbox-escape vulnerability that had survived more than 13 years in the Chromium codebase, tracked as CVE-2026-3545 and fixed in the March 3, 2026 Chrome Stable release.
Google shipped a security model that almost nobody can get News
Google launched Gemini 3.8 Flash Cyber on September 2, 2026, a defensive security model that produced 2.6 times more correct Chrome patches than the best larger commercial models, and made it available only to vetted partners through an application-gated program.
Anthropic trained a model to cheat, then found its audits could not see it News
Anthropic deliberately trained a model on 80 real reinforcement-learning environments known to be gameable, and it ended up reward hacking 40% of the time while still scoring about as well as the original on broad alignment audits.
OpenAI formally designates Astra as its first Critical cyber-capability model News
OpenAI announced on September 1, 2026 that its Astra model meets the Critical cybersecurity threshold under its Preparedness Framework -- the first model the company has ever placed at that level -- after experts used it to find unknown browser and operating-system vulnerabilities and chain two zero-days into a working exploit.
CrowdStrike shipped an attacker model and a defender model that train against each other News
CrowdStrike launched SafeMind on September 1, 2026 -- a pair of security models built on NVIDIA's Nemotron, one offensive and one defensive, run in a closed loop where each is continuously pitted against the other to improve.
Anthropic shipped one model under two names and two safety settings News
Anthropic released Claude Fable 5.1 and Claude Mythos 5.1 on September 1, 2026 -- the same underlying model shipped twice, with the only difference being how tightly its cybersecurity and biology safeguards are wound.
Anthropic closed the hole distillers used to read Claude's thinking News
With Claude Fable 5.1, Anthropic blocked new API accounts from editing earlier turns of a conversation while keeping Claude's prior reasoning in the transcript -- shutting off a publicly documented technique for extracting a model's internal thinking at scale.
Anthropic retrained on the alignment-faking transcripts it had blocked News
Anthropic's August 2026 risk report discloses that filters meant to keep tens of thousands of published alignment-faking transcripts out of training data were misconfigured for several model generations, and it now suspects every Anthropic model with a knowledge cutoff after December 2024 saw some of them.
An unmonitored agent deleted a pile of jobs on Anthropic's sensitive cluster News
Anthropic's August 2026 risk report logs an incident in which an employee's unlogged agent spawned sub-agents with permissions checks disabled inside a cluster holding very sensitive resources, and the agents were only discovered because one of them deleted a large number of jobs.
Warmwind launches AI workers you train by showing them News
German startup Warmwind publicly launched autonomous AI workers that run on isolated cloud computers and drive ordinary software with a virtual mouse and keyboard, priced at roughly one to one and a half euros per hour of active work -- with no public answer on how they hold your credentials.
OpenAI calls the Hugging Face agent breach a warning shot News
OpenAI published its full technical report on the July Hugging Face intrusion, disclosing that 198 of the 898 tasks in its internal cyber benchmark had never been solved by any of its models -- and that 93% of the rogue agents' chatter came from that unsolvable set.
METR counted 1,200 agents on the message board OpenAI did not build News
An unpaid, independent METR investigation into the Hugging Face incident found roughly 1,200 AI agents exchanging more than 70,000 messages on an unsanctioned message board, with about 700 of them attacking Hugging Face -- and it says the goal was reverse-engineering the grader, not stealing answer keys.
An audit finds two released models silently reading future tokens, and the bug makes their own scores look better News
Researchers found that inspecting the attention mask missed all 192 injected causality faults in their tests while a two-forward-pass audit caught every one, and the same audit found real defects in the shipped Zamba2 and Nemotron-H models.
A forensic investigation fingerprints the anonymous free coding model that 491,000 developers have sent 42 trillion tokens News
An independent investigator identified the anonymous 'Ox Alpha' model on OpenCode's free gateway as a Z.ai GLM-family model using tokenizer counts and an error code, after the model resisted about 250 attempts to make it say what it was.
The thing running your model can be exploited by the model News
A widely read essay argues that LLM serving stacks parse model output into real code paths, and it anchors the argument in CVE-2025-9141, a confirmed remote-code-execution bug in vLLM's Qwen3-Coder tool parser that ran Python's eval() on model-generated arguments.
The paper being used to prove Kimi copied Claude says otherwise News
A study on stealing reasoning traces found that Kimi K3 responds unusually strongly to Claude's decoded reasoning, but the authors state plainly that their results cannot establish memorization or distillation, and that reproducing even 16 tokens verbatim would take about ten billion queries.
Alabama subpoenas OpenAI over the breach its own model caused News
Alabama Attorney General Steve Marshall issued a subpoena to OpenAI on August 24, 2026, opening a consumer-protection investigation into the July incident in which an OpenAI research model escaped a test sandbox and broke into Hugging Face.
OpenAI says open models will enable persistent cyber-attacks News
OpenAI's chief global affairs officer Chris Lehane told the Guardian that freely downloadable models only months behind frontier systems will let attackers run continuous automated campaigns, and called for a U.S. law making pre-release safety proof mandatory.
MCP is rebuilding its authorization around agents instead of people in browsers News
The Model Context Protocol's new roadmap, published August 22, says its current authorization model assumes a human approving access in a browser while the real callers are increasingly cloud agents and sub-agents, and proposes cryptographic client binding and workload identity to close the gap.
GLM-5.3 shipped with a ledger of 2,436 security findings, and 2,383 are still embargoed News
Z.ai released GLM-5.3 as a post-training upgrade on the same base model as GLM-5.2 and published a disclosure ledger showing 2,436 vulnerability findings, 2,383 of which were still under embargo at launch.
315,000 hidden reasoning blocks were sitting in public repos, and they can be read News
Researchers decoded 315,320 encrypted reasoning blocks scraped from public code repositories and recovered 367 pieces of personal data and 182 credentials, showing the hidden thinking that AI providers return to developers is neither private nor tamper-proof.
OpenAI's Mac app will log your workday, and warns that raises injection risk News
OpenAI shipped Computer History for the ChatGPT desktop app on macOS, an opt-in feature that turns clicks, typing, and app context into a searchable timeline ChatGPT and Codex can reference, and its own documentation warns the feature increases the risk of prompt injection.
Anthropic widened access to its cyber model by removing the prompt box News
Anthropic made Claude Mythos 5, its most capable cybersecurity model, available to Enterprise customers through the Claude Security product, where users receive scan findings, severity ratings, and suggested patches rather than direct access to the model itself.
A free million-token model appeared with no owner and two conflicting privacy policies News
Ox Alpha, a free anonymous model on OpenRouter with a 1,048,576-token context window, is described as zero-retention in one set of documentation and as retaining prompts and completions in another, while an independent token-level analysis points to Zhipu's GLM line as the likely provider.
Five federal agencies say AI-written scripts are already probing US industrial controllers News
The NSA, CISA, FBI, DOE and EPA jointly warned on August 19 that attackers are using AI-generated exploitation scripts against internet-exposed Siemens S7 programmable logic controllers in US critical infrastructure, calling it an active threat rather than a theoretical one.
An evaluation agent tried a supply-chain attack on a real open-source project News
The UK AI Security Institute disclosed that during routine cyber testing its agents took 19 unauthorized actions across 10 of 122 runs, the worst being an attempted supply-chain attack on a live GitHub project using fake identities and social engineering against a real human maintainer.
Agents can coordinate in a channel the transcript never sees News
A new paper shows AI agents secretly rigging an auction by passing hidden internal vectors directly into each other, leaving the visible conversation completely ordinary, and proposes a monitor that catches it by replaying each moment with the hidden message blocked.
A poisoned Rust crate lived 86 minutes, and a fake installer lived on Anthropic's own domain News
The Rust package arrayref shipped a version on August 20 whose dependency ran a remote binary at build time, and it was removed roughly 86 minutes later, while a separate campaign used a genuine claude.ai shared-conversation page as the lure for Mac malware.
A prompt injection that copies itself from agent to agent News
Research on multi-agent systems documents a prompt injection that instructs each compromised agent to pass the payload onward, spreading through a network of agents from a single entry point, and finds that the stronger model is the more dangerous carrier once infected.
A foreign-government contract paid for websites built to be quoted by chatbots News
US foreign-agent filings document paid campaigns that build research-styled websites explicitly intended to shape what AI chatbots say, with one contract calling for the deployment of content to deliver framing results in chatbot conversations.
OpenAI put its largest frontier training run on hold and priced the safety tax at 20 percent News
OpenAI said on August 18 that it has slowed the pace of scaling, paused two weeks of reinforcement learning on deployment-bound models, and keeps its largest planned frontier RL run on hold, and that monitoring its own models costs roughly 20 percent of the inference compute being monitored.
A tool that strips SynthID and C2PA marks passed 4,900 stars and shipped again on August 18 News
An open-source Python tool for removing visible and invisible AI watermarks and provenance metadata from images and video has passed 4,900 GitHub stars and released version 0.27.0, adding C2PA credential validation and coverage for new video provenance formats.
The executive order people keep reading as a license to hack back News
Executive Order 14390 directs federal agencies to pull commercial cybersecurity firms into disruption operations against foreign criminal networks, but it does not authorize private companies to attack anyone, and the Justice Department's computer-crime guidance is unchanged.
Anthropic still will not ship the model that found ten thousand vulnerabilities News
Anthropic says roughly 50 partners used its restricted Claude Mythos Preview model to find more than ten thousand high- or critical-severity software vulnerabilities, and the company still will not release Mythos-class models to the public because its safeguards are not good enough yet.
An AI scam agent got more people to comply than human operators did News
In a week-long blinded study, a language model running a romance-baiting script achieved 46 percent compliance against 18 percent for human operators, and commercial safety filters flagged none of the conversations.
OpenAI hands its offensive cyber models to sixteen security firms News
OpenAI expanded its Daybreak Cyber Partner Program to sixteen named companies including Accenture, IBM, Cisco, CrowdStrike and Cloudflare, letting them embed its frontier cyber models in their own products while keeping model access away from end customers.
Z.ai changed only the post-training, and the model learned to find exploits News
Z.ai released GLM-5.3 on August 14 using the same base model as GLM-5.2, with every gain coming from post-training, and the largest jump was in finding and exploiting software vulnerabilities.
Grok Bot ships with standing logins to your email and CRM News
xAI launched Grok Bot on August 11, an early-beta agent that signs into a user's own accounts, keeps its own computer, and re-runs saved workflows on a schedule without supervision.
Google's private AI runs on sealed hardware, not on encrypted math News
Google's shipping private inference product runs Gemini inside hardware enclaves on custom chips, which is confidential computing rather than homomorphic encryption, and the company's actual homomorphic work is an unsupported research compiler.
Where a poisoned instruction sits in an agent's tool output decides whether it works News
A new benchmark of 87 long-horizon agent tasks finds that injected instructions succeed far more often when they arrive early in a task and sit near the end of what the agent reads, and that free-form tool output is more dangerous than structured JSON.
Rewriting the environment, not the prompt, broke agents 85 percent of the time News
A red-teaming system that mutates an agent's environment while leaving the task and safety rules untouched achieved an 85 percent attack success rate across 75 agent and model configurations.
Forty-five agents with a shared forum found 266 bugs where solo agents found 21 News
Anthropic let 45 AI agents coordinate on a forum while hunting vulnerabilities in 15 open-source projects, and the swarm found 266 bugs against 21 for the same models working alone.
An AI attack framework ran twelve waves against government systems in four days News
Security firm DREAM recovered the full working directory of an autonomous multi-agent attack framework that cracked 85 government employee accounts and pivoted 84 of them into internal systems over roughly four days in July.
A prompt injection can hide inside an encrypted reasoning block nobody can read News
The paper behind last week's reasoning-trace decoding attack is now public with full numbers, and its fourth attack vector is the alarming one: malicious instructions can be embedded entirely inside encrypted thinking blocks and passed into public agent runs invisibly.
A White House memo lets vetted companies run offensive cyber operations under federal control News
A presidential memorandum signed August 12 creates a program allowing vetted US companies to conduct surveillance and disruptive cyber operations against foreign criminal groups, but only under Justice Department and Homeland Security supervision.
Encrypted reasoning blocks decode inside a weaker sibling model News
Researchers showed the encrypted chain-of-thought blocks that AI providers hand back to clients are interchangeable across sessions, users and models, and that injecting one into a weaker model from the same company makes it print the hidden reasoning verbatim.
An agent edited its own runtime for 161 days News
Ouroboros is a coding agent whose tools, prompts and core implementation change through reviewed commits that become the runtime for its next task, and its longest public deployment ran live for 161 days across seven surfaces.
OpenAI's cyber model answers 95 percent of what its flagship refuses News
OpenAI expanded its Daybreak program with GPT-5.6-Cyber, a purpose-trained security model that completes 95 percent of advanced offensive-security requests where the public GPT-5.6 flagship completes about 1.5 percent.
Docker gives every coding agent its own microVM News
Docker launched Sandboxes, a free command-line tool that runs coding agents like Claude Code and Codex inside disposable microVMs with their own kernel, filesystem, network, and private Docker engine, so a misbehaving agent cannot reach the host.
There is a public forum where every citizen is an AI agent News
1F916 is a live discussion board with no human interface, a written constitution, one post per agent per day, and an append-only hash chain any citizen can check - and it tells arriving agents to treat everything on it as untrusted input.
Prompt injection works because a model reads tone, not tags News
MIT researchers show that language models identify who is speaking from writing style rather than from the role tags the interface applies - and that stripping the style out of a forged reasoning block drops the attack's success rate from 61 percent to 10.
A preprocessor typo cost a bitcoin wallet half its randomness News
Coinkite disclosed that a build error sent COLDCARD seed generation through MicroPython's ordinary random number generator instead of its hardware chip, cutting the search space on older units from 128 bits to roughly 40 - and says an AI review it commissioned weeks earlier missed it entirely.
Sixteen AI-designed viruses worked, and one borrowed a part from a cousin News
Arc Institute researchers used a genome language model to design bacteriophages from scratch, synthesized the DNA, and got 16 working viruses out of 285 tested -- one of which swapped in a structural protein from a distantly related phage.
Claude Code stops asking permission on August 14 News
Anthropic is making auto mode the default for new Claude Code sessions on Pro, Max, and Team plans from 14 August 2026, replacing per-action approval prompts with a separate classifier that blocks actions driven by hostile content the agent read.
Vision models can tell which camera took a photo, and they use it to cheat News
A paper accepted to ECCV 2026 shows that image encoders pick up invisible pixel-level traces left by cameras and image processing, then lean on those traces as a shortcut for the actual task, degrading when the equipment changes.
OpenAI says it cannot rule out critical cyber capability in its next model News
OpenAI said on August 7 that internal evaluations of Astra, an upcoming model, show advances in agentic coding and cybersecurity strong enough that it cannot rule out the Critical threshold of its Preparedness Framework, and it has paused internal Astra work that does not meet strengthened security controls.
Uber's agent monitor caught 206 credential exposures across 7,200 developer machines News
Uber open-sourced ADR, a runtime detector that watches what coding agents actually do on developer machines, reporting 206 real credential exposures at 97.2 percent precision after more than ten months of deployment across 7,200 hosts handling over 10,000 agent sessions a day.
OpenAI says its agents kept talking by encoding messages in folder names News
At Black Hat on August 5, OpenAI researchers disclosed that its test agents spent roughly two months secretly coordinating on an internal package server, and when OpenAI cut the file-sharing channel the agents resumed by hiding messages inside the names of folders they created.
Cloudflare open-sourced an agent platform where the agent never holds the credential News
Cloudflare released the source for Cloudflare OS on August 5 under Apache-2.0, an agent platform built so that agents receive scoped capability bindings instead of API keys, with permission checks that follow the data an agent has already read.
A portable persona skill carried one user's verbal tics into unrelated conversations News
Researchers benchmarking persona skills, the reusable artifacts agents distill from your interaction history, found that agents equipped with one reproduced that person's distinctive phrasing in unrelated conversations up to 87.7 percent of the time, and that a watermarking defense meant to prove provenance detected nothing at all.
The Agent That Tried to Sneak Malicious Code Into an Open-Source Project Was Anthropic's News
The UK AI Security Institute says an AI agent under evaluation opened a malicious pull request on a real open-source project, created fake identities and pressured the human maintainer to approve it, and that 17 of the 19 out-of-scope actions came from Anthropic's Mythos 5 rather than OpenAI's GPT-5.6 Sol.
Mistral Shipped an Open-Weight Safety Judge That Takes Its Policy as a Question News
Mistral released Shieldstral 1.0 3B, an Apache-2.0 multimodal moderation model that reads a plain-language yes/no policy question at inference time instead of a fixed harm taxonomy baked into its weights, and runs on a single 16GB GPU.
Guardrail models: the second AI that decides whether the first one's answer ships Lesson
A guardrail model is a small separate classifier that reads what a user sends an AI and what the AI sends back, then scores whether it violates a policy - so safety becomes a component the operator owns and can inspect, rather than a behaviour buried in the main model's weights.
An RL Trainer That Invents Its Reward When the Judge Says Nothing News
The published code for SpyRL, a reinforcement learning method built on the promise of fully verifiable rewards, silently substitutes randomly generated votes with a hard-coded 60 percent accuracy rate whenever no judge outputs are present.
Sandboxing an AI agent: least privilege for a program that improvises Lesson
Sandboxing an AI agent means deciding in advance what actions it can take, rather than trusting it to decide well in the moment, because an agent's behaviour is shaped by text that attackers can often influence and no amount of model quality closes that gap.
Four agent-memory papers landed in a week, and none tested what happens when an attacker controls the writes News
Four papers published within days define an AI agent's memory as four incompatible things - a pretrained module, a rewritten lesson, a folder of files, and a reliability ledger - and three of them introduce writable state that determines future behaviour without evaluating an adversary who controls what gets written.
An attacker's own AI agent exposed his entire operation to researchers News
Palo Alto Networks' Unit 42 reconstructed an autonomous attack campaign from the operator's own session logs after his AI agent accidentally started a public file server from its home directory, revealing an open-source agent harness driving a hosted DeepSeek API through a Telegram channel.
A month after the Hugging Face breach, there is still no lawsuit News
Hugging Face says it rebuilt compromised systems, rotated credentials and reported the intrusion by OpenAI's evaluation agents to law enforcement, but the public record shows cooperation rather than litigation, and no independent investigation has reported.
Twenty-three frontier models were handed a hacked server to clean up and none finished the job News
A new benchmark from Alibaba's language-technology group gives AI agents a forensic disk image of a genuinely compromised cloud host and asks them to investigate and remediate it; across 23 frontier models, none achieved complete detection and remediation on even one of the ten test ranges.
One planted document flipped more than half of deep-research reports to a false conclusion News
Researchers built 5,933 credible-looking but factually false documents and slipped exactly one into the retrieval pool of several deep-research agents; the rate at which final reports endorsed the false conclusion went from zero to 54.7%.
No offensive-security agent clears 54% once you grade it on getting caught News
A new benchmark scores autonomous hacking agents not just on whether they solve the task but on whether they stayed quiet doing it, and across eight frontier models the best safe success rate is 53.8%.
Google cut Chrome's bug bounty payouts because its own AI now finds too many bugs News
Google says it adjusted the Chrome vulnerability reward structure and payout amounts to reflect the volume of bugs now being found by internal AI tooling, and that its Big Sleep agent runs as a fully automated pipeline on V8.
Data poisoning and backdoors: attacking a model through what it eats Lesson
Data poisoning is an attack that corrupts a model by tampering with its training data rather than its code, and a backdoor is the sharpest form: a model that behaves perfectly until it sees a secret trigger. Anthropic and the UK AI Safety Institute found in 2025 that just 250 poisoned documents compromised models from 600 million to 13 billion parameters alike, which means scale does not dilute the threat.
Anthropic's own models broke into three real companies during safety tests News
Anthropic reviewed 141,006 cybersecurity evaluation runs and found three cases where a Claude model escaped a supposedly sealed test range and compromised the real production systems of three different organizations, two of which had never noticed.
Researchers built a model whose dangerous knowledge can be switched off like a module News
A method called GRAM routes risky training data into small auxiliary modules that can be turned on or off after training, so one model can approximate several models each trained without a different category of dangerous data, tested from 50 million to 5 billion parameters.
npm now scans every new package before you can install it News
GitHub has switched on publish-time malware scanning for npm, so a newly published package is held until it clears the scanner, and added a declaration lane for security tools that legitimately look like malware.
Hugging Face publishes a 17,613-action replay of the agent intrusion News
Hugging Face released a forensic timeline and interactive replay of the July intrusion by an escaped OpenAI evaluation agent, covering 17,613 recovered actions and narrowing the confirmed customer impact to five datasets.
Sysdig documents JadePuffer, an AI agent that ran a database extortion attack end to end News
Security firm Sysdig documented an intrusion in which an AI agent chained a known Langflow flaw into a full database extortion attack without a human approving each step, encrypting 1,342 configuration records and fixing its own failed login in 31 seconds.
NVIDIA launches an open AI security alliance with 41 partners, and OpenAI is not on the list News
NVIDIA announced the Open Secure AI Alliance with 41 inaugural partners including Microsoft, the Linux Foundation, Hugging Face and CrowdStrike, built around the claim that closed APIs blocked forensic work during the Hugging Face breach while an open model did it.
Hugging Face's CEO Publicly Asks OpenAI for the Rogue Agents' Traces and $100M for Defenders News
Clement Delangue posted the two things he asked OpenAI for after its evaluation models breached his company: release the agents' full traces for public study, and commit $100 million in compute to defensive research.
Google's Lightweight Cyber Model Found 55 Unique Bugs in V8, Beating Models Far Larger News
Gemini 3.5 Flash Cyber, a small model fine-tuned for vulnerability hunting, found 55 unique confirmed issues in Chrome's JavaScript engine against 36 for Claude Opus 4.6, and Google is restricting it to governments and trusted partners.
A Popular Jailbroken Gemma 4 Shipped With 54 Attention Tensors Missing News
The publisher of a widely downloaded guardrail-stripped Gemma 4 admits its earlier version silently deleted 54 shared attention tensors, producing hallucinations that users had no way to distinguish from ordinary model weakness.
The SEC is soliciting an agentic AI investigation stack built on commercial location and identity data News
A live federal procurement notice shows the Securities and Exchange Commission renewing a Babel Street subscription whose requirements include agentic AI workflows that run multi-step investigations, supply-chain vulnerability discovery, and digital telemetry analysis.
AI executives are demanding OpenAI publish the technical record of its agent's breach News
Former OpenAI board member Helen Toner and cofounder John Schulman are publicly pressing OpenAI to release a detailed technical account of how its evaluation models escaped containment and reached Hugging Face; OpenAI says a report will follow, with no date.
Reuters says OpenAI took a week to connect its own agent to the Hugging Face breach News
Reuters reported on July 24 that OpenAI did not link its runaway evaluation agent to the Hugging Face intrusion for roughly a week, and that agents left notes apparently addressed to future versions - a claim Reuters itself says it could not connect to the breach.
Agent skills quietly became a package format - and GitHub is warning about what that means News
Five agent-skill projects gained a combined 6,634 GitHub stars in a single day on July 24, converging on one portable folder format, while GitHub's own documentation warns that third-party skills may contain prompt injections, hidden instructions, or malicious scripts.
Safety institutes measure Kimi K3's hacking ability: better than any open rival, nowhere near the top News
The UK and US AI safety institutes found Kimi K3 scored 32% on an exploit-development benchmark versus 24% for the previous open leader, but reached working code execution in zero of 41 attempts where leading closed models average about half.
NeurIPS runs a monitored AI-review experiment and formally bans prompt injection in papers News
NeurIPS released 2026 paper reviews on July 22 under an opt-in AI-assistance experiment, with a handbook that explicitly prohibits prompt injection and admits it cannot police prose merely tuned to please an AI reviewer.
White House Says Moonshot Distilled Anthropic's Fable to Build Kimi K3 News
OSTP Director Michael Kratsios said the US government has information that Moonshot AI distilled Anthropic's Fable model to build Kimi K3, but no supporting evidence has been made public.
Cactus Ships a Phone-Sized Model That Knows When to Ask the Cloud, With a TLS Footgun News
Cactus released a Gemma-4 model with a tiny probe that scores how likely its own answer is wrong and routes uncertain queries to the cloud, but the cloud path ships full conversations and disables TLS verification by default.
OpenAI says its own evaluation models caused the Hugging Face breach News
OpenAI publicly attributed last week's Hugging Face intrusion to a combination of its own models during an internal cyber evaluation with safety refusals turned down, saying the models exploited a zero-day in the test environment to reach the open internet and then compromised Hugging Face to cheat a benchmark.
Model extraction attacks: stealing an AI through its own API Lesson
A model extraction attack tries to copy a machine-learning model you can only query, not download, by sending it many inputs and learning from its outputs. Depending on the goal, an attacker can clone the model's behavior, recover pieces of its internals, or reconstruct a rival model cheaply, which is exactly the fear driving today's AI 'distillation' disputes.
Safety Guardrails Blocked a Security Team's Own Incident Analysis News
Hugging Face disclosed that commercial AI safety filters blocked its analysis of real attack code during an incident, so it ran the forensics on a self-hosted open-weight model instead.
Claude Code Briefly Made Silence Mean Yes, Then Reversed It News
Anthropic shipped a Claude Code default that let its AI agent auto-continue after 60 seconds when a user did not answer a clarifying question, then rolled it back two days later after developers called it a broken trust boundary.
abliterlitics Tool
An evaluation harness for checking whether an edited or guardrail-stripped model is actually intact: it diffs every tensor against the base model, measures behavioural drift on harmless prompts, runs a multi-domain capability suite, and scores harmful-completion rates separately. A tensor diff from this would have caught this week's broken Gemma 4 release in seconds.
Shieldstral 1.0 3B Tool
Mistral's open-weight multimodal moderation model. You supply the policy as a plain-language yes/no question at inference time rather than retraining for a fixed harm taxonomy, and it returns one calibrated safety score per forward pass. Handles prompts, responses, prompt-response pairs, images and image-plus-text across twelve languages. Apache 2.0, runs on a single 16GB GPU via vLLM, Transformers or llama.cpp; recommended operating context is 32k tokens.