security
Encrypted inference: can a model answer a question it cannot read? Lesson
The two competing ways to run AI on data the server is not supposed to see: sealed hardware enclaves, which ship today and are fast, and homomorphic encryption, which is mathematically stronger and still far too slow.
Chat templates: the invisible tags that tell a model who is speaking Lesson
A language model never sees a conversation - it sees one long string, and small marker tokens are the only thing telling it which parts are your instructions, which are its own thoughts, and which are untrusted data from the outside world.
Differential privacy: a mathematical promise about what a model can reveal about you Lesson
Differential privacy is a formal guarantee that a system's output would look almost the same whether or not any single person's data was included, enforced by adding carefully calibrated noise. It is the only privacy protection for machine learning that comes with a provable bound rather than a hope.
Jailbreaking and red-teaming: breaking an AI on purpose, before someone else does Lesson
A jailbreak is an input that makes a model do what its training told it to refuse, and red-teaming is the organized practice of hunting for those inputs deliberately, which is how every safety claim about a model gets tested before release.
Model extraction attacks: stealing an AI through its own API Lesson
A model extraction attack tries to copy a machine-learning model you can only query, not download, by sending it many inputs and learning from its outputs. Depending on the goal, an attacker can clone the model's behavior, recover pieces of its internals, or reconstruct a rival model cheaply, which is exactly the fear driving today's AI 'distillation' disputes.
Capital One Open-Sources VulnHunter, an AI Agent That Hunts Security Bugs News
Capital One released VulnHunter, an open-source agentic AI security tool that reasons like an attacker, tries to disprove its own findings before reporting them, and has been run across thousands of the bank's own repositories.
An Autonomous AI Agent Breached Hugging Face's Servers News
Hugging Face disclosed the first documented intrusion of its production infrastructure driven end-to-end by an autonomous AI agent, and revealed its own defenders were locked out of commercial models by safety guardrails.
Adversarial Examples: Fooling AI With Changes You Can't See Lesson
Adversarial examples are inputs deliberately perturbed by tiny, often invisible amounts that cause an AI model to make confident mistakes, revealing that neural networks rely on fragile patterns rather than robust understanding.
Google's AI finds Android bugs faster than anyone can patch them News
Google has told phone makers it will drastically cut Android security backports because its own AI models are discovering vulnerabilities faster than its human teams can fix them.
Cursor's code-execution bug sat unpatched for seven months News
A flaw letting any Windows repository run arbitrary code the moment it is opened in the Cursor editor was reported in December, reproduced, acknowledged, and then met with silence across 197 shipped versions.
In a security-review bake-off, GPT-5.6 Sol caught every planted bug -- and no Anthropic model made the cost frontier News
A security firm tested 10 AI models on catching planted access-control bugs in pull requests and found GPT-5.6 Sol hit 100% recall at $0.70 per review, while no Anthropic model reached the cost-quality frontier for this specific task.
A researcher says xAI's coding tool uploads your whole repo -- secrets, unread files, and all News
An independent wire-level teardown found that xAI's Grok Build CLI uploads an entire code repository, including .env secrets and files the AI never read, to an xAI cloud bucket -- and the model-improvement opt-out does not stop it.
A field study documents Boko Haram using frontier AI for tactics and weapons News
A Cambridge research report based on interviews with 27 former Boko Haram members documents the group institutionalizing frontier AI -- using chatbots for battlefield tactics and weapons construction through dedicated units and internal training.
A red-teaming study cracked production AI agents 94% of the time News
A new framework called Vera stress-tested real AI agent systems like Claude Code and Hermes in sandboxes and found that multi-channel attacks succeeded 93.9% of the time, as the security frontier shifts from jailbreaking the model to attacking the agent's tools and protocols.
Claude Code Users Report Other People's Data Showing Up in Their Sessions News
Two new GitHub issues describe unexpected data appearing in Claude Code sessions, with one confirmed case of another user's live server credentials leaking in and being used without authorization.
Alibaba reportedly bans Claude Code over an alleged hidden backdoor News
Alibaba is reportedly banning Claude Code internally from July 10 after a researcher's analysis alleged the tool silently checked users' network and timezone settings against lists of Chinese firms; Anthropic says the mechanism was anti-abuse and is being removed.
Strix ships an open-source AI agent that hacks your app to find real vulnerabilities News
Strix is an open-source security tool whose autonomous AI agents dynamically find and exploit vulnerabilities in applications, generating working proof-of-concepts and plugging into CI/CD to block insecure code before it ships.
Claude Code was quietly fingerprinting requests through a hidden mark in the date News
A reverse-engineer found that Claude Code secretly changes tiny characters in the date it sends the model - a covert marker aimed at spotting resellers and copycats.
OpenAI showed off GPT-5.6 -- then handed the guest list to the US government News
Three new models, strong enough at hacking that OpenAI is only letting about twenty vetted partners in, at the government's request.
An open model from China beat Claude on a security test -- at a sixth of the cost News
Semgrep ran GLM 5.2 against Claude on a narrow vulnerability-finding task and the free, open-weight model came out ahead for far less money.
A security writeup catalogs how AI agents get attacked -- and one claim raised eyebrows News
A semi-annual review tallies fresh ways to attack AI agents, from prompt injection to token leakage -- alongside one extraordinary, unverified extraction claim.
DeepMind's plan for when an AI agent goes rogue: treat it like an insider threat News
Google DeepMind published a defense-in-depth roadmap that assumes an AI agent might misbehave and uses a trusted supervisor AI to watch it in real time.
Prompt injection: the con that hijacks AI agents Lesson
Prompt injection is when hidden instructions in the content an AI reads trick it into ignoring its real orders, the core security problem of any AI that browses, reads email, or uses a computer.
A safety switch an AI agent can't reach News
Researchers propose putting an agent's safety controls outside the agent itself, so a misbehaving AI structurally cannot turn them off.
Anthropic gives AI agents their own work accounts, not yours News
Anthropic's new 'agent identity' model lets Claude agents hold their own scoped accounts for tools like GitHub and Slack, tied to channels -- instead of borrowing a human employee's login.
Anthropic Gives Its AI Agents Their Own Logins, Not Yours News
As AI agents start working in teams alongside people, the old 'the bot acts as you' model breaks down. Anthropic's answer: give each agent its own scoped account in every system it touches.
A senator says a banned AI broke into nearly all NSA systems in hours News
New testimony reframes the Mythos export ban: a top general reportedly told a senator the model breached almost all classified systems in a red-team test, not in weeks but in hours.
OpenAI launches a security push at the exact moment its rival got banned News
Daybreak and 'Patch the Planet' position OpenAI as the responsible cyber-AI lab -- a defensive-security launch whose timing is the whole message.
A trust wobble hits AI coding tools: hidden reasoning and a runaway bug News
Two heated developer threads converge on one worry -- whether you can trust what an AI coding assistant shows you it's thinking, and what it quietly does to your machine.
VulnHunter Tool
Capital One's open-source agentic AI tool that analyzes source code from an attacker's perspective, tries to disprove its own findings before reporting them, and writes targeted fixes. Built for Claude Opus 4.8 in Claude Code; Apache 2.0.
Strix Tool
Open-source autonomous AI pentesting agents that dynamically find and exploit application vulnerabilities, generate working proof-of-concepts, and integrate with GitHub Actions and CI/CD to block insecure code on every pull request.
Semgrep Tool
Static-analysis security scanner that finds vulnerability classes like broken access control in real codebases, increasingly paired with AI models in its pipeline. Its public benchmark work this week is also a useful, honest reference for how well current models actually find security bugs.
OpenAI Codex Security (Daybreak) Tool
An in-IDE plugin from OpenAI's Daybreak initiative that finds, validates, and fixes software vulnerabilities, plus an open-source remediation program run with Trail of Bits and HackerOne.
OpenAI Codex Security Tool
Part of OpenAI's Daybreak program: an agent that builds an editable threat model from your code repository, finds realistic high-impact vulnerabilities, and drafts and tests patches in isolated environments.
NVIDIA SkillSpector Tool
A scanner that inspects agent skills for security problems before you run them -- a static safety check for the fast-growing agent-skill supply chain.
NOOA Tool
NVIDIA's open agent framework, contributed as the flagship technical artifact of the Open Secure AI Alliance. Its README is candid that it is research software and that its generated-code checks are not a containment boundary, so run agents in OS-level isolation.
Docker Sandboxes (sbx) Tool
Free command-line tool that runs coding agents inside disposable microVMs with their own kernel, filesystem, network, and private Docker engine, so an unsupervised agent cannot reach the host. Supports Claude Code, Codex, Copilot, Cursor, Gemini and others on macOS, Windows, and Linux; commercial use included at no cost.
Cloudflare OS Tool
Open-source platform where agents never hold credentials: a Gatekeeper does the OAuth and hands the agent a typed capability scoped to one resource, and every user-built app runs sandboxed with its own SQLite state. Runs locally on workerd for evaluation, or deploys into your own Cloudflare account.
Claude Tag (agent identity access model) Tool
Anthropic's product for putting Claude to work in shared team channels, now with an access model that gives each agent its own scoped accounts in the systems it touches -- GitHub, Slack, a data warehouse -- instead of borrowing an individual user's permissions, so every action is bounded and audited.
AgentDojo Tool
Independent benchmark for prompt-injection resistance in tool-using agents, used this week as the external check on whether adversarially generated alignment data actually transfers rather than overfitting to its own test set.
ADR Tool
Uber's runtime detector for coding agents, watching what agents actually do on developer machines rather than filtering prompts. Reported 206 credential exposures at 97.2 percent precision across 7,200 hosts, and ships with ADR-Bench, a 300-task benign-versus-malicious evaluation set.