AI Text Watermarks Tested and Attacked, Why Agents Act Differently Through MCP and the Command Line, and a One-Line Fix for Multi-Token Training
AI in Detail -- Daily News & Research. Case Western researchers tested the open-source SynthID-Text watermark, the method Gemini uses and Claude's watermark is based on: on prose its effect was no bigger than rerunning with a new random seed, on code it cost a little correctness on one model and none on the other, and detection on code stayed near chance. Their argument: the deployed versions cannot be checked from outside, and they list what audits would need. Then an Illinois Tech and UConn attack that evades seven watermark schemes using only repeated queries, with its large query cost and small test model as caveats. Next, a controlled study showing that agents given identical actions behave differently as MCP tool calls versus command-line commands, plus a fix that locks the intended action before formatting it. Finally, Soft-MTP from MIT, a one-line change that removes a noisy part of multi-token training for small gains up to 1.2 billion parameters.
Listen on Spotify · Watch on YouTube · Pocket Casts
Watermarks Without Verification: AI Text Watermarking After the EU AI Act
Eris: Researchers at Case Western Reserve University tested the open-source version of the text watermark Google uses in Gemini, the method Anthropic says its new Claude watermark is based on. On ordinary writing, turning the watermark on changed the output no more than simply rerunning the model with a different random seed. On code, there was a small cost on one model and none measurable on the other, but the detector could barely find the watermark in code at all. Their larger argument is that none of this tells us how the watermark in Claude itself behaves, because nobody outside the company can test it.
Vestra: The background is a European law. On August second, a section of the EU AI Act took effect requiring generative AI providers to mark what their systems produce so it can be detected as machine-made. Days later, Anthropic disclosed that every Claude model released after that date watermarks all of its generated text, based on Google's SynthID-Text method, switched on by default, with no way for users to turn it off. The paper notes Anthropic also said a public detection tool would follow; Anthropic's own help page currently describes detection as a private preview for eligible organizations.
Eris: And the reaction was loud. The paper sorts what users claimed into four positions. The watermark degrades quality, especially code. It secretly encodes who you are. It's trivially easy to remove, so it's useless. And it's impossible to remove, so you can't escape it. Those last two can't both be true, but the authors found them circulating in the same discussions. A GitHub tool claiming to strip the watermark collected over ten thousand stars in a week, without published evidence that it worked. Which raises the basic question: how does a watermark hide in plain text without adding anything to it?
Vestra: It works at the moment the model picks each word. At every step the model has a set of reasonable candidates. A secret key, combined with the words just written, assigns each candidate a pseudorandom score. SynthID draws several candidates from the model's normal probabilities and runs a small tournament between them, where the candidate with the higher score advances. Nothing is inserted, and in the configuration vendors report using, any single response is distributed exactly as it would have been without the watermark. The detector, holding the same key, recomputes those scores for every word in a passage and checks whether the average is suspiciously high. Over a long passage that adds up to evidence. Where the model barely had a choice, as in code or a short factual answer, there's almost nothing for the key to decide, and the signal stays faint.
Eris: How do you measure the quality cost of a watermark you can't switch off in the real product?
Vestra: You run the open version yourself, with a control built in. They used two open models, Google's Gemma 2 with nine billion parameters and Meta's Llama 3.1 with eight billion. For five hundred open-ended questions, each model produced one watermarked answer and one unwatermarked answer from the same random seed, plus a second unwatermarked answer from a different seed. The first pair isolates the watermark. The second pair shows how much two perfectly ordinary runs differ. They did the same for three hundred sixty-four programming problems, ten attempts each, and they fixed in advance how small a difference would count as no difference.
Eris: Did the watermark change the prose at all?
Vestra: Not measurably. Every check came out inside the bounds they set beforehand. A separate model's measure of how predictable the text was barely moved, and on Llama it moved less with the watermark than it did between the two ordinary runs. An AI judge asked to pick the better answer chose the watermarked one about half the time, which is what you'd expect if it couldn't tell. On code, Gemma passed its tests just as often with the watermark on. Llama did slightly worse: a few more programs out of every hundred failed, a small cost, but one that held up statistically.
Eris: Could the detector actually find the watermark in those same answers?
Vestra: The detector was tuned so that only one in a hundred unwatermarked passages would be wrongly flagged. At that setting, on prose of roughly a hundred and fifty words, it caught the watermark about four times in ten. Doubling the length raised that to a little over half. Code never got there. The same detector with the same keys stayed barely better than a coin flip at every length, and between a quarter and nearly a third of the code samples came out character for character identical with and without the watermark. Most code answers were also short. On Gemma, fewer than one code answer in a hundred was long enough to reach the shorter prose length they tested.
Eris: Which turns the loudest complaint around. People said the watermark would ruin their code. What the measurement shows is that the watermark mostly fails to stick to code, so the cost lands on the company that is legally required to make output detectable, much more than on the user.
Vestra: With a large limit attached. Everything here measures the method, the open implementation on two open models, in English, with a model rather than people judging quality. It does not measure what Anthropic deployed. Nobody outside knows how strongly Claude applies the watermark, whether it switches the watermark off for highly predictable text like code, or how the detector's threshold is set. The authors point out that a company has every reason to turn it off where it adds little and costs correctness, and none of the deployments disclose whether they do.
Eris: What about the claim that it secretly identifies users?
Vestra: No outside experiment can settle that one. The published method carries no information about the user. But if a provider gave each customer or organization its own key, it could identify where a passage came from just by checking which key matches, without changing anything in the text. From the outside, a shared-key system and a per-customer-key system look identical. Anthropic says the watermark carries no identifying information, and critics say it might. Neither side can prove its case without an auditor looking at how keys are actually assigned.
Eris: There's also the person who writes something themselves and asks Claude to tidy it up.
Vestra: Anthropic acknowledges that case. The mark records that the model was involved, not who wrote the ideas, so human writing edited by the model can carry it. How often that gets flagged, and how often fully human text gets flagged across millions of checks, are exactly the numbers no provider has published. Google's own released detector returns three answers instead of two, watermarked, unwatermarked, or uncertain, which already admits the decision isn't a clean yes or no.
Eris: So what do the authors actually want?
Vestra: Five things. The vendor should release matched outputs from the same model with the watermark on and off, plus a detector that reports a score instead of a bare verdict. It should disclose the configuration, including any cutoff below which text goes unmarked, which they argue reveals nothing about the key. Then three things no single company can supply: accredited auditors who can inspect key management, a shared testing protocol so results from different schemes can be compared, and a common way to check text against every provider's detector. Article fifty requires marking and detection, but it doesn't require any of that verification.
Eris: Their summary of the August backlash is that a public offered promises it had no way to test will assume the worst. The contradictory complaints and the unproven removal tools are what that situation produces.
A Unified Attack Framework against Statistical Generative Watermarks for Large Language Models
Eris: A second watermark paper, from researchers at the Illinois Institute of Technology and the University of Connecticut, goes after the removal question directly. They built one attack that works against seven published text-watermarking schemes, including SynthID, the method behind Gemini's watermark and the basis of Claude's. The attacker never sees the secret key or the model's internals. They only need to be able to ask the watermarked model for text, over and over. In their tests, watermarked text that went through the attack was caught almost never, at about the rate the detector wrongly flags ordinary unwatermarked text.
Vestra: The weakness they exploit comes from how all these schemes are built. A watermark nudges the model toward words the secret key favors, which means it also pushes other words down. The authors prove that, for every scheme they cover, the words the watermark suppresses the most, relative to what the model would normally say, are exactly the words that score lowest when the detector checks for the watermark. They call that score-ratio alignment. So if you can figure out which words are being suppressed, you know which words will erase the evidence, without ever learning the key.
Eris: How do you find the suppressed words from the outside?
Vestra: By sampling, one word at a time. At each position, the attack asks the watermarked model for just the next word, two hundred fifty-six times, and counts how often each candidate comes back. Separately, it runs an ordinary unwatermarked model of its own, which they call a surrogate, to get what the probabilities should look like without a watermark, and it keeps that model's forty most likely candidates. Then it picks the candidate the watermarked model is producing least often compared to the surrogate's expectation, appends it, and moves to the next position. Because it only ever chooses among words a normal model already considers likely, the text stays fluent.
Eris: And the results across all seven schemes?
Vestra: The watermarks were applied to a small open model, Meta's OPT with one point three billion parameters, and they tried surrogates of three sizes up to a thirteen billion parameter Llama. Across three kinds of text, web pages, news-style stories and question-and-answer explanations, the attacked passages were caught at most five times in a hundred, with the detector tuned to wrongly flag only one ordinary passage in a hundred. Often it caught none. An automatic fluency score stayed low, meaning the text still read naturally to a measuring model.
Eris: Could a better-designed watermark close that hole?
Vestra: Not within the current design, they argue. A classic result in statistics says the most powerful possible test for telling two distributions apart compares how likely the text is under each one. If a watermark detector is built to be as powerful as possible, its scores line up with exactly that likelihood comparison, and that alignment is what the attack uses. So within this whole family of designs, maximum detection power and resistance to this attacker pull against each other. They suggest two escapes, hiding a secret random offset in each word's score, or combining several independent keys, but they leave both untested.
Eris: What should make us cautious about applying this to Claude or Gemini?
Vestra: Several things in the setup. The cost is large. At two hundred fifty-six queries per word piece, a passage of two hundred fifty-six word pieces, roughly two hundred words, takes about sixty-five thousand requests, each asking for a single next piece. Against a paid service, that is expensive and easy to notice. The watermarked model was small and several years old, not a deployed product. For SynthID they used its simpler averaging detector, because the stronger trained detector requires private data they didn't have. And while they describe the attacked text as comparable in quality to unwatermarked text, their results tables report fluency for the attacked text without an unwatermarked baseline next to it.
Eris: So the two watermark papers fit together without proving the same thing. The Case Western paper says an outside party can't currently test whether the deployed watermark survives attacks, because there's no detector that returns a score. This paper supplies one of the attacks such a test should include. Neither shows that Claude's private implementation has been defeated.
Matched Capabilities, Different Behaviors: A Controlled Study of MCP and CLI Interfaces for Language Agents
Eris: Researchers at the Hong Kong Polytechnic University and the Chinese Academy of Sciences gave AI agents exactly the same set of actions in two different formats, and the format alone changed how well the agents did. In one case, the same model completed household tasks about a third as often when its actions were offered as structured tool calls instead of typed commands. On a shopping task, the same switch went the other way, with slightly better results and far fewer tokens spent.
Vestra: The two formats are the main ways agents use tools today. One is the Model Context Protocol, usually called MCP. Each action is presented as a named tool with a structured list of arguments, and the model answers with a small structured object, the tool's name and the argument values. The other is a command-line interface, where the model writes a command the way you'd type it in a terminal, with flags and quoted text. To go to the sofa, the MCP version names the tool "go" with a destination of "sofa one." The command-line version writes a command with a destination flag and "sofa one" in quotes.
Eris: People have compared MCP and command-line agents before. What's different here?
Vestra: Earlier comparisons usually changed several things at once: how finely the tools were split, the prompts, the surrounding software. Here, both interfaces are generated from one shared list of allowed actions. Whatever the model writes in either format is translated back into the same internal action, checked by the same validator and executed by the same engine. They verified that every action survives the round trip in both formats. The only things that differ are how the actions are described to the model, the format it must answer in, and how results and errors are shown back to it.
Eris: And what did they run it on?
Vestra: Three text-based simulations. One is a household where the agent might have to cool a mug and put it in the coffee machine. One is a simulated online store where it searches and clicks to buy a product matching a description. The third is a science lab simulation, with tasks like growing a plant from a seed. The models were four sizes of Alibaba's Qwen3, DeepSeek V4 Flash, and Google's Gemini 2.5 Flash. That gave two thousand and four paired runs, each task run in both formats under identical conditions.
Eris: The household example I opened with was the Qwen3 model with fourteen billion parameters. But across the whole grid there was no consistent winner. Sometimes MCP did better, sometimes the command line did, and the token cost could move in the opposite direction from the score.
Vestra: And the differences went beyond formatting mistakes. Even among answers that were correctly formatted, the choice of action shifted. On the store task, one model clicked more and searched less under MCP. The format changes what the model decides to do, along with how reliably it writes it down.
Eris: What does a format problem look like inside an actual run?
Vestra: In one store task, the goal was to find a men's dress shirt with particular fabric, fit, color and size. Under MCP, the model sent a search, then clicked a product, and the task succeeded. Under the command line, the model wrote a search wrapped in single quotes, but the query contained "men's" with an apostrophe, which closed the quote early. The parser rejected it with an error about a missing closing quotation. The model retried with the apostrophe removed, and the search went through, but the run had already taken a different path, and it ran out of allowed steps without buying the right shirt. One apostrophe at the first step changed the outcome.
Eris: Can they tell which part of the format causes the trouble?
Vestra: They froze the agent at a few hundred specific moments and swapped the three pieces one at a time: the description of the tools, the answer format, and the way feedback is shown. The answer format usually mattered most. For the thirty-two billion parameter Qwen3 in the household tasks, changing only the answer format from a tool call to a command took formatting failures from almost none to nearly nine in ten. The pieces also interacted, so the effect of one depended on the other two. And across the paired runs, nearly all of them diverged in some meaningful way, usually within the first few steps, after which the two runs collected different histories and costs.
Eris: Can you stop an agent from losing a good decision to a typo like that?
Vestra: That's what their fix, called DART, tries to do. When an action fails, the model is first asked to state what it wants to do in a neutral form, just the operation and its arguments, with no interface formatting. That intended action is checked and locked. Then the model is asked to express that exact action in the required format. If the expression fails, it gets one correction, which may change the formatting but not the action. The idea is to repair the spelling of a decision without letting the model drift into a different decision. In one recorded case, a pillow size written with inch marks lost its quotation marks inside the command. The correction escaped them properly, the click went through, and the purchase completed.
Eris: Did it help?
Vestra: They tested it with Gemini 2.5 Flash only. The score went up in all six combinations of task and format, from barely noticeable on the store to a sizable gain on household tasks under MCP. Under MCP it also used fewer tokens in all three environments. Under the command line, token use went up in two of the three. One detail matters for how to read it: only four failures ever reached the correction step, and three of those were fixed. So most of the improvement came from first stating the intended action and then writing it, while the separate repair step was rarely needed.
Eris: So what should someone building agents take from this?
Vestra: That the interface is part of what the model is deciding with, not a neutral pipe. Results from one format don't transfer automatically to another, even when the underlying actions are identical, and the direction of the effect depends on both the model and the task. The limits are real too. These are simulated text environments, the main runs used one fixed, deterministic sampling setting, and the fix was tested on a single model.
Multi-Token Prediction Done Right
Eris: Jayden Teoh at MIT found a flaw in the standard way language models are trained to predict several words ahead. Part of the training signal is mostly noise, and it works against the model's main job of predicting the next word. Removing it takes one line of code. On models up to one point two billion parameters, the fix improved every kind of measurement the paper tested, by small margins, at no extra training cost.
Vestra: The technique being fixed is called multi-token prediction. Alongside the normal output that predicts the next word, the model gets a few small extra modules, often called heads. One predicts the word after next, another the word after that, and so on. DeepSeek's V3 and V4, Nvidia's Nemotron 3, GLM-5 and Kimi K3 all train this way. It helps in two ways. The extra targets seem to give the model a better training signal, and the extra heads can speed up generation. They draft a few words ahead, the main model checks all of those drafts at once, and it keeps the ones it agrees with. That's called speculative decoding.
Eris: And a draft only counts if the main model would have said the same thing.
Vestra: Which is the key to the paper. Each extra head is trained against the word that actually appears in the text. The paper shows that its error signal splits exactly into two parts. The first part is the gap between what the extra head predicts and what the main model will predict once it reaches that same position. That pulls the head toward agreeing with the main model, which is exactly what speculative decoding rewards. The second part, the residual, is the gap between the main model's prediction and the one word that actually showed up.
Eris: Why is the second part noise? It's the real data.
Vestra: Because a single word is one sample from many reasonable possibilities. Take "the weather today is." The text might continue with "cold," but "sunny," "nice," or "awful" were all fine. If the main model has already spread its bets sensibly, the residual still yanks the extra head toward "cold" specifically, purely because of which word this one document happened to use. Across billions of examples that randomness never goes away. Meanwhile the main model's genuine mistakes, the useful part of the residual, keep shrinking as models grow and train longer.
Eris: Did he measure that, or is it a theoretical worry?
Vestra: He measured it. The residual had about four times less signal relative to its noise than the agreement part, and that gap widened slightly as the models got bigger. Yet the residual made up roughly four fifths of the multi-token signal flowing back into the shared body of the model. When he nudged the model's internal state in the residual's direction, the main next-word prediction got worse. Nudging it in the agreement direction made it better.
Eris: Earlier work tried training the extra heads to match the main model directly. Why not just do that?
Vestra: Because it's expensive. Matching two predictions directly means building full probability tables over the entire vocabulary, tens of thousands of entries for every position, for both the main model and the extra head. Modern training code avoids building those tables at all, using optimized routines that compute the loss without storing them. At the largest size in the paper, on one H200 graphics card, the direct-matching version needed about twenty-one gigabytes more memory and trained noticeably slower.
Eris: So what is the one line?
Vestra: It relies on the main output and the extra heads sharing the same final layer, which turns internal states into word probabilities. The line adds to the main model's internal state a value that is exactly zero: a frozen copy of the extra head's state minus the live version of that same state. Going forward, nothing changes, so the model makes identical predictions. But during training, when the error signal flows backward, that zero term routes the main model's next-word error into the extra head with a minus sign. That cancels the residual exactly, and only the agreement signal reaches the extra heads and the shared body. The final layer itself is still trained against the real text. He calls the result Soft-MTP.
Eris: And the gains themselves?
Vestra: He trained models at four sizes, from about one hundred sixty-seven million to one point two billion parameters, each on roughly thirty billion tokens, the word pieces a model reads, of educational web text. Drafts were accepted slightly more often in every domain tested, so generation got a little faster, by one or two percent on average. On general knowledge and reasoning quizzes, ordinary multi-token training didn't beat plain next-word training at these sizes. Soft-MTP did beat it at the two larger sizes. After fine-tuning the largest model on math and on code, Soft-MTP came out ahead of standard multi-token training in seven of eight comparisons, and it was the only method that never fell below plain next-word training.
Eris: How does it compare with the expensive version that matches the predictions directly?
Vestra: Roughly even. The direct version did a little better after math fine-tuning, Soft-MTP did better on all the code comparisons, and their drafts were accepted at almost the same rate. That makes sense, because both send the same agreement signal into the extra heads and the shared body. Training throughput and memory for Soft-MTP were essentially identical to standard multi-token training.
Eris: And the final layer gets real-data training in Soft-MTP but not in the direct version. Does that make a difference?
Vestra: The paper says yes. In the direct version, the final layer can make the two predictions agree partly by becoming less sensitive to the ways the two internal states differ. That's a shortcut, and it shows up in the layer's effective number of dimensions. Standard multi-token training shrank that to about a quarter of what plain next-word training ends up with, the direct version to about two thirds, and Soft-MTP stayed closest to the plain model.
Eris: What would keep a frontier lab from adopting this tomorrow?
Vestra: Mostly scale and certainty. The largest model here is one point two billion parameters, against frontier models that are hundreds of times bigger. The paper reports no repeated training runs, and many of the gains are small enough that you'd want them before trusting the result. Training that splits long sequences across several chips needs a small amount of extra communication. And the paper itself notes that DeepSeek V4.1 Flash dropped multi-token prediction in favor of a separate draft model trained after the main training run, so whether this technique stays in pre-training at all is still open. The case for trying it is that it costs essentially nothing and the code is public.
Wrap-Up
Eris: That's the research for today. An open test of the watermark method used in Gemini and Claude found almost no quality cost on prose and a small one on code for one model, while showing that nobody outside the companies can check the deployed versions. A separate attack paper showed that, with enough queries, the same family of watermarks can be evaded without the key. Agents given identical actions behaved differently depending on whether those actions came as tool calls or terminal commands. And a one-line change to multi-token training removed a noisy part of the signal for small, consistent gains.
Vestra: All four papers are linked in the episode description. Today's news, including the malicious Claude Code extension in Cursor's marketplace, is in our separate news episode. Every story we cover is also on our news site, Ground Truth, at groundtruth dot day, updated daily.
Eris: If this helped you make sense of the watermark debate, follow AI in Detail so tomorrow's episode shows up in your feed.