News · 2026-07-05

ICML caught AI-written peer reviews by hiding secret phrases in submitted papers

ICML 2026, the field's largest machine-learning conference, caught reviewers who secretly used AI to write their peer reviews by planting invisible traps inside the papers themselves. Each submitted PDF carried machine-readable instructions -- unseen by a human, but read by any large language model processing the file -- ordering the model to slip two rare, randomly chosen phrases into whatever review it produced. Reviews that came back carrying those phrases had almost certainly been generated by an LLM. About 1% of all reviews were flagged, and 497 papers were desk-rejected because their authors had pledged not to use AI and then did.

Key facts

• The trap: two phrases sampled from a 170,000-phrase dictionary; the odds of any pair recurring by chance are below 1 in 10 billion.
• The catch: 795 reviews (~1%) flagged; 398 'reciprocal' reviewers had 497 papers (~2% of submissions) desk-rejected.
• The false-alarm rate: the chance of wrongly flagging even a single honest review was computed at 0.0001.
• Primary source: Rao, Kumar, Lakkaraju, Shah, 'Detecting LLM-generated peer reviews,' PLOS ONE, Sept. 22, 2025.

Peer review is the unglamorous machinery that decides which research the field takes seriously: unpaid experts read submissions and judge whether they're sound. As ICML submissions roughly doubled year over year -- from about 12,000 in 2025 to nearly 24,000 in 2026 -- the temptation to outsource that reading to a chatbot grew, and so did the pile of shallow, copy-pasted 'AI reviews' that authors complained were useless. ICML's response was to make the papers fight back. The method is a controlled use of prompt injection: the same trick attackers use to smuggle hidden commands into a model's input, turned into an integrity tool. Because a human reviewer never sees the buried instruction, only an LLM that ingests the whole PDF will obey it and stamp the review with the tell.

The underlying technique was published in 2025 by Carnegie Mellon's Nihar Shah -- also ICML's scientific-integrity chair -- with co-authors at Harvard. Their paper describes 'indirect prompt injection via the paper's PDF, prompting the LLM to embed a covert watermark in the generated review,' plus a statistical test that controls the family-wise error rate 'across multiple reviews... while making no assumptions about the nature of human-written reviews.' In pre-deadline tests, frontier LLMs followed the hidden instruction more than 80% of the time. Every flagged review was then read by a human before any action. Shah told The Transmitter that in years of working on peer review he has 'hardly seen such strong support for anything -- people were really tired of reviewers copy-pasting AI-generated reviews.'

Not everyone applauded. Researcher Soeren Auer called hidden prompts a 'problematic enforcement mechanism' and argued 'it's not good to prohibit the use of AI -- we should rather have a discussion on how to use it.' Sara Atito of the University of Surrey called it a 'poor mechanism' that catches some cheating without fixing the deeper strain on peer review. The organizers agree about the limits: they conceded on the ICML blog that inserting hidden instructions 'is not a difficult measure to circumvent, particularly if it is known publicly -- which was the case for almost the entire review period.' The trap only snares reviewers who dump the full PDF into a chatbot and paste the answer back; anyone who paraphrases, drafts in pieces, or works around the injected line sails through. So the honest reading of the 1% figure is that it's a floor: if the crudest possible method already caught one in a hundred reviews, the real rate of AI use is almost certainly higher.

The practice is spreading. NeurIPS, the year's other flagship conference, has quietly adopted similar prompt-injection enforcement and declined to describe it publicly for fear of tipping off reviewers. The whole episode is a preview of an arms race the field is only beginning: as AI writing becomes indistinguishable from human writing, the institutions that depend on human judgment are reaching for machine-readable booby traps to tell the two apart. ICML runs July 6-11 in Seoul; see the conference site and the 2025 detection paper for the full method.

Primary source, verified: read the paper →

Ground Truth. (2026, July 5). ICML caught AI-written peer reviews by hiding secret phrases in submitted papers. Ground Truth. https://groundtruth.day/news/icml-hidden-prompt-injection-peer-review.html

@misc{groundtruth:icml-hidden-prompt-injection-peer-review,
  title  = {ICML caught AI-written peer reviews by hiding secret phrases in submitted papers},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/icml-hidden-prompt-injection-peer-review.html}
}

Topics: icml · peer-review · prompt-injection · llm-detection · research-integrity · watermarking