training-data
Benchmark contamination: when the test is already in the training data Lesson
Benchmark contamination is what happens when the questions used to evaluate a model were in the data used to train it, turning a test of reasoning into a test of memory and inflating scores in ways that are hard to detect after the fact.
Anthropic retrained on the alignment-faking transcripts it had blocked News
Anthropic's August 2026 risk report discloses that filters meant to keep tens of thousands of published alignment-faking transcripts out of training data were misconfigured for several model generations, and it now suspects every Anthropic model with a knowledge cutoff after December 2024 saw some of them.
Amazon quietly put Mechanical Turk in maintenance mode, and the shutdown date going around is not in any AWS document News
AWS documentation states that Mechanical Turk is closed to new customers with existing customers unaffected and no new features planned, but no AWS page confirms the September 30 shutdown date circulating in coverage.
Training data attribution: which examples actually made the model do that? Lesson
Training data attribution is the set of techniques for tracing a model's output back to the specific training examples responsible for it, and the honest state of the art is that it often fails.
Stack Overflow took 1,490 questions in July News
Stack Overflow received 1,490 new questions in July 2026, down from 6,414 in July 2025 and 176,610 in July 2014 -- a 118-fold collapse in the public programming corpus that trained today's coding models.
A Munich Court Found Suno's Models Memorised Six Songs News
The Regional Court of Munich I largely granted GEMA's claims against Suno, holding that six well-known works were reproducibly stored in Suno's models and could be extracted through simple prompts, and assigning responsibility to Suno rather than its users.
Data poisoning and backdoors: attacking a model through what it eats Lesson
Data poisoning is an attack that corrupts a model by tampering with its training data rather than its code, and a backdoor is the sharpest form: a model that behaves perfectly until it sees a secret trigger. Anthropic and the UK AI Safety Institute found in 2025 that just 250 poisoned documents compromised models from 600 million to 13 billion parameters alike, which means scale does not dilute the threat.
Google loses its DMCA claim against a search scraper News
A federal judge dismissed both of Google's copyright anti-circumvention claims against SerpApi, ruling that a general anti-bot wall around non-copyrightable search results cannot be treated as a copyright access control.
Leaked Suno code names YouTube, Deezer, Genius and podcast RSS feeds as collection sources News
A hack of AI music generator Suno exposed source-code files and comments naming YouTube Music, Deezer, Genius, stock libraries and podcast RSS feeds as data collection targets, the most specific provenance evidence yet in the music industry's copyright fight.
Training data deduplication: why the same text twice makes a model worse Lesson
Deduplication is the process of finding and removing repeated or near-repeated documents from a training corpus, and it reliably makes language models better, cheaper, and safer - the same text seen many times gets memorised rather than learned from.
Hugging Face ships The Stack v3, and this time the code is actually in the box News
Hugging Face released The Stack v3, a code dataset of roughly 4.9 trillion tokens across 173 million repositories with source text embedded inline, ending the v2 arrangement where researchers had to fetch the actual file contents themselves.
This model's job is to make better training data for other models News
DataClaw0 turns the grind of cleaning and labeling training data into a learned skill -- a small model that refines raw, messy multimodal streams into dense, purpose-built lessons.
Synthetic Data: When AI Makes Its Own Training Material Lesson
The internet is running out of fresh text to train on, so the most advanced models increasingly learn from data that other AI made or shaped. Here is how that works, why it helps, and how it can quietly poison a model.
An open project publishes the recipe for training capable AI agents News
OpenThoughts-Agent releases its full data-curation pipeline, dataset, and experiments -- showing that what an agent learns from matters more than raw size, and letting anyone reproduce it.
The Stack v3 Tool
Hugging Face's code corpus, now with source text embedded inline rather than behind identifiers. A 15.9 TB deduplicated, PII-redacted training split of roughly 4.9 trillion tokens, plus a 113.7 TB unfiltered bucket for teams that want to build their own mix.
GPTurk Tool
EPFL's released code for detecting language-model-assisted submissions in crowd work, combining keystroke logging with a synthetic-versus-real text classifier. Practical for anyone buying human-labeled data who needs to check whether the labels were actually written by people rather than pasted from a chatbot.