distillation
Garry Tan says American open-weight labs should be free to distill closed frontier models News
Y Combinator chief executive Garry Tan said on 10 and 11 September 2026 that he would do nothing about AI model distillation and that US open-weight labs should be free to learn from closed American frontier models, putting him at odds with a federal advisory that treats distillation as a security threat.
Anthropic says Moonshot and DeepSeek quietly sent their users' requests to Claude News
Anthropic's 10 September 2026 threat report says Moonshot AI and DeepSeek silently forwarded some of their own customers' requests to Claude, showed Claude's answers as their own and saved the exchanges for training, exposing sensitive corporate and government data that those users never knew had left the service they chose.
US agencies name six Chinese AI firms and say copying American models is their core strategy News
A joint NSA, CISA and FBI advisory says China-based AI companies including DeepSeek, Moonshot, Alibaba, MiniMax, StepFun and Z.AI have run industrial-scale distillation campaigns against US frontier models, and recommends that American labs quietly degrade responses to suspected copiers rather than block them.
Attackers sent more than 100 million prompts to copy Google's models News
Google reported coordinated campaigns exceeding 100 million prompts aimed at systematically extracting its models' strongest capabilities, alongside theft of AI API credentials and hijacking of victim cloud accounts to run unauthorized AI workloads.
Anthropic closed the hole distillers used to read Claude's thinking News
With Claude Fable 5.1, Anthropic blocked new API accounts from editing earlier turns of a conversation while keeping Claude's prior reasoning in the transcript -- shutting off a publicly documented technique for extracting a model's internal thinking at scale.
The paper being used to prove Kimi copied Claude says otherwise News
A study on stealing reasoning traces found that Kimi K3 responds unusually strongly to Claude's decoded reasoning, but the authors state plainly that their results cannot establish memorization or distillation, and that reproducing even 16 tokens verbatim would take about ten billion queries.
Evoke keeps a generated world's memory outside the video model News
Alaya Lab released Evoke, a 14-billion-parameter open-weights world model that stores scene geometry in an external bank indexed by camera position instead of inside the image generator, letting an interactive generated world stay consistent for hours.
The cheapest way to teach a model turned out to be blindfolding the student News
Researchers got most of the benefit of an expensive teacher model by degrading the student's view of the image instead of upgrading the teacher, lifting a 4-billion-parameter model past open models nearly sixty times its size with no labels, rewards or larger teacher.
A stronger model built a wrapper that nearly doubled a weaker one's score News
Researchers had a strong model design inference-time scaffolding for weaker models, lifting their average score on four reasoning benchmarks from 0.49 to 0.91 without changing a single parameter.
A model improved itself by training only where it disagreed with itself News
U-OPSD drops the teacher entirely: it samples several attempts, takes a majority vote as a stand-in answer, then trains only on the attempts that disagreed with that vote, matching or beating methods that use real ground-truth labels.
DistillAlign explains why fast video models get prettier and more repetitive at the same time News
A new preprint shows that the standard technique for speeding up video generators optimises a mathematical objective that deliberately concentrates on safe, common outputs, and that adding a diversity-preserving loss back during that stage more than doubled the paper's coverage measure without giving up the quality gain.
Beijing says U.S. firms distilled Chinese models, and names none of them News
China's Ministry of Commerce said in a written statement on 27 July that many U.S. AI companies had distilled Chinese models during research and training, identifying no company, no model and no evidence, mirroring a U.S. accusation five days earlier that named two companies but published no logs either.
A new distillation method lets the teacher model interrupt the student mid-thought News
Researchers found that when a student model starts reasoning down a wrong path, its teacher's next word tends to be a redirection like But or Wait, and turned that disagreement into an automatic trigger for the teacher to briefly take over.
Forward and Reverse KL Divergence Lesson
KL divergence measures how far one probability distribution is from another, and it is asymmetric: swapping which distribution goes first changes whether your model tries to cover everything the target does or commit to one part of it.
The open-weights industry letter grew from 25 names to 35 - and OpenAI is on it News
A cross-industry statement titled Open Weights and American AI Leadership now lists 35 signatories on its live Microsoft-hosted page, including OpenAI, Nous Research, GitHub and Cisco, contradicting the widely shared claim that OpenAI declined to sign.
Diffusion Distillation Lesson
Diffusion distillation trains a fast student model to reproduce in a handful of steps what a slow diffusion model does in dozens, cutting image and video generation from many neural-network passes down to one to four, which is what makes real-time generative rendering possible.
US floats sanctions over AI 'distillation' as Anthropic details 16 million scraped chats News
The Treasury secretary suggested the US could sanction Chinese AI labs over model 'theft' while Anthropic and OpenAI allege large-scale unauthorized scraping of their models' outputs, but the verified record shows provider allegations and a proposed sanctions bill, not enacted policy or any proof that model weights were copied.
Model extraction attacks: stealing an AI through its own API Lesson
A model extraction attack tries to copy a machine-learning model you can only query, not download, by sending it many inputs and learning from its outputs. Depending on the goal, an attacker can clone the model's behavior, recover pieces of its internals, or reconstruct a rival model cheaply, which is exactly the fear driving today's AI 'distillation' disputes.
Grafting a Verified Solution Cache Lets a Frozen Model Skip Fine-Tuning Entirely News
A new paper shows a frozen small language model's accuracy on a hard math test can jump from 80.0% to 93.3% simply by grafting a byte-exact cache of verified solutions into it, with no weight changes at all.
Weak-to-Strong Generalization: how a worse teacher can train a better student Lesson
Weak-to-strong generalization is the finding that a strong model trained on a weaker model's flawed labels can substantially outperform its teacher, which is the only reason humans have any hope of supervising systems smarter than themselves.
A weaker model just taught a stronger one, by passing on the lesson instead of the answers News
Researchers from Tsinghua and ByteDance show that a small model's reinforcement learning gains can be distilled into a larger, already-stronger model by transferring the change in the teacher's policy rather than its outputs, letting a 1.5B teacher improve a 7B student.
Nadella calls it 'ironic' that AI labs train on the world but restrict everyone else from learning back News
Microsoft CEO Satya Nadella published an essay arguing that in the AI era buyers pay twice -- once in money, once in the proprietary knowledge they must reveal to use a model -- and called it ironic that labs claim fair-use training rights while restricting distillation.
A 4B model on your device nearly matched a 72B one - by copying its memories News
Researchers distilled a large AI agent's memory skills onto a compact 4-billion-parameter model, raising its success rate on a household-task benchmark from 4% to 78% while running three times faster than its 72B teacher.
A 32B model reaches frontier level by learning what to remember News
New research reframes agent memory as a trainable skill rather than a growing transcript - and shows that optimizing memory alone lets a 32B open model rival Claude Opus, and a 4B model leap from 4% to 78% on a benchmark.
Samsung's Trick Makes a Tiny 4B Agent Nearly Match a Model 18 Times Bigger News
Samsung R&D UK and Queen Mary University of London published DuoMem, a distillation method that took a 4-billion-parameter agent from a 4.3 percent task-success rate to 77.9 percent, nearly matching a 72-billion-parameter teacher model's 87.1 percent.
Two new papers push 'on-policy distillation' to fix privileged teachers and merge specialist skills News
DOPD and MOPD advance on-policy distillation -- training a student on its own outputs -- with DOPD routing supervision to avoid a 'privilege illusion' and MOPD merging multiple specialist RL teachers into one model without cross-domain interference.
On-Policy vs Off-Policy Learning Lesson
On-policy learning trains a model on data generated by its own current behavior, while off-policy learning trains it on data generated by something else -- an old version, a different policy, or a fixed dataset -- and the choice shapes how stable, sample-efficient, and reliable the training is.
A wave of new methods trains AI without a human answer key News
Several research groups landed on the same idea at once - improve a model by learning from its own attempts instead of expensive human labels - and the field is debating whether it really removes the labeling burden or just hides it.
NVIDIA shrinks video generation down to real time News
A new NVIDIA recipe distills slow video-generating AI into a fast version that can stream frames live and react to your actions.
Distillation: how a small AI learns from a big one Lesson
Distillation trains a smaller, cheaper model to imitate a larger, smarter one, the idea behind both efficient deployment and the 'copying' accusations now driving AI geopolitics.
Anthropic says Alibaba ran the biggest 'copy Claude' campaign yet News
Anthropic told U.S. senators that Alibaba's Qwen team quietly milked Claude for its best skills. Alibaba says nothing back, and the whole fight may be as much about price as theft.
SimpleOPD Tool
An on-policy distillation implementation that works between models with different tokenizers, aligning only the text spans both vocabularies agree on. That lets a teacher in one model family train a student in another without building a vocabulary bridge.
AuK-Flash Tool
The distilled sibling of AuK, generating in four steps instead of the full sampling schedule for markedly faster inference at the same 6.8 GB download size. Same MIT licence and the same instruction interface.