scaling-laws
Sample complexity: how many examples does learning actually take? Lesson
Sample complexity is the number of training examples a learning method needs to reliably learn a given pattern. It is the formal way of asking why a child learns a word from three exposures while a language model needs the internet, and it is what separates a method that scales from one that merely works.
Training on the best of K guesses is a third scaling axis alongside parameters and data News
A paper from UIUC and Harvard shows that generating several candidate outputs per training example and learning only from the closest match improves sample efficiency 6.2-fold, and that the benefit grows rather than shrinks as models and datasets get bigger.
ByteDance says AI agents double their learning speed every three months News
ByteDance's Seed team released EdgeBench, a benchmark of 134 day-long tasks, and reported that agents' rate of learning from real environments has roughly doubled every three months -- a possible new scaling law measured over about 38,000 hours of agent activity.
Scaling laws — does bigger always mean better? Lesson
For years, AI progress ran on a simple recipe: make the model bigger, feed it more data, get a better model. That pattern is real and predictable — but it has limits and surprises. Here's what scaling laws actually say.