training
Fine-tuning and LoRA: teaching an old model a new job without retraining it Lesson
You almost never train an AI from scratch. You take one that already knows language and nudge it toward your specific task - and a trick called LoRA lets you do that by adding a tiny sticky note instead of rewriting the whole brain.
A 35-billion-parameter agent that punches like a trillion-parameter model News
Shanghai AI Lab argues you can reach giant-model performance on long tasks not by adding parameters, but by training on much longer chains of real work.
Backpropagation: how a neural network learns from its mistakes Lesson
The single algorithm behind nearly all AI training - assigning blame for an error backward through millions of dials, so each one knows which way to turn.
An AI's hallucinations turned out to be a map with blank spots News
Researchers showed that when a world-model AI imagines impossible futures, it's usually in places it barely saw in training - and that you can predict and fix those blind spots cheaply.
A wave of new methods trains AI without a human answer key News
Several research groups landed on the same idea at once - improve a model by learning from its own attempts instead of expensive human labels - and the field is debating whether it really removes the labeling burden or just hides it.
Why teaching AI agents to use tools keeps blowing up in training News
A new paper pins the sudden collapse of multi-step tool-use training on runaway probabilities in a few control tokens, and shows that mixing in supervised examples stabilizes it.
Training vs inference: the two very different jobs inside every AI Lesson
Why building an AI model and using it are separate worlds with separate costs, and why that split explains custom chips, model prices, and where the real money in AI actually goes.
Distillation: how a small AI learns from a big one Lesson
Distillation trains a smaller, cheaper model to imitate a larger, smarter one, the idea behind both efficient deployment and the 'copying' accusations now driving AI geopolitics.
Teaching AI with rewards — minus the expensive second model that grades it News
The standard way to polish a model with rewards quietly runs a second 'critic' model alongside it. A new method derives the critic's judgment from the model itself, dropping the extra cost.
Scaling laws — does bigger always mean better? Lesson
For years, AI progress ran on a simple recipe: make the model bigger, feed it more data, get a better model. That pattern is real and predictable — but it has limits and surprises. Here's what scaling laws actually say.
Polishing AI by looking inside its 'mind' instead of just thumbs-up, thumbs-down News
Reward training usually treats the model as a black box — thumbs up, thumbs down, hope for the best. A new method peers inside to see why an answer was preferred, and shapes the lesson on purpose.
The little words that keep AI from getting boring News
Rewarding a reasoning model too hard makes it repetitive — and the casualties are tiny words like "but" and "instead" that let it branch to a better thought. A near-free fix protects them.
Faster AI training by quietly cloning the model News
Teaching a model with rewards is slow because it has to write out endless practice answers. A new trick: make a cheap, shrunk-down copy of the model to crank those out faster.
Crediting an AI for the right steps — without a second model to judge them News
When you reward an AI for a good final answer, it's hard to know which of its steps earned the credit. The usual fix is training a second 'judge' model. This skips that.