Learn · Beginner
In-context learning
In-context learning is the ability of a language model to learn a new task from examples written into its prompt, without any change to the model's weights. Show a model three translations and ask for a fourth, and it will do the task it was never explicitly trained to do. Nothing inside the model has been updated -- the adaptation exists only for the length of that one conversation, and it disappears the moment the context is cleared. This is the mechanism underneath essentially every prompting technique you have heard of.
The phenomenon was named and popularised in 2020 by the GPT-3 paper, Language Models are Few-Shot Learners, from a team at OpenAI led by Tom Brown. Their central observation was that a model trained on nothing but next-word prediction could, when given a handful of worked examples, perform tasks like translation, arithmetic, and unscrambling words at levels that previously required task-specific training. They distinguished three settings that are still the standard vocabulary: zero-shot (just an instruction), one-shot (an instruction plus a single example), and few-shot (an instruction plus several).
Why a next-word predictor can do this at all
The usual way to teach a neural network something new is gradient descent: show it labelled data, measure the error, and nudge every weight slightly. In-context learning does none of that. So where does the learning happen?
The most useful intuition comes from thinking about what a model trained on the whole internet is actually doing. It has seen millions of documents that begin with a pattern and continue it: glossaries, code with docstrings, bilingual pages, exam papers with worked solutions. To predict text well, the model had to get good at inferring "what kind of document am I in, and what rule is it following" from the first few lines, then applying that rule. In-context learning is that skill pointed at a task you invented.
A 2021 paper by Sang Michael Xie and colleagues, An Explanation of In-context Learning as Implicit Bayesian Inference, makes this precise. The model behaves as if it holds a distribution over latent concepts, and the examples in the prompt act as evidence that sharpens which concept it thinks is in play. The prompt does not teach the task. It identifies a task the model already latently knows.
A different line of work found something stranger. What learning algorithm is in-context learning?, by Ekin Akyurek and colleagues, showed that on simple problems like linear regression, transformers trained in context implement something mathematically close to standard learning algorithms -- effectively running a small optimisation inside the forward pass. The attention layers, in other words, can act as a tiny learner operating on the examples in front of them.
The result that should unsettle you
In 2022 Sewon Min and colleagues published Rethinking the Role of Demonstrations, which tested an obvious-seeming assumption: that the example answers in a few-shot prompt need to be right. They replaced the correct labels with random ones and found performance barely moved on many tasks. What mattered was the format, the distribution of input text, and the set of possible labels -- not the mapping between them.
The practical reading is that a few-shot prompt is doing less teaching and more instructing: it tells the model what shape of answer to produce and what universe of answers is allowed. If you have ever written five examples and been disappointed that the sixth was wrong in the same way, this is why. You showed the model the format perfectly and the reasoning not at all. That is also part of why chain-of-thought prompting helps -- it changes the format to one that includes intermediate steps, so the model produces steps.
Where it comes from, and what it costs
In-context learning does not appear in every model. Work by Stephanie Chan and colleagues, Data Distributional Properties Drive Emergent In-Context Learning in Transformers, showed that it emerges when the training distribution has particular shapes: many rare classes rather than a few common ones, and "bursty" data where the same entity recurs within a document. Natural language has both. So the capability is not purely a gift of scale -- it is a gift of scale applied to data that happens to look like this.
The cost is the honest limitation. Every example you put in the prompt is re-read on every request, consuming part of the context window and adding latency and expense. Learning that lives in the prompt is learning you pay for repeatedly, which is why prompt caching exists and why, past a certain volume, fine-tuning becomes cheaper than prompting. The rule of thumb: in-context learning is right when you have few examples or a task that changes often, and fine-tuning is right when you have many examples and a task that does not.
Language Models are Few-Shot Learners (Brown et al., 2020)
An Explanation of In-context Learning as Implicit Bayesian Inference (Xie et al., 2021)
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (Min et al., 2022)
What learning algorithm is in-context learning? (Akyurek et al., 2022)
Data Distributional Properties Drive Emergent In-Context Learning in Transformers (Chan et al., 2022)
Key questions
Is in-context learning the same as fine-tuning?
Do the example answers in a prompt actually need to be correct?
Why did in-context learning appear only in large models?
Cite this
APA
Ground Truth. (2026, August 10). In-context learning. Ground Truth. https://groundtruth.day/learn/in-context-learning.html
BibTeX
@misc{groundtruth:in-context-learning,
title = {In-context learning},
author = {{Ground Truth}},
year = {2026},
month = {aug},
url = {https://groundtruth.day/learn/in-context-learning.html}
}