Learn · Intermediate

Contrastive learning: teaching models by pulling likes together and pushing unlikes apart

Contrastive learning is a way to train a model to build useful internal representations of data without needing labels, by teaching it a simple objective: things that belong together should end up close in the model's internal space, and things that don't should end up far apart. It is the technique behind landmark systems like SimCLR and CLIP, and it is the classic alternative to generation-based methods for teaching a machine to perceive - a contrast made vivid this week by new research arguing that generation, not comparison, may be the better teacher for vision.

Start with the core problem. To be useful, a model needs to convert raw data - an image, a sentence - into a compact vector of numbers called an embedding that captures its meaning. The old way to learn good embeddings was supervised: show the model millions of hand-labeled examples ('this is a cat,' 'this is a dog') and let it figure out representations that support those labels. But labeling is expensive and limited. Self-supervised learning asks: can the model learn good representations from the structure of unlabeled data alone? Contrastive learning is one powerful answer.

Here is the mechanism. Take an unlabeled image. Create two different augmented versions of it - crop it, flip it, shift its colors - so you have two views that are clearly the same underlying thing. These form a 'positive pair.' Now grab a bunch of other, unrelated images; each of those, paired with your original, is a 'negative pair.' The training objective is to arrange the embedding space so the two views of the same image land close together while the unrelated images are pushed far away. Do this across millions of images, and the model is forced to discover what actually makes an image that image - its content and structure - because that is the only signal robust to cropping and color shifts. Surface details get discarded; meaning gets kept.

The analogy: imagine sorting a giant pile of photographs with no captions, using one rule - two photos of the same scene taken from slightly different angles go in the same bin, and photos of different scenes go in different bins. To follow that rule well, you have to start recognizing the underlying content of each photo, not its incidental details. After sorting long enough, you have implicitly learned what scenes are. That implicit understanding is exactly the representation contrastive learning produces.

The most influential example is CLIP, from OpenAI, which applied the idea across two modalities at once. Instead of two views of the same image, CLIP's positive pairs were images and their real text captions, with mismatched image-caption combinations as negatives. Trained on hundreds of millions of image-text pairs from the web, it learned a shared embedding space where a picture of a dog and the words 'a photo of a dog' land in nearly the same spot. That shared space is what enables 'zero-shot' classification (recognize categories it was never explicitly trained on, just by comparing to text) and underpins the text understanding in many text-to-image and multimodal systems. SimCLR and MoCo, meanwhile, showed how far the image-only version could go, with MoCo introducing clever tricks to keep a large, consistent supply of negatives during training.

Why this concept matters right now: contrastive learning has been one of the two dominant recipes for self-supervised vision, the other being masked prediction (hide part of the input, predict what was hidden). This week's GenCeption research throws a third contender into the ring - generation. Its argument is that a model which learns to generate video internalizes the physics and geometry of the world so thoroughly that it becomes a better perceiver than a model trained to merely compare or reconstruct, and it backs this with striking data-efficiency gains. Understanding contrastive learning is what lets you appreciate that debate: it is the reigning 'learn by comparing' paradigm that the 'learn by generating' challengers are trying to unseat.

The honest caveats are worth knowing. Contrastive methods are sensitive to how you construct augmentations (bad augmentations teach the wrong invariances) and often need large batches or memory banks to supply enough negatives, which is computationally demanding. And the choice of what counts as a 'positive pair' quietly encodes your assumptions about what should be treated as the same. But the core idea - learn meaning by arranging an embedding space so like attracts like and unlike repels - is one of the most important and reusable ideas in modern machine learning, and it shows up far beyond vision, from sentence embeddings to recommendation systems to the retrieval step in retrieval-augmented generation.

Ground Truth. (2026, July 13). Contrastive learning: teaching models by pulling likes together and pushing unlikes apart. Ground Truth. https://groundtruth.day/learn/contrastive-learning.html

@misc{groundtruth:contrastive-learning,
  title  = {Contrastive learning: teaching models by pulling likes together and pushing unlikes apart},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/learn/contrastive-learning.html}
}

Topics: self-supervised-learning · representation-learning · embeddings · clip · computer-vision