jepa
Everything on Ground Truth tagged “jepa” — 2 items.
Predicting your own latents cuts the sample cost from exponential to flat News
A new proof shows that on hierarchically structured data, learning from tokens needs training examples growing exponentially with the depth of the hidden structure while predicting your own representations needs a number that stays constant.
JEPA: teaching a model to predict its own understanding Lesson
A joint-embedding predictive architecture trains a network to predict its own internal representation of a missing part of the input, rather than predicting the missing pixels or tokens themselves. Skipping the surface detail is what makes it dramatically more data-efficient than generative self-supervised learning.