compression
Everything on Ground Truth tagged “compression” — 2 items.
Arithmetic coding: turning a model's probabilities into a compressed message Lesson
Arithmetic coding is a lossless compression method that represents an entire message as a narrowing interval, making precise why a model that assigns better probabilities can also compress better.
A gzip language-model demo shows why prediction and compression are related News
Nathan Barry's gzip experiment uses compressed length to rank continuations, making a real information-theory connection visible while also showing why local byte copying is not language understanding.