embeddings
Encoder, decoder, or both: the three ways to build a transformer Lesson
The same transformer block can be assembled three ways -- encoder-only, decoder-only, or encoder-decoder -- and the choice determines whether a model reads text, writes text, or converts one into the other.
Approximate nearest neighbor search: how a vector database finds a needle in a billion haystacks Lesson
Approximate nearest neighbor search is the algorithm that makes vector search fast enough to be useful - it finds the closest matches to a query embedding without comparing it against every item in the database, trading a small, tunable amount of accuracy for speedups of a hundred times or more. Every vector database and every retrieval-augmented system runs on it, and the accuracy it gives up is the hidden knob behind a lot of 'the retrieval just missed it' bugs.
Contrastive learning: teaching models by pulling likes together and pushing unlikes apart Lesson
Contrastive learning is a self-supervised training method that learns useful representations without labels by pulling matching pairs closer together in an embedding space and pushing mismatched pairs apart - the technique behind SimCLR and CLIP, and the classic alternative to generation-based approaches for teaching a model to perceive.
Retrieval-Augmented Generation: giving a model an open book Lesson
A language model only knows what it learned in training, and it cannot cite sources. RAG fixes both by letting the model look things up in a real document store at answer time, then write its answer from what it found.
Embeddings: how AI turns words into directions in space Lesson
Before a model can reason about a word, an image, or a paragraph, it has to turn it into a list of numbers -- a vector. Embeddings are how meaning gets encoded as position in space, so that closeness becomes similarity.
Tokenization: how an AI chops your words into pieces it can read Lesson
A language model never sees letters or words. It sees tokens, chunks of text turned into numbers. How that chopping works quietly shapes cost, context limits, multilingual fairness, and why models stumble on spelling and arithmetic.
NVIDIA Nemotron 3 Embed 8B Tool
8-billion-parameter retrieval encoder that turns queries and documents into normalized dense vectors for semantic search. NVIDIA claims state-of-the-art results on the multilingual RTEB leaderboard as of July 16; released under OpenMDW 1.1.
DenseOn Tool
A fully open 149-million-parameter dense retrieval model from LightOn for multilingual, long-context and code search, released with its training data and training code rather than weights alone.