embeddings
Orthogonal Procrustes: aligning embedding spaces without changing their geometry Lesson
Orthogonal Procrustes fits a rotation or reflection between two matched embedding spaces while preserving distances, making shared structure usable without retraining both models.
K-means clustering: grouping examples around representative centers Lesson
K-means groups numerical examples around a chosen number of centers by minimizing squared distance, producing useful structure without proving the groups have meaningful labels.
Matryoshka representations: useful embeddings at several sizes Lesson
Matryoshka representation learning trains nested prefixes of an embedding to remain useful, allowing smaller search vectors without a separate model.
Google releases EmbeddingGemma 2 for local search across words, pictures, and sound News
Google’s EmbeddingGemma 2 maps text and media into a shared search space, with a quantized full configuration measured at about 567 MB of active phone RAM.
Where people go tells a model what a place actually is News
Google Research showed that combining a place's text description with anonymized visit patterns lets a model infer things text alone cannot, improving prediction of why people visit a location by over 80 percent and cutting busyness prediction error by about a quarter.
Encoder, decoder, or both: the three ways to build a transformer Lesson
The same transformer block can be assembled three ways -- encoder-only, decoder-only, or encoder-decoder -- and the choice determines whether a model reads text, writes text, or converts one into the other.
Approximate nearest neighbor search: how a vector database finds a needle in a billion haystacks Lesson
Approximate nearest neighbor search is the algorithm that makes vector search fast enough to be useful - it finds the closest matches to a query embedding without comparing it against every item in the database, trading a small, tunable amount of accuracy for speedups of a hundred times or more. Every vector database and every retrieval-augmented system runs on it, and the accuracy it gives up is the hidden knob behind a lot of 'the retrieval just missed it' bugs.
Contrastive learning: teaching models by pulling likes together and pushing unlikes apart Lesson
Contrastive learning is a self-supervised training method that learns useful representations without labels by pulling matching pairs closer together in an embedding space and pushing mismatched pairs apart - the technique behind SimCLR and CLIP, and the classic alternative to generation-based approaches for teaching a model to perceive.
Retrieval-Augmented Generation: giving a model an open book Lesson
A language model only knows what it learned in training, and it cannot cite sources. RAG fixes both by letting the model look things up in a real document store at answer time, then write its answer from what it found.
Embeddings: how AI turns words into directions in space Lesson
Before a model can reason about a word, an image, or a paragraph, it has to turn it into a list of numbers -- a vector. Embeddings are how meaning gets encoded as position in space, so that closeness becomes similarity.
Tokenization: how an AI chops your words into pieces it can read Lesson
A language model never sees letters or words. It sees tokens, chunks of text turned into numbers. How that chopping works quietly shapes cost, context limits, multilingual fairness, and why models stumble on spelling and arithmetic.
NVIDIA Nemotron 3.5 Embed Tool
A retrieval component that embeds text and document pages, including images, into vectors; the verified model card states OpenMDW 1.1 licensing. Download size and required GPU memory are not established in the dossier.
NVIDIA Nemotron 3 Embed 8B Tool
8-billion-parameter retrieval encoder that turns queries and documents into normalized dense vectors for semantic search. NVIDIA claims state-of-the-art results on the multilingual RTEB leaderboard as of July 16; released under OpenMDW 1.1.
EmbeddingGemma 2 Tool
Released modular embedding model for local retrieval across text, code, images, audio, and sampled video; applications supply the index and interface.
DenseOn Tool
A fully open 149-million-parameter dense retrieval model from LightOn for multilingual, long-context and code search, released with its training data and training code rather than weights alone.