News · 2026-11-12

Apple's learned support functions route retrieval queries faster by modeling each cluster's best match

Apple researchers say a small neural network that learns the mathematical shape of a cluster of stored vectors can route retrieval queries faster than the standard method of comparing a query to cluster centres. In Apple's report of the work, the learned router reaches 95% routing recall about 1.9 times faster than centroid routing, and a second network, KeyNet, reaches 99% recall in up to about 8 times less time, on one dataset.

Key facts

• Paper: "Amortizing Maximum Inner Product Search with Learned Support Functions" by Theo X. Olausson, Joao Monteiro, Michal Klein and Marco Cuturi, presented at ICML 2026, arXiv 2603.08001. Apple's page and arXiv differ slightly on the title ("Amortizing" versus "Amortized"); check before quoting.
• Headline gain: KeyNet reaches 99% recall up to about 8x faster and scans up to about 11x fewer keys (Apple-reported).
• Test set: Natural Questions, about 2.6 million passages split into 128 clusters.
• Primary sources: Apple's research highlight, titled "Faster Retrieval for Better, More Efficient RAG", at machinelearning.apple.com, and the paper page.

The problem in plain terms

Retrieval-augmented generation (see our lesson on RAG) works by turning a question into a vector and finding the stored vector with the highest inner product, a matching score. Searching everything is too slow, so systems cluster the stored vectors, compare the query to each cluster's centre, and search only the nearest clusters. That fails when one cluster holds an outlier far from its centre: the best answer sits in a cluster whose centre looks unpromising. This is the setting of approximate nearest-neighbor search.

The idea: a support function

The function "the best score any key in this set gives this query" is called a support function in convex analysis. It is convex, and its gradient returns the best key itself. Apple's SupportNet is a small input-convex neural network, trained once offline on queries from the expected distribution, that learns each cluster's support function. It routes a query by how good the best key inside a cluster is, not how close the cluster's centre is. KeyNet exploits the gradient fact in a different way: it predicts the best-matching key directly and acts as a query rewriter in front of an ordinary index. Its training signal is exhaustive-search answers computed once for sampled queries.

An analogy: instead of judging a library shelf by the average book on it, SupportNet learns to say "the best book on this shelf for this topic is very good", even if the shelf's average is mediocre.

What Apple reports

These are vendor-reported results on a single dataset:

• SupportNet reaches 95% routing recall about 1.9x faster than centroid routing. At a fixed budget of scanning 5% of the database, routers find the right cluster about 90 to 93% of the time versus 87.5% for the baseline, a modest gain.
• KeyNet reaches 99% recall in up to about 8x less time and with up to about 11x fewer keys scanned. At a tiny scan budget it finds the true key 89 to 97% of the time versus 86% for the raw query.
• The networks, sized "XS to L", are 1%, 5%, 10% and 20% of the database size.

The cost the post does not weigh

That last point is the trade. Training is an upfront cost, and a network that is a fifth of the database is not small for a large corpus. The post does not weigh that against the retrieval speedup, and we did not read the paper's own cost accounting. The method also works only when queries resemble the training distribution; a shift in what users ask would degrade it. We did not verify baselines beyond centroid routing or any independent reproduction. arXiv dates (v1 March 9, last revision July 1) came from a small summarizing model and are unverified.

How people received it

On Hacker News (item 50455633, about 216 points and 38 comments when we looked), the debate was less about whether this method works than whether retrieval matters at all. One recurring view held that for LLM-driven agents, a plain BM25 or grep tool with a wide net beats learned retrieval; replies said grep cannot handle meaning, other languages or typos at terabyte scale. Another commenter found the post's examples laughably easy for anyone who has used embedding search. Nobody reproduced anything. For the lexical side of that argument see BM25.

Why it matters

Most improvements to retrieval change the index. This one changes how queries are routed, and does it by learning a property of the stored data that mathematicians have long understood. The idea "a stored set of vectors has a mathematical shape; learn it once, search faster forever" is clean, and the weak spots are clear: training cost and distribution shift.

Primary source, verified: read the paper → (arXiv 2603.08001)

Ground Truth. (2026, November 12). Apple's learned support functions route retrieval queries faster by modeling each cluster's best match. Ground Truth. https://groundtruth.day/news/apple-learns-the-shape-of-stored-vectors-to-route-rag-queries-faster.html

@misc{groundtruth:apple-learns-the-shape-of-stored-vectors-to-route-rag-queries-faster,
  title  = {Apple's learned support functions route retrieval queries faster by modeling each cluster's best match},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/news/apple-learns-the-shape-of-stored-vectors-to-route-rag-queries-faster.html}
}

Topics: apple · retrieval · rag · vector-search · icml · machine-learning-research