Google's paper on ranking without an index tested only reranking
Summary
A Google paper showing one model can rank a corpus without a separate index is a benchmark result. Its experiments reranked candidates that were already handed to the model, so nothing about crawling, indexing or rank tracking changes today.
The one idea worth acting on is older than the paper: pages that are not separable from each other compete for the same slot. Merge them.
Google researchers published a paper in January proving that one autoregressive model with a fixed hidden dimension can express any ranking of a corpus of any size. A dual encoder, the fast model that today’s two-stage search pipelines use to pull candidates from an index, needs its embedding dimension to grow in step with the number of documents to do the same. The paper, revised in February, works on an approach called autoregressive ranking and adds a training loss, SToICaL, that rewards the whole ranked list rather than only the top result. A Search Engine Journal piece takes that further and argues the ranked list stops being something a user sees and becomes something the model computes on its way to an answer.
The experiments behind the proof are narrow. The team fine-tuned Mistral-7B rather than a frontier model and placed the candidate document identifiers in the prompt, so the model chose among options it had been handed. Choosing among supplied candidates is reranking, not retrieval from an open corpus, and no index was replaced by a model anywhere in the paper. WordNet, one of the two test sets, is a Princeton database of about 82,000 English nouns arranged from broader to narrower concept, which hands the researchers a correct order for every query. The shopping evaluation, on the ESCI product-search dataset, ran on 310 examples.
What the experiments measured is ranking below position one. On WordNet, the rank-aware loss raised recall across positions two through five. The figure moved from the fifties into the mid-nineties. One configuration got worse at putting the single best result first while improving everything beneath it. A reranker that is much stronger at positions two to five and no stronger at position one is a useful component, and that is the honest description of what was demonstrated.
Search Engine Journal’s piece labels its own forecast. Duane Forrester, who wrote the piece, discloses founding CitationIQ, a measurement platform for AI visibility, and says directly that everything past the experiments is informed speculation. The piece also names the state it expects first: classical retrieval feeding a generative ranker. That hybrid is what the paper actually tested.
The strongest argument in the piece is about beam search, the method a generative model uses to produce a short list. At each step it keeps a fixed number of the most probable partial sequences and discards the rest, so the finished list can never be longer than that number. A beam of ten has no position eleven to rank badly, and from outside the system a page that ranked poorly would look identical to a page that was never generated. That much follows from the architecture. Rebuilding measurement around it does not, because nothing is deployed, and AI visibility tools already disagree with each other enough that a 20-point swing can be noise.
One thread is worth acting on, and it is the least speculative. A document in this design has no URL address. It has a docID, a short token sequence the model generates the way it generates words. The shopping experiment built each product’s identifier from the embedding of its title, compressed to three numbers, so similar products share leading digits. Pages that are not separable in embedding space cannot hold separate, stable identifiers, which means two near-duplicates would compete for one address rather than for two adjacent positions. The paper does not test that, and it is the piece’s inference rather than a measured result.
What to do
Consolidate near-duplicate pages on grounds that hold today rather than on this paper. Thin variants and faceted duplicates waste crawl capacity now, and AI Overviews skip thin, stale pages sitting at the same organic rank as better ones. Choose the version that should win, merge the others into it, and point the canonicals at it.
Nothing else here calls for a response. Rank tracking below the top ten still measures something real, because the results page it measures still exists. When a client asks whether Search is about to stop returning links, the plain answer is that Google researchers proved a capacity on two benchmarks and published a training method for reranking, and that no web index has been replaced by a model.