https://arxiv.org/abs/2509.01092
Meta Superintelligence Labs' first paper is about RAG
paddedinputs.substack.com
Meta Superintelligence Labs' first paper is about RAG
1–10 of 283 posts
Re: Meta Superintelligence Labs' first paper is about RAG
#2> Long awaited first paper from Meta Superintelligence Labs is not a model layer innovation. What does this mean?
It means you're reading into it too much and need to be let down, gently, from the hype train.
Re: Meta Superintelligence Labs' first paper is about RAG
#3A great post, it starts with this:
TL;DR
• MSI’s first paper, REFRAG, is about a new way to do RAG.
• This slightly modified LLM converts most retrieved document chunks into compact, LLM-aligned chunk embeddings that the LLM can consume directly.
• A lightweight policy (trained with RL) decides which chunk embeddings should be expanded back into full tokens under a budget; the LLM runs normally on this mixed input.
• The net effect is far less KV cache and attention cost, much faster first-byte latency and higher throughput, while preserving perplexity and task accuracy in benchmarks.
I wish more long posts followed this model of a scientific paper.
Re: Meta Superintelligence Labs' first paper is about RAG
#4Interesting. All developers I know who tinkered around with embeddings and vector similarity scoring were instantly hooked. The efficiency of computing the embeddings once and then reusing as many times as needed, comparing the vectors with a cheap IMO vector embedding is the most important innovation in computing of the last decade. There's something magical about it. These people deserve some kind of prize. The idea that you can reduce almost any intricate concept including whole paragraphs to a fixed-size vector which encapsulates its meaning and proximity to other concepts across a large number of dimensions is pure genius.
Re: Meta Superintelligence Labs' first paper is about RAG
#5This was a very obvious next step, I played around with implementing something similar at one point.
In general we need to make it simpler for LLMs to take in different forms of embeddings. At least frameworks that simplify it.
Re: Meta Superintelligence Labs' first paper is about RAG
#6Interesting. All developers I know who tinkered around with embeddings and vector similarity scoring were instantly hooked. The efficiency of computing the embeddings once and then reusing as many times as needed, comparing the vectors with a cheap IMO vector embedding is the most important innovation in computing of the last decade. There's something magical about it. These people deserve some kind of prize. The ide…
Vector embedding is not an invention of the last decade. Featurization in ML goes back to the 60s - even deep learning-based featurization is decades old at a minimum. Like everything else in ML this became much more useful with data and compute scale
Re: Meta Superintelligence Labs' first paper is about RAG
#7At first I thought the super intelligence wrote a novel scientific paper
Re: Meta Superintelligence Labs' first paper is about RAG
#8Interesting. All developers I know who tinkered around with embeddings and vector similarity scoring were instantly hooked. The efficiency of computing the embeddings once and then reusing as many times as needed, comparing the vectors with a cheap IMO vector embedding is the most important innovation in computing of the last decade. There's something magical about it. These people deserve some kind of prize. The ide…
Vector embedding is not an invention of the last decade. Featurization in ML goes back to the 60s - even deep learning-based featurization is decades old at a minimum. Like everything else in ML this became much more useful with data and compute scale
Yup, when I was at MSFT 20 years ago they were already productizing vector embedding of documents and queries (LSI).
Re: Meta Superintelligence Labs' first paper is about RAG
#9Interesting. All developers I know who tinkered around with embeddings and vector similarity scoring were instantly hooked. The efficiency of computing the embeddings once and then reusing as many times as needed, comparing the vectors with a cheap IMO vector embedding is the most important innovation in computing of the last decade. There's something magical about it. These people deserve some kind of prize. The ide…
Vector embeddings are slightly interesting because they come pre-trained with large amounts of data.
But similar ways to reduce huge numbers of dimensions to a much smaller set of "interesting" dimensions have been known for a long time.
Examples include principal component analysis/single value decomposition, which was the first big breakthrough in face recognition (in the early 90s), and also used in latent semantic indexing, the Netflix prize, and a large pile of other things. And the underlying technique was invented in 1901.
Dimensionality reduction is cool, and vector embedding is definitely an interesting way to do it (at significant computational cost).
Re: Meta Superintelligence Labs' first paper is about RAG
#10I'm curious whether this is work that was specifically begun under the "superintelligence" umbrella, or if it's just that the people who were working on it had been shifted to the Superintelligence team by the time they wrote the paper. I would guess the former?