> Reranking: the highest value 5 lines of code you'll add. The chunk ranking shifted a lot. More than you'd expect. Reranking can many times make up for a bad setup if you pass in enough chunks. We found the ideal reranker set-up to be 50 chunk input -> 15 output. What is re-ranking in the context of RAG? Why not just show the code if it’s only 5 lines?
OP. Reranking is a specialized LLM that takes the user query, and a list of candidate results, then re-sets the order based on which ones are more relevant to the query. Here's sample code: https://docs.cohere.com/reference/rerank
What is the difference between reranking versus generating text embeddings and comparing with cosine similarity?
I concur: The big LLM-based rerankers (e.g. Qwen3-reranker) are what you always wanted your cross-encoder to be, and I highly recommend giving them a try. Unfortunately they're also quite computationally expensive. Your metadata/tabular data often contains basic facts that a human takes for granted, but which aren't repeated in every text chunk - injecting it can help a lot in making the end model seem less clueless.…
I agree completely with your point, especially the difficulty of developing the user's mental model for what's going on with context and the need to move away from chat UX. It's interesting that there are still few public examples of non-chat UIs that make context management explicit. It's possible that the big names tried this and decided it wasn't worth it -- but from comments here it seems like everyone that has built a production RAG system has come to the opposite conclusion. I'm guessing the real reason is otherwise: likely for the consumer apps controlling context (especially for free users) and inference time is one of the main levers for cost management at scale. Private RAGs, on the other hand, are more concerned with maximizing result quality and minimizing time spent by employee on a particular problem with cost per query much less of a concern --- that's been my experience at least.
I must be missing something, this says it can be self-hosted. But the first page of the self-hosting docs say you need accounts with no less than 6 (!) other third-party hosted services.
We have very different ideas about the meaning of self-hosted.
OP. Reranking is a specialized LLM that takes the user query, and a list of candidate results, then re-sets the order based on which ones are more relevant to the query. Here's sample code: https://docs.cohere.com/reference/rerank
What is the difference between reranking versus generating text embeddings and comparing with cosine similarity?
text similarity finds items that closely match. Reranking my select items that are less semantically "similar" but are more relevant to the query.
It's described in the remainder of the point - they use an LLM to generate additional search queries, either rephrasings of the user's query or bringing additional context from the chat history.
Here's an interesting read on the evolution beyond RAG: https://www.nicolasbustamante.com/p/the-rag-obituary-killed-... One of the key features in Claude Code is "Agentic Search" aka using (rip)grep/ls to search a codebase without any of the overhead of RAG. Sounds like even RAG approaches use a similar approach (Query Generation).
The article raises several interesting points, but I find its claim that Claude Code relies primarily on grep for code search unconvincing. It's clear that Claude Code can parse and reason about code structure, employing techniques far beyond simple regex matching. Since this assumption underpins much of the article's argument, it makes me question the overall reliability of its conclusions a bit.
Or am I completely misunderstanding how Claude Code works?
I must be missing something, this says it can be self-hosted. But the first page of the self-hosting docs say you need accounts with no less than 6 (!) other third-party hosted services. We have very different ideas about the meaning of self-hosted.
I consider this to be good open source and I'm a happy user of their OSS offering. Want no hosted dependencies? Then go write it all in Rust.