11–20 of 67 posts
I would have titled it "Improving ranking..." I like that it works with `sentence_transformers`
Edit: ok, done. Submitted title was "Show HN: Improving RAG with chess Elo scores".
What’s the expected additional latency due to running this re-ranker?
It actually runs pretty fast, our benchmarks show ~149ms for 12665 bytes. It's faster than many other models
For a slightly different take using a similar intuition, see our paper [at ACL 2024](https://arxiv.org/abs/2402.14860) on ranking LLMs which may be of interest.
Our HuggingFace space has some examples: https://huggingface.co/spaces/ibm/llm-rank-themselves
My questions: what languages do your models currently support? Did you perform multilingual benchmarks? Couldn't find an answer on the website
Fun fact about ELO. It's natural to think that it is some kind of initialism, but in fact ELO doesn't stand for anything. It's the name of the guy who invented the system. https://en.wikipedia.org/wiki/Arpad_Elo So don't say it "E.L.O." (unless you're talking about the band, I guess), say "ee-low"
Fun fact about ELO. It's natural to think that it is some kind of initialism, but in fact ELO doesn't stand for anything. It's the name of the guy who invented the system. https://en.wikipedia.org/wiki/Arpad_Elo So don't say it "E.L.O." (unless you're talking about the band, I guess), say "ee-low"
What was his ELO rating?
Happy to see competition in rerankers! Good luck with your product. My questions: what languages do your models currently support? Did you perform multilingual benchmarks? Couldn't find an answer on the website
Interesting work. For a slightly different take using a similar intuition, see our paper [at ACL 2024]( https://arxiv.org/abs/2402.14860 ) on ranking LLMs which may be of interest. Our HuggingFace space has some examples: https://huggingface.co/spaces/ibm/llm-rank-themselves
Earlier quoted context omitted.
It actually runs pretty fast, our benchmarks show ~149ms for 12665 bytes. It's faster than many other models
I would prominently display your benchmarks (against your competitors, of course). That's your selling point, right?
Cool stuff! We use a similar process internally to rerank and filter our cold outbound lists. We just use an off-the-shelf model as the judge, give it a custom criteria, and let it run until some set number of iterations. It's helped narrow down wide searches to the maximally relevant set of people (few thousand medium-bad matches to few hundred good matches) It's not cheap and it's not fast, but it definitely works…