11–20 of 57 posts
Great article, I’ve had similar findings! LLM based “document-chunk” ranking is a core feature of PaperQA2 ( https://github.com/Future-House/paper-qa ) and part of why it works so well for scientific Q&A compared to traditional embedding-ranking based RAG systems.
A concept that I've been thinking about a lot lately: transforming complex problems into document ranking problems to make them easier to solve. LLMs can assist greatly here, as I demonstrated at inaugural DistrictCon this past weekend.
So would this be 1600 commits and one of which fixes the bug (which might be easier with commit messages?) or is this a diff between two revisions, with 1600 chunks, each chunk a “document” ? I am trying to grok why we want to find the fix - is it to understand what was done so we can exploit unpatched instances in the wild? Also also “identifying candidate functions for fuzzing targets“ - if every function is a docu…
This furthers an idea I've had recently that we (and the media) are focusing too much on creating value by making more ever more complex LLMs, and instead we are vastly underestimating creative applications of current generation AI.
Should be "document ranking reduces to these hard problems",
I never knew why the convention was like that, it seems backwards to me as well, but that's how it is.
I'm curious - why is LLM ranking preferred over cosine similarity from an embedding model (in the context of this specific problem)?
Minor nitpick, Should be "document ranking reduces to these hard problems", I never knew why the convention was like that, it seems backwards to me as well, but that's how it is.
Minor nitpick, Should be "document ranking reduces to these hard problems", I never knew why the convention was like that, it seems backwards to me as well, but that's how it is.
"Document ranking reduces to these hard problems" would imply that document ranking is itself an instance of a certain group of hard problems. That's not what the article is saying.