Earlier quoted context omitted.
Why?
Not sure about the semantic search, but preprints are peer reviewed and hence not vetted. However, at the current pace of papers on arXiv (5k+/week) peer review alone might halt the progress.
Show HN: I made a website to semantically search ArXiv papers
81–90 of 107 posts
Re: Show HN: I made a website to semantically search ArXiv papers
#82Also, this site is not Reddit. You don't have to reply to every comment.
Re: Show HN: I made a website to semantically search ArXiv papers
#831. The option to somehow see _how_ the paper was reviewed and/or cited, if at all. There are things like OpenReview, see example [1]
2. The ability to "tell me a story to get up to speed" about a collection of papers. Generative models could help here -- but essentially, I want this thing to be able to write a paragraph for what one might find in the literature review / related work of a paper, with citations. :-)
All the best!
Re: Show HN: I made a website to semantically search ArXiv papers
#84I enjoy seeing projects like this! If you expand beyond arxiv, keep in mind since coverage matters for lit reviews, unfortunately the big publishers (Elsevier and Springer) are forcing other indices like OpenAlex, etc. to remove abstracts so they're harder to get. Have you checked out other tools like undermind.ai, scite.ai, and elicit.org? You might consider what else a dedicated product workflow for lit reviews inc…
Re: Show HN: I made a website to semantically search ArXiv papers
#85Re: Show HN: I made a website to semantically search ArXiv papers
#86I enjoy seeing projects like this! If you expand beyond arxiv, keep in mind since coverage matters for lit reviews, unfortunately the big publishers (Elsevier and Springer) are forcing other indices like OpenAlex, etc. to remove abstracts so they're harder to get. Have you checked out other tools like undermind.ai, scite.ai, and elicit.org? You might consider what else a dedicated product workflow for lit reviews inc…
The Cloudflare challenge screen at the beginning is a dealbreaker.
Random question - does anyone know why so many papers are missing from ArXiv? Do they need to be submitted manually, perhaps by their author(s)? I'll often find papers on mathematics, physics and computer science. But papers on biology, chemistry and medicine are usually missing.
I think a database of all paper ids in existence and where they're posted or missing could be at least as useful as this. Because no papers written with any level of public funding (meaning most of them) should ever be missing.
Re: Show HN: I made a website to semantically search ArXiv papers
#87This is great! I just tried some queries and the results were pretty decent, in terms of semantics. But, just thinking of it as a user, if this were to be part of my daily workflow (instead of say something like Google Scholar), I would like: 1. The option to somehow see _how_ the paper was reviewed and/or cited, if at all. There are things like OpenReview, see example [1] 2. The ability to "tell me a story to get up…
2. This is good feedback, making models write the Introduction section! I was planning to keep this search engine a little more traditional, however if the results are good, then it should be the way forward.
Thank you, Happy Holidays! :D
Re: Show HN: I made a website to semantically search ArXiv papers
#88This is great! I just tried some queries and the results were pretty decent, in terms of semantics. But, just thinking of it as a user, if this were to be part of my daily workflow (instead of say something like Google Scholar), I would like: 1. The option to somehow see _how_ the paper was reviewed and/or cited, if at all. There are things like OpenReview, see example [1] 2. The ability to "tell me a story to get up…
1. I was not aware of OpenReview. I love the transparency and would definitely look into integrating it. 2. This is good feedback, making models write the Introduction section! I was planning to keep this search engine a little more traditional, however if the results are good, then it should be the way forward. Thank you, Happy Holidays! :D
Re: Show HN: I made a website to semantically search ArXiv papers
#89Compare that to scholar which returns all relevant results:
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=leak...
You might want to retrain/finetune your own embedding model instead of using a general-purpose one.