Just curious where do you source all those papers?
Show HN: I mapped 8.5M research papers into an interactive atlas
11–20 of 33 posts
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#12Neat! Two questions I had after using it: 1) Is there a way to filter the visual atlas by the search term? For instance, I searched "ribosome" and it gave me a list, but I couldn't seem to visualize the list 2) I notice there's an MCP tool. I've used https://paperclip.gxl.ai/ in the past to good effect, curious if there are any standout features from tomesphere?
The project has four parts, and I think you may have used the search in the navigation bar. That search is for the paper page (detailed information about that specific paper). When you search, you get a list of results, and if you click one of them, it takes you to a paper page with all the in-depth details about that paper. I should have made the map UX better.
To highlight things in the atlas or map, you might want to go to the filter panel on the left side and scroll down a bit. You will see a search area that helps you search for genes, diseases, and proteins. However it might not highlight any dot for the “ribosome” because filter search for now is only connected to genes, diseases and protein. I noted this, and I will improve it. I may also move that filter search to a different place. Thank you so much.
Tomesphere includes web pages and a browser extension overlays all of this directly on the arXiv, PMC, bioRxiv, Google Scholar, and medRxiv pages you are already reading. I had noticed the Paperclip MCP before, and from what I can see, they have very good data. In some cases, they may even have better data. We also have some additional sources, such as peer review from OpenReview, video links from YouTube and SlidesLive, GitHub links, AlphaFold protein entities, citations, and semantic neighbors.
Thank you for the questions. Will improve the project more.
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#13Just curious where do you source all those papers?
It is all open, bulk-downloadable sources. Here are some links.
- arXiv: bulk on S3 (https://info.arxiv.org/help/bulk_data_s3.html)
- PubMed Central Open Access: AWS Open Data (s3://pmc-oa-opendata).
- bioRxiv / medRxiv: their monthly S3 dumps
- OpenAlex: for citations, metadata, abstracts (https://developers.openalex.org/download/download-to-machine)
- AlphaFold — structures (https://alphafold.ebi.ac.uk/download)
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#14this is fun! even for casual users. I searched "James Webb" and got 'James Webb Space Telescope: data, problems, and resolution
A. D. Dolgov · 2023 · 1 cites' as my first result, but clicking it didn't do anything - I expected it to either zoom into or see the article contents. Was I doing something wrong? (Chrome, OSX).
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#15This is amazing!
Would love to collaborate on a position paper I've been authoring if you have bandwidth
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#16I have a few questions:
1. I see there is a Chrome extension, but I have not used Chrome since their adblocker blocker announcement. Is a Firefox extension planned?
2. What is the business model? Is this an open beta of a paid product? If this is not a product, will the code be released at some point?
3. It would be helpful to be able to filter papers by institution or author. Is this planned or out of scope?
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#17I remember similar kind of visualization from a decade ago, called paperscape. Looked cool, worked on clustering using citations and references. Never got any idea on any use case that would be covered by such visualizations, apart from looking cool.
ResearchRabbit is free and has this feature! https://www.researchrabbit.ai/ ConnectedPapers also has this but they started to limit unless you pay: https://www.connectedpapers.com/ A few other ones I know of: https://litmaps.com https://consensus.app/home/features/citation-graph/
I also made my own variant recently. I really liked the idea of litmaps but I didn't really like the UI/UX or "graph expand" feature (it uses some internal heuristic that is not very clear).
https://paper-explorer.aziis98.com/ (also on github https://github.com/aziis98/paper-explorer)
This uses OpenAlex as a source of articles and to let you explore the citation graph of papers. This is still a prototype mostly made to test out how far I could go with vibecoding (well I still checked the code now and then) something without a js framework. Someday I think I will add more features to it, but now there is already a somewhat working version of import/export so I'm fine with it.
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#18Very cool. By the way, you can render many more datapoints on mobile if you use WebGL. Here’s a similar example - embeddings rendered using a T-SNE layout
https://blog.scottlogic.com/2021/10/15/efficiently-loading-m...
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#19Very cool. By the way, you can render many more datapoints on mobile if you use WebGL. Here’s a similar example - embeddings rendered using a T-SNE layout https://blog.scottlogic.com/2021/10/15/efficiently-loading-m...
Will check it, it will be really useful. Thanks
Re: Show HN: I mapped 8.5M research papers into an interactive atlas
#20this is fun! even for casual users. I searched "James Webb" and got 'James Webb Space Telescope: data, problems, and resolution A. D. Dolgov · 2023 · 1 cites' as my first result, but clicking it didn't do anything - I expected it to either zoom into or see the article contents. Was I doing something wrong? (Chrome, OSX).
I am sorry, it should have taken you to the paper page/article, I will investigate it, but I did the same search and clicked on the first link, it to me to this - https://tomesphere.com/paper/2301.01365
Thanks for letting me know.