Live data from Hacker News

Show HN: I mapped 8.5M research papers into an interactive atlas

tomesphere.com

1–10 of 33 posts

Show HN: I mapped 8.5M research papers into an interactive atlas

#1
When I read papers, I have to jump between multiple tabs to find the dataset, code, videos, peer reviews, and so on. I tried to fix this with this project.

It started as a project just for papers on arXiv, but after its initial success on Twitter (got like 1.9k views: the most I have gotten for a post), I have now expanded it to include other openly available papers from PubMed Central, bioRxiv, medRxiv, and eLife. These papers have been linked with their genes, proteins, diseases, drugs, clinical trials, 3D protein structures, code, and cited and similar papers.

This project now has four parts:

First, a map. I embedded nearly 8.5M papers (with SPECTER2), ran UMAP for 2D representation, and rendered them as a scatterplot. The dots can be clicked to see brief information about the papers, like an LLM TLDR, key findings, peer reviews, linked entities, and more. The clusters are also labeled, though you might have to zoom in.

Second, I built a detailed paper page for each paper. They give you the paper's full text, images, videos, peer reviews (from OpenReview), GitHub links, Hugging Face dataset/model links, clinical trials, genes, diseases, 3D protein structures, cited papers, and similar papers. You can also copy the whole page, including the full paper text and image URLs, as markdown for your LLM.

Third, I have released an extension so you can read all this information in your sidebar by clicking "open in Tomesphere" that shows up in arXiv, PMC, bioRxiv, Google Scholar, or medRxiv. I have tried to provide as much information as possible in the extension, though for things like viewing all the images or a 3D protein structure, you might still have to go to the paper page using the link provided in the extension.

Fourth, all this data is available for your LLM via MCP. The MCP does have a 50-query free limit (this jumps 10x with signup).

Note: this project is still in beta, so papers might have some mismatched information. I am rolling out feedback forms soon to improve the data quality. Thank you so much for taking the time to read this.

Show HN: I mapped 8.5M research papers into an interactive atlas
tomesphere.com

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#4
Neat! Two questions I had after using it:

1) Is there a way to filter the visual atlas by the search term? For instance, I searched "ribosome" and it gave me a list, but I couldn't seem to visualize the list

2) I notice there's an MCP tool. I've used https://paperclip.gxl.ai/ in the past to good effect, curious if there are any standout features from tomesphere?

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#5
post #3

I remember similar kind of visualization from a decade ago, called paperscape. Looked cool, worked on clustering using citations and references. Never got any idea on any use case that would be covered by such visualizations, apart from looking cool.

That's usually the case with graph visualizations or clustering for networks, imo (beyond revealing obvious statistics(

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#6
post #5
post #3

I remember similar kind of visualization from a decade ago, called paperscape. Looked cool, worked on clustering using citations and references. Never got any idea on any use case that would be covered by such visualizations, apart from looking cool.

That's usually the case with graph visualizations or clustering for networks, imo (beyond revealing obvious statistics(

I love them! It's a really nice, fun way to explore a corpus. Cosmograph for this sort of thing is great, it supports graphs as well as 2D projections, and is blazing fast.

That said, I've never had a client or stakeholder show any interest in using one, beyond an initial "that's cool".

And UMAP etc., is just as much an art as a science. You'll go mad trying to get the perfect layout.

Great toy if you're into that sort of thing, but yeah, fiddly and overwhelming for most.

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#7
post #3

I remember similar kind of visualization from a decade ago, called paperscape. Looked cool, worked on clustering using citations and references. Never got any idea on any use case that would be covered by such visualizations, apart from looking cool.

ResearchRabbit is free and has this feature!

https://www.researchrabbit.ai/

ConnectedPapers also has this but they started to limit unless you pay:

https://www.connectedpapers.com/

A few other ones I know of:

https://litmaps.com

https://consensus.app/home/features/citation-graph/

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#8
post #3

I remember similar kind of visualization from a decade ago, called paperscape. Looked cool, worked on clustering using citations and references. Never got any idea on any use case that would be covered by such visualizations, apart from looking cool.

Hello, I agree with you, viz are just cool and might not really have a usecase. In this project map is not the product, it is 1 of 4 parts and to be honest the least important. The value is what is under each dot, the enriched page (TLDR, genes/drugs/diseases, trials, protein structures, code, datasets, full text, images, reviews, etc) and the MCP for agents. You are welcome to use whichever part of the project is most useful to you (whether that is the map, paper pages, browser extension, or MCP).

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#9
post #6
post #5

Earlier quoted context omitted.

That's usually the case with graph visualizations or clustering for networks, imo (beyond revealing obvious statistics(

I love them! It's a really nice, fun way to explore a corpus. Cosmograph for this sort of thing is great, it supports graphs as well as 2D projections, and is blazing fast. That said, I've never had a client or stakeholder show any interest in using one, beyond an initial "that's cool". And UMAP etc., is just as much an art as a science. You'll go mad trying to get the perfect layout. Great toy if you're into that so…

Hi, I love the genre too. Cosmograph is wonderful, I did try it, but because of its license restriction I could not use it for this project. I do agree that beyond an initial "that's cool" this map may not contribute much, "and that's why I didn't make it the main product. I already had the data as I was building other things (extension, paper page) and wanted a bit of a cool factor so people would take a look at the project. The value is what's under each dot, the enriched page (TLDR, genes/drugs/diseases, trials, 3D structures, code, datasets, full text), extension and the MCP for agents.

Re: Show HN: I mapped 8.5M research papers into an interactive atlas

#10
post #3

I remember similar kind of visualization from a decade ago, called paperscape. Looked cool, worked on clustering using citations and references. Never got any idea on any use case that would be covered by such visualizations, apart from looking cool.

Hello, I agree with you, viz are just cool and might not really have a usecase. In this project map is not the product, it is 1 of 4 parts and to be honest the least important. The value is what is under each dot, the enriched page (TLDR, genes/drugs/diseases, trials, protein structures, code, datasets, full text, images, reviews, etc) and the MCP for agents. You are welcome to use whichever part of the project is mo…

[deleted]
Post reply on HN