Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The business of extracting knowledge from academic publications

theseedsofscience.pub

Post reply on HN
Print view View original

1–10 of 85 posts

Re: The business of extracting knowledge from academic publications

#2

Post by Der_Einzige »

There is a lot of momentum in academic AI publishing to try to stop some of the worst of the BS that the author identifies. It's becoming normalized to publish code in a demo form available on a place like huggingface, which is massively improving reproducibility. Websites like paperswithcode have done similar to improve things.

But neither of these are enough to combat the plethora of issues that the author identifies and it makes me so sad, particularly once you get into publishing and see how insane peer review is from every corner of it (paper author, reviewer, conference chair). Academia truly feels like a cartel at times.

Re: The business of extracting knowledge from academic publications

#3

Post by VeninVidiaVicii »

As a fifth year genetics PhD student, I found myself agreeing to mostly everything here. I also think the logic applies to those of us in the field!

> Divergent Tasks are Hard to Evaluate and Reason About By “divergent” I mean loosely defined tasks where it's unclear when they're done. That includes "mapping a domain", "gathering evidence", "due diligence" and generally anything without a clear outcome like "book a hotel", "find a barber", "run this assay", "order new vials"...

This is not just a problem for outsiders, or building a map, it’s actually true. I can’t count how many projects I was told to “find some kind of meaningful evidence” for something that is super tenuous.

Re: The business of extracting knowledge from academic publications

#4

Post by Gooblebrai »

Der_Einzige wrote: post #2

There is a lot of momentum in academic AI publishing to try to stop some of the worst of the BS that the author identifies. It's becoming normalized to publish code in a demo form available on a place like huggingface, which is massively improving reproducibility. Websites like paperswithcode have done similar to improve things. But neither of these are enough to combat the plethora of issues that the author identifi…

Unfortunately AI academic publishing is not representative or the general academic publishing. Luckily, AI academics are heavily influenced by the open source culture of software engineering. It's going to be hard to see the same movement in areas like drug discovery or material science

Re: The business of extracting knowledge from academic publications

#5

Post by ttpphd »

Well done discussion of the issues. This is correct in broad strokes. Knowledge is not a static consumable transmitted in journal articles. Science is a social dance, and it's the movement and process that's important. This kind of "knowledge extraction" would be like analyzing song lyrics to try to understand why people dance.

Re: The business of extracting knowledge from academic publications

#7

Post by nextos »

I agree with the OP but only partially. UniProt is a good counterexample of a database that has been built by extracting knowledge from publications and that is incredibly useful.

But it took decades of expert hand-curators going through piles of articles to get to the current state. Also, proteomics articles report very simple outcomes that relatively easy to annotate. For instance, the subcellular localization of a given protein isoform.

Re: The business of extracting knowledge from academic publications

#8

Post by gus_massa »

Gooblebrai wrote: post #4
Der_Einzige wrote: post #2

There is a lot of momentum in academic AI publishing to try to stop some of the worst of the BS that the author identifies. It's becoming normalized to publish code in a demo form available on a place like huggingface, which is massively improving reproducibility. Websites like paperswithcode have done similar to improve things. But neither of these are enough to combat the plethora of issues that the author identifi…

Unfortunately AI academic publishing is not representative or the general academic publishing. Luckily, AI academics are heavily influenced by the open source culture of software engineering. It's going to be hard to see the same movement in areas like drug discovery or material science

In some areas you can upload the code to github (but very few people does that anyway).

In others you need a hardware like a microscope X with a lens Y and a light Z, and use cells of W cultivated with nutrients V by the graduate student U that is the only one that can keep the cells happy. You can't just git clone & config & make it.

Re: The business of extracting knowledge from academic publications

#9

Post by YossarianFrPrez »

This was a fascinating read. The well-meaning and hard-working author went through several iterations of trying to make a profit on 'academic-knowledge-graph-adjacent' products, but things ultimately fell through.

The article describes two separate things likely to appeal to HN readers. The first is that there is a lot of tacit knowledge not captured in scientific publications. The second is that the author and his team, despite best efforts, never found product-market fit.

To the first point: it wasn't until I got to graduate school that I realized that the scientific literature isn't exactly 'An accurate record of true facts.' It is instead the paper-trail of a slow-moving conversation among researchers, where old ideas are slowly jettisoned and new ideas are evaluated and tried on.

To the second point, a few reactions:

* If your primary market / audience is graduate students or post-docs, good luck! You can't sell to people who don't have money: grad students are paid around minimum wage, postdocs its slightly better. If I had to sell a science or research-adjacent product, I'd either sell to entire departments or colleges / campuses. This is likely a pretty protracted sales process and doesn't seem pleasant.

* I wonder if Law would be a better place to start. Either selling to law-firms directly (building tools for generating internal knowledge graphs when they are given 80k documents in the discovery process), or particularly for IP lawyers. IP lawyers have the money and expertise to have their skills augmented by AI-powered literature searches. I have to imagine it's not just patents they read when looking for prior art.

* I wonder if the author and his team tried to solve too broad of a problem: they never seem to have gotten hyper-specific, and built something from the bottom up.

Post reply on HN