Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The business of extracting knowledge from academic publications

markusstrasser.org

Post reply on HN
Print view View original

41–50 of 122 posts

Re: The business of extracting knowledge from academic publications

#41

Post by mellavora »

hoseja wrote: post #8
civilized wrote: post #4

Interesting but I don't know how to make sense of it. How can it be that "close to nothing of what makes science actually work is published as text on the web"? - Is the information that makes science actually work mostly in images that the machines don't yet understand? - Was the information paywalled or in private databases and inaccessible to this researcher? - Are the papers mostly just advertisements for researc…

As the article states, papers are mostly career advancement tools and scientists are incentivized to put the least amount of useful information into them they can get away with. Real scientists mostly learn from their instructors who possess all the jealously guarded institutional knowledge. Yes, it is very broken.

Hard disagree. With a caveat -- I do acknowledge that for an important number of professional academics your statement may be true, and I have heard a former post-doc at ETH Zurich describe their papers as career points (so also a grain of truth at elite institutes).

But for most of the academics I have known and worked with, publications are taken quite seriously, and institutional knowledge is freely shared. There is an incentive to reduce the content in papers, but it is out of respect for the reader (a paper is not a textbook) and an honest attempt to limit the discussion to the core hypothesis of the work. You have 6 pages to 1) describe the content of 6*100 pages (the 100 other relevant papers on the topic), 2) present your addition to this body of knowledge, 3) discuss the insights your work brings, again referring to the content of 600 pages.

and those 600 pages you are summarizing are as information-dense as your work.

Re: The business of extracting knowledge from academic publications

#42

Post by [deleted] »

jakub_g wrote: post #25
cousin_it wrote: post #16

I can confirm that in my current area of interest (how to synthesize a cello or saxophone sound), there are hundreds of academic papers published over decades, each of them says "our method sounds more realistic than others", but code and audio samples are never available, and verbal descriptions always skip crucial details. I have no doubt that academics have a ton of expertise, but their output in paper form is bas…

I'm an outsider but it seems to me the difference between academia and opensource/hobby forums is massive: In opensource the attitude is "See bug? Send a PR!" Whereas academic papers are like publishing software into a blockchain (and not source but binaries, i.e. PDFs full of shortcuts): you don't want for people to easily find bugs and contribute fixes, so you handwave a lot so that no one can reproduce your exact…

[deleted]

Re: The business of extracting knowledge from academic publications

#43

Post by [deleted] »

cousin_it wrote: post #16

I can confirm that in my current area of interest (how to synthesize a cello or saxophone sound), there are hundreds of academic papers published over decades, each of them says "our method sounds more realistic than others", but code and audio samples are never available, and verbal descriptions always skip crucial details. I have no doubt that academics have a ton of expertise, but their output in paper form is bas…

[deleted]

Re: The business of extracting knowledge from academic publications

#44

Post by geoduck14 »

urthor wrote: post #23
civilized wrote: post #6

Earlier quoted context omitted.

To your point but even more general: the ML/AI space is far too focused on replacing people rather than helping people. There is a suffocating cultural conceit that we are on the verge of general AI and oh my gosh what will the humans do, we better institute universal basic income right away, etc. What a joke. Try to help humans think better first. If you succeed at that, you might be on the right track towards devel…

Unfortunately you'll run into the rapid fact that in the ML/AI space you get almost zero points for building something. You get a whole lot of points for discovering something, designing something, or a proof. But there's a very large amount of people focused entirely on aims that are very, very distant from actually making human lives genuinely better. Mostly because everyone quietly understands all the extraordinar…

>But there's a very large amount of people focused entirely on aims that are very, very distant from actually making human lives genuinely better.

I can't speak to each and every person working on ML, but I thought I would share a fun use case I ran across the other day.

There is a business in some foreign country that is similar to Uber Eats: customer goes to an app, browses for food from various restaurants, orders, it gets delivered.

The business was using ML to help the restaurants: the restaurants upload a pic of the dishes, a title, and a description (usually all from an existing menu). The business would parse the description to guess at what was in the dish. Scan the picture to guess at the quantity of food (entre, side, desert, etc). Compare ingredients against publicly available nutrition info. Now the end consumer can do things like: search for gluten free, vegetarian, pork free, Almost all of this was "possible" before, but it would have required enormous effort from the restaurants inputting the data or customers reading each item. Now it is "easy", and it actually helps the end customers - and the restaurants.

Re: The business of extracting knowledge from academic publications

#45

Post by rm445 »

civilized wrote: post #4

Interesting but I don't know how to make sense of it. How can it be that "close to nothing of what makes science actually work is published as text on the web"? - Is the information that makes science actually work mostly in images that the machines don't yet understand? - Was the information paywalled or in private databases and inaccessible to this researcher? - Are the papers mostly just advertisements for researc…

The 'what makes science work' is stored in the scientists.

They learn by reading the literature, but also by communicating, and by an active process of testing their own understanding and resolving gaps and inconsistencies. Even when a self-taught genius like Ramanujan comes along, they benefit from being brought into the community.

The question of how one would determine the state of the art in a field has an answer, but at present it would be indistinguishable from training a scientist, rather than running a clever software tool that could synthesize from the literature.

Re: The business of extracting knowledge from academic publications

#46

Post by viewfromafar »

Al-Khwarizmi wrote: post #19
civilized wrote: post #4

Interesting but I don't know how to make sense of it. How can it be that "close to nothing of what makes science actually work is published as text on the web"? - Is the information that makes science actually work mostly in images that the machines don't yet understand? - Was the information paywalled or in private databases and inaccessible to this researcher? - Are the papers mostly just advertisements for researc…

I can't speak for biomedicine, but speaking as an academic in CS the claim that "close to nothing of what makes science actually work is published as text on the web" looks like a huge hyperbole to me. It's true that the so-called "folk knowledge", knowledge that exists in the community but no one bothers to publish in the form of papers, is a real problem, but at least in my field, it's by no means the majority of k…

Also CS, my interpretation of "what makes science work" is a little different and I would argue that - despite a lot of foundations and techniques being shared in research papers - this field more than any other is constraining the free circulation and application of knowledge.

The equivalent to those biomedical industry players are the big tech who develop closed source and push the edge in some area. They will publish but that does not mean you can replicate any of it.

Software is also fragmented, crippled by IP lawsuits, patent trolls and so on. This does inhibit ability of society to benefit from software since it depends on the private sector to sort things out. The PhDs go and build businesses to "make the science work" in that sense.

The ideal of detached pursuit of knowledge is not a complete fiction (despite the hyperbole), but it does remain an ideal that can only be approximated.

Re: The business of extracting knowledge from academic publications

#47

Post by geoduck14 »

i000 wrote: post #11

When I was in grad school, I joined a startup incubator and build a prototype which combined two of the tools mentioned in the article: "a query builder (by demonstration)" and "A paper recommender system", a simple companion which would help scientist to not miss relevant research to them. This was 10 years ago, before Google Scholar has similar features. The incubator introduced me to advisors with business experie…

>And I got told in no uncertain terms what is the gist of this article: The value lies in the molecular and clinical data. In 2021 I would add digital pathology / imaging data.

I feel like you are trying to tell me something REALLY valuable, but I don't quite understand it. Can you please elaborate?

Re: The business of extracting knowledge from academic publications

#48

Post by Fomite »

kovvy wrote: post #40
jakub_g wrote: post #25

Earlier quoted context omitted.

I'm an outsider but it seems to me the difference between academia and opensource/hobby forums is massive: In opensource the attitude is "See bug? Send a PR!" Whereas academic papers are like publishing software into a blockchain (and not source but binaries, i.e. PDFs full of shortcuts): you don't want for people to easily find bugs and contribute fixes, so you handwave a lot so that no one can reproduce your exact…

Generally, anyone writing a paper about something that could benefit from bugfixes would love to accept them, but doesn't have the time or resources to actually do so - unless there's another paper in it. If they have somehow managed to find enough personal time to have a hobby project, then they probably do accept bugfixes - and you should get them in before that person burns out.

It also doesn't happen enough to design for - I once presented a fairly open-source contributor friendly project at SciPy that I hoped would be compelling (it was about modeling the zombie epidemic), actively asked for help, had set up a couple open requests of varying levels of complexity.

I think there was one pull request total?

The juice just didn't end up being worth the squeeze.

Re: The business of extracting knowledge from academic publications

#49

Post by dekhn »

cousin_it wrote: post #16

I can confirm that in my current area of interest (how to synthesize a cello or saxophone sound), there are hundreds of academic papers published over decades, each of them says "our method sounds more realistic than others", but code and audio samples are never available, and verbal descriptions always skip crucial details. I have no doubt that academics have a ton of expertise, but their output in paper form is bas…

Hey, that unusability of papers is a form of job security.

Seriously though, you're totally right. I got very dissatisfied with science when I realized that many people were effectively publishing unreproducible crap created by terrible code. Fortunately, more and more people are learning how to recognize the crap.

Re: The business of extracting knowledge from academic publications

#50

Post by Mezzie »

remram wrote: post #27
jakub_g wrote: post #25

Earlier quoted context omitted.

I'm an outsider but it seems to me the difference between academia and opensource/hobby forums is massive: In opensource the attitude is "See bug? Send a PR!" Whereas academic papers are like publishing software into a blockchain (and not source but binaries, i.e. PDFs full of shortcuts): you don't want for people to easily find bugs and contribute fixes, so you handwave a lot so that no one can reproduce your exact…

The biggest difference IMHO is when comparing to something like Wikipedia or Stackoverflow. I wish the fabric of scholarly communication similarly allowed for browsing reviews, updating papers, commenting with new references, etc.

I think this is a valuable idea. There are online archives that allow for paper updating for academics, like SSRN, but as a CONSUMER of academic literature, the land is pretty barren.

The difficulty in such a thing would be the journals and database companies are holding on to their exclusivity and profit motives with an iron fist, so unless you want to get sued into oblivion, you'd have to stick with open source or accessible articles, so you'd need to either specialize in disciplines that have moved away from closed-source enough that the tool wouldn't have massive holes in it.

Also determining which new references and reviews have relevance (like if anybody can comment with new references, who goes through to check they're actually relevant or say what the person says they say?), preventing academics/administrators from gaming the system if it DOES get popular, etc. In open source, this is crowd-sourced, but for some academic fields the number of people who are qualified to speak on a matter is extremely small.

/academic librarian thoughts

Post reply on HN