Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The business of extracting knowledge from academic publications

markusstrasser.org

Post reply on HN
Print view View original

31–40 of 122 posts

Re: The business of extracting knowledge from academic publications

#31

Post by toss1 »

And the bottom line is:

>>... nothing of it will go anywhere.

>>Don’t take that as a challenge. Take it as a red flag and run. Run towards better problems.

Wow, speaking of the value of negative results, that is hugely valuable! Could easily save person-decades of work & funds for more productive results.

The insights that the most relevant knowledge is not written into the publications (for a variety of reasons), and that the few that are are of limited use to the target audience, and even when it is useful it is a small part of the workload (i. e., not a real pain point), are key to seeing that the entire category of projects to extract & encode such knowledge is doomed.

Re: The business of extracting knowledge from academic publications

#32

Post by TOMDM »

eurasiantiger wrote: post #7
bryanph_ wrote: post #3

One thing that strikes me about most academic knowledge tools is that they seem to focus on parsing the current set of academic literature and producing supposedly interesting insights out of them (which quickly tends to snowball into wanting some kind of generalized model for knowledge as a whole). What I think is much more interesting is creating tools that help people create better academic writing in the first pl…

It likely wouldn’t take much to craft an ”arXiv Copilot” out of GitHub Copilot.

"It's well understood how to"

And

"It likely wouldn't take much to"

Are worlds apart in this case, training and deploying models on that scale is a huge investment, even if you already had all the code and cleaned training data.

Re: The business of extracting knowledge from academic publications

#33

Post by totetsu »

grlass wrote: post #5
bryanph_ wrote: post #3

One thing that strikes me about most academic knowledge tools is that they seem to focus on parsing the current set of academic literature and producing supposedly interesting insights out of them (which quickly tends to snowball into wanting some kind of generalized model for knowledge as a whole). What I think is much more interesting is creating tools that help people create better academic writing in the first pl…

I recall seeing a Show HN post a while back about a research focussed web browser that helps as a thinking tool: https://news.ycombinator.com/item?id=28446147

It's amazing what you miss on HN when you skip a day

Re: The business of extracting knowledge from academic publications

#34

Post by kwertyoowiyop »

remram wrote: post #27
jakub_g wrote: post #25

Earlier quoted context omitted.

I'm an outsider but it seems to me the difference between academia and opensource/hobby forums is massive: In opensource the attitude is "See bug? Send a PR!" Whereas academic papers are like publishing software into a blockchain (and not source but binaries, i.e. PDFs full of shortcuts): you don't want for people to easily find bugs and contribute fixes, so you handwave a lot so that no one can reproduce your exact…

The biggest difference IMHO is when comparing to something like Wikipedia or Stackoverflow. I wish the fabric of scholarly communication similarly allowed for browsing reviews, updating papers, commenting with new references, etc.

Now THAT might be a realistic technical goal & business opportunity.

Re: The business of extracting knowledge from academic publications

#35

Post by PaulHoule »

1. The real value is in operational documents such as clinical notes, maintenance records, soldier and police notebooks, etc. this info is proprietary to a organization and its partners and is directly linked to how it produces and pays for value.

2. Superhuman accuracy at limited tasks is not good enough. For instance transcribing audio at 95% word-level accuracy would be good for a human but it means every other sentence is garbled. People communicate despite this because they ask questions. A useful text-to-structured information tool has to exert back pressure on bullshit, give feedback about what it understands and push the author to tell a story that makes sense and has adequate detail.

Re: The business of extracting knowledge from academic publications

#36

Post by mellavora »

civilized wrote: post #4

Interesting but I don't know how to make sense of it. How can it be that "close to nothing of what makes science actually work is published as text on the web"? - Is the information that makes science actually work mostly in images that the machines don't yet understand? - Was the information paywalled or in private databases and inaccessible to this researcher? - Are the papers mostly just advertisements for researc…

Science is a profession like others. When you are earning your Ph.D. you learn to think about the field by reading papers and discussing with peers and colleagues, yes.

The intro of a well-structured research paper should follow this pattern:

- This is a really important topic and here is why.

- What is the current state of the art in this field? (this comes from reading 100-1000 publications on the topic and selecting the 5-10 most relevant to the next point). HOWEVER, the state of the art leaves this question unanswered.

- Here are some reasons why the idea in this paper can help answer that question (cite another 3-10 papers).

- Our hypothesis is that XXX can answer the important unanswered question (where X is derived from the prior section).

So, what I am getting at, a scientific publication is part of a conversation. When I'm citing the 5-10 papers to summarize the state of the art, I'm assuming the reader has read 50% of the 100-1000 papers which I also read, and knows where the 5-10 which I cite fit into that broader context.

So any paper, in isolation, only has a fraction of its meaning in the publication. The real information is the surrounding context.

Pro tip: if I'm reading a paper and want to understand it better, I also read one or two of the papers it cites, and one or two papers which cite it. Also, it can take a few times through before I start to understand what the author is trying to say.

Re: The business of extracting knowledge from academic publications

#37

Post by [deleted] »

remram wrote: post #27
jakub_g wrote: post #25

Earlier quoted context omitted.

I'm an outsider but it seems to me the difference between academia and opensource/hobby forums is massive: In opensource the attitude is "See bug? Send a PR!" Whereas academic papers are like publishing software into a blockchain (and not source but binaries, i.e. PDFs full of shortcuts): you don't want for people to easily find bugs and contribute fixes, so you handwave a lot so that no one can reproduce your exact…

The biggest difference IMHO is when comparing to something like Wikipedia or Stackoverflow. I wish the fabric of scholarly communication similarly allowed for browsing reviews, updating papers, commenting with new references, etc.

[deleted]

Re: The business of extracting knowledge from academic publications

#38

Post by viewfromafar »

anonymousDan wrote: post #22
hoseja wrote: post #8

Earlier quoted context omitted.

As the article states, papers are mostly career advancement tools and scientists are incentivized to put the least amount of useful information into them they can get away with. Real scientists mostly learn from their instructors who possess all the jealously guarded institutional knowledge. Yes, it is very broken.

I mean honestly this is just total bullshit. There is plenty of value in academic papers. It's just that there is very little money to be made in developing tools such as those mentioned by the OP as there is very little money in academia.

I understood the criticism directed at the value of papers as instruments of knowledge sharing. The argument is not that papers are completely useless in terms of knowledge sharing but that this pure purpose of dissemination is largely overshadowed by considerations of carreer, prestige, funding or any interest other than knowledge sharing.

This is the world we live in. A scientist is a person that needs to make a living and is subject to various constraints.

The reason that there is little money to be made is that society hasn't found a way to set up the scientific process in such a way that the constraints would value the increase in public domain knowledge higher than the incentives to hold some knowledge back.

Part of this may stem from leaving specialized knowledge to academia while letting only companies reap the monetary rewards of putting the knowledge to use. Society benefits only indirectly (better drugs, machines, etc) but industry players will rather shield knowledge and adapt its representation to their own needs.

Re: The business of extracting knowledge from academic publications

#39

Post by kkylin »

cousin_it wrote: post #16

I can confirm that in my current area of interest (how to synthesize a cello or saxophone sound), there are hundreds of academic papers published over decades, each of them says "our method sounds more realistic than others", but code and audio samples are never available, and verbal descriptions always skip crucial details. I have no doubt that academics have a ton of expertise, but their output in paper form is bas…

I'm an academic (applied math) and want to respond to this: academic papers are the way they are for lots of reasons, many of which (not so good) have been mentioned on HN. There are a couple that I do not see very often however:

(1) Many academics aren't aware non-academics read their papers at all: we work with other academics, go to conferences with other academics, and on the rare occasions we hear from readers, it's from other academics. Big exception: in some fields academia and industry have much more interaction, biomedical research (the subject of the linked article) being one of them. Extracting knowledge from that literature has a large number of practical and economic implications.

(2) There seems to be a perception that published papers are a repository of established or state-of-the-art knowledge. Perhaps they were meant to be that way, and perhaps more of them should be. But for many journals in many fields, publications are a form of moderated discussion. Reconstructing the state of knowledge from snippets of conversation is always going to be hard.

What can help make the literature more accessible? some of the forces are structural, some are due to current limitations of technology. But one thing that can help if you find the results of a paper interesting and are able to track down the authors, write them. People like hearing their work is noticed, and they like talking to people about things they're interested in.

Another is to make constructive suggestions (or even pitch in to improve code where it's open source & available). Between teaching, advising, committee work, etc (not to mention family), most of us have to prioritize, and as much as I'd like to clean up old code for release in the hopes someone finds it useful, it isn't going to get my grad students out the door with a degree or a job -- I'm generally spending more time on their research problems these days than my own. But if I know there's interest / use I might prioritize time a little differently.

Re: The business of extracting knowledge from academic publications

#40

Post by kovvy »

jakub_g wrote: post #25
cousin_it wrote: post #16

I can confirm that in my current area of interest (how to synthesize a cello or saxophone sound), there are hundreds of academic papers published over decades, each of them says "our method sounds more realistic than others", but code and audio samples are never available, and verbal descriptions always skip crucial details. I have no doubt that academics have a ton of expertise, but their output in paper form is bas…

I'm an outsider but it seems to me the difference between academia and opensource/hobby forums is massive: In opensource the attitude is "See bug? Send a PR!" Whereas academic papers are like publishing software into a blockchain (and not source but binaries, i.e. PDFs full of shortcuts): you don't want for people to easily find bugs and contribute fixes, so you handwave a lot so that no one can reproduce your exact…

Generally, anyone writing a paper about something that could benefit from bugfixes would love to accept them, but doesn't have the time or resources to actually do so - unless there's another paper in it. If they have somehow managed to find enough personal time to have a hobby project, then they probably do accept bugfixes - and you should get them in before that person burns out.
Post reply on HN