TLDR: Extreme Summarization of Scientific Documents

#1

Post by lopespm » Fri, Nov 27, 2020, 9:53 PM UTC

TLDR: Extreme Summarization of Scientific Documents
arxiv.org

Re: TLDR: Extreme Summarization of Scientific Documents

#2

Post by behnamoh » Fri, Nov 27, 2020, 10:55 PM UTC

While there are lots of TLDR websites out there, I want to know how this one is different from them. I get it; many scientific papers are to some extent bs, and many are just wrong. For PhDs, it's a hassle to go through all of that bs to find something that is actually true. I feel like PhDs basically have to spend hundreds of hours reading papers that don't really benefit them. Tools like this could probably help with that, but as long as scientific success is measured by how may papers you've published and/or how long your papers are, I don't see any hope of actually doing science in the coming years when the academia will be essentially "saturated" with papers.

Re: TLDR: Extreme Summarization of Scientific Documents

#3

Post by DrNuke » Fri, Nov 27, 2020, 11:13 PM UTC

I am afraid we will still need humans actually going through papers to get nuances, original contributions, if any, and wider narratives? (plug: the content project in my profile)

Re: TLDR: Extreme Summarization of Scientific Documents

#4

Post by Der_Einzige » Fri, Nov 27, 2020, 11:24 PM UTC

This is very neat work, and I would never have predicted that abstractive summarization would end up advancing so much more quickly in general than extractive summarization did from transformers being introduced. Makes me wish that simple highlighting of a document at the word-level was actually a sorta "solved" (gives compelling output more often then not) problem like condensed abstractive summarization is...

Re: TLDR: Extreme Summarization of Scientific Documents

#5

Post by [deleted] » Sat, Nov 28, 2020, 12:06 AM UTC

[deleted]

Re: TLDR: Extreme Summarization of Scientific Documents

#6

Post by justarandomq » Sat, Nov 28, 2020, 12:22 AM UTC

behnamoh wrote: post #2

While there are lots of TLDR websites out there, I want to know how this one is different from them. I get it; many scientific papers are to some extent bs, and many are just wrong. For PhDs, it's a hassle to go through all of that bs to find something that is actually true. I feel like PhDs basically have to spend hundreds of hours reading papers that don't really benefit them. Tools like this could probably help wi…

You could link other TLDR tools and then we could read the paper to learn how they're different.

Re: TLDR: Extreme Summarization of Scientific Documents

#7

Post by justarandomq » Sat, Nov 28, 2020, 12:36 AM UTC

What you get from applying TLDR to their paper:

We introduce SCITLDR, a new multi-target data set of 5.4KTLDRs over 3.2Kpapers.

Keeping pdf's copy-paste artifacts:

We introduceTLDRgeneration, a new formof extreme extreme summarization, for scientific pa-pers.

Adding intro and conclusion (optional):

We introduce SCITLDR, a new data set of 5.4KTLDRs over 3.2Kpapers.

[0] https://scitldr.apps.allenai.org/

Re: TLDR: Extreme Summarization of Scientific Documents

#8

Post by pcrh » Sat, Nov 28, 2020, 12:57 AM UTC

This kind of effort serves the function of helping people to _approximately_ "know what is known", but it's really not very useful to the more important part of research efforts, which is to know what is not known.

Re: TLDR: Extreme Summarization of Scientific Documents

#9

Post by petercooper » Sat, Nov 28, 2020, 1:56 AM UTC

Oh! I listened to Scott Hanselman interview one of the authors of this paper on his podcast the other day. It might interest some of you as she explains it all in a very accessible way: https://www.hanselminutes.com/763/tldr-extreme-summarization...

Re: TLDR: Extreme Summarization of Scientific Documents

#10

Post by etaioinshrdlu » Sat, Nov 28, 2020, 2:02 AM UTC

I played around with this on the demo page and found that while the generated "TLDR" are pretty good, they tend to generate sentences composed of fragments of existing sentences. Basically, it seems vaguely extractive in nature. Never did I see it summarize a concept in new words, or try to dumb down a complicated concept further than the original paper did. Given the results of GPT3 I would think that it should be possible to do much better by now, at least with enough data and compute time.