TLDR: Extreme Summarization of Scientific Documents • Hacker News Board

Re: TLDR: Extreme Summarization of Scientific Documents

#21

Post by justarandomq » Sat, Nov 28, 2020, 6:53 AM UTC

ultrastito wrote: post #12

That's what abstract are for

After some thought I agree with you that this is the wrong problem to solve.

I took a narrative detour I wanted to share:

Suppose we make the analogue of a scientific paper to a piece of mineral ore (in terms of their raw content, and without written symbols in them for the sake of the analogy) extracted from some mine or quarry. This ore is somehow useful to someone, even if its value is structural: the shingles on an academic roof or a heavyweight desk. What a summarizer attempts to do is use a generic refinement process that will grind up the ore and then separate the components of interest such as Iron, Uranium, or Gold.

Anyone thinking that all of metallurgy reduces to simply throwing the slab into a machine and have it spew out the precious metals will find, instead, more complexity than they bargained for, and have more questions on machines or methods to resolve. Gold, Iron, Uranium, all have different extraction process.

I believe this approach may give some insight in what problems to solve instead with AI: focus on those discoveries that have helped advance "metallurgy", those of discovering and understanding the structure of the mineral ore and contents (scientific papers) and their relation with current technologies at the time, not on the philosopher's stone of 'summarizing' process more akin to a hammer that makes everything seem like a nail.

Re: TLDR: Extreme Summarization of Scientific Documents

#22

Post by bjornsing » Sat, Nov 28, 2020, 7:37 AM UTC

visarga wrote: post #19

behnamoh wrote: post #2

While there are lots of TLDR websites out there, I want to know how this one is different from them. I get it; many scientific papers are to some extent bs, and many are just wrong. For PhDs, it's a hassle to go through all of that bs to find something that is actually true. I feel like PhDs basically have to spend hundreds of hours reading papers that don't really benefit them. Tools like this could probably help wi…

Never saw a paper lauded for its long length. Maybe the authors would have written shorter papers but they didn't have enough time.

No? Turing 1936 is well known as a lengthy paper.

(Just kidding.)

Re: TLDR: Extreme Summarization of Scientific Documents

#23

Post by pixiemaster » Sat, Nov 28, 2020, 8:01 AM UTC

English only, as usual for NLG

Re: TLDR: Extreme Summarization of Scientific Documents

#24

Post by thorncorona » Sat, Nov 28, 2020, 8:17 AM UTC

Very cool.

Consider the paper's abstract: "We introduce TLDR generation, a new form of extreme summarization, for scientific papers. TLDR generation involves high source compression and requires expert background knowledge and understanding of complex domain-specific language. To facilitate study on this task, we introduce SciTLDR, a new multi-target dataset of 5.4K TLDRs over 3.2K papers. SciTLDR contains both author-written and expert-derived TLDRs, where the latter are collected using a novel annotation protocol that produces high-quality summaries while minimizing annotation burden. We propose CATTS, a simple yet effective learning strategy for generating TLDRs that exploits titles as an auxiliary training signal. CATTS improves upon strong baselines under both automated metrics and human evaluations. Data and code are publicly available at this https URL."

The algorithm summarizes it as:

“We introduce TLDR generation, a new form of extreme summarization, for scientific papers that produces high-quality summaries while minimizing annotation burden.”

Re: TLDR: Extreme Summarization of Scientific Documents

#25

Post by R0b0t1 » Sat, Nov 28, 2020, 8:34 AM UTC

ultrastito wrote: post #12

That's what abstract are for

Some abstract too big, author word smaller good

Re: TLDR: Extreme Summarization of Scientific Documents

#26

Post by anonymousDan » Sat, Nov 28, 2020, 9:10 AM UTC

This is potentially a godsend for me - I was faced with having to write a 1 paragraph summary of 70 student dissertations for an accreditation process next week. Abstracts are too long. Fingers crossed it works as advertised!

Re: TLDR: Extreme Summarization of Scientific Documents

#27

Post by mschuetz » Sat, Nov 28, 2020, 9:32 AM UTC

ultrastito wrote: post #12

That's what abstract are for

Abstracts nowadays are way too large, bordering on a full blown introduction. I agree that's what they should be for, but they are not, in practice.

Re: TLDR: Extreme Summarization of Scientific Documents

#28

Post by johnnujler » Sat, Nov 28, 2020, 9:46 AM UTC

Not sure what I was expecting. It gave me back the first line of the abstract as response. (For anyone wondering, the paper I tried was: "A Heterarchy of values determined by the topology of nervous nets" by Warren S. McCulloch.)

Re: TLDR: Extreme Summarization of Scientific Documents

#29

Post by m1sta_ » Sat, Nov 28, 2020, 10:03 AM UTC

justarandomq wrote: post #7

What you get from applying TLDR to their paper: We introduce SCITLDR, a new multi-target data set of 5.4KTLDRs over 3.2Kpapers. Keeping pdf's copy-paste artifacts: We introduceTLDRgeneration, a new formof extreme extreme summarization, for scientific pa-pers. Adding intro and conclusion (optional): We introduce SCITLDR, a new data set of 5.4KTLDRs over 3.2Kpapers. [0] https://scitldr.apps.allenai.org/

Yikes

Re: TLDR: Extreme Summarization of Scientific Documents

#30

Post by austinjp » Sat, Nov 28, 2020, 10:46 AM UTC

ultrastito wrote: post #12

That's what abstract are for

Everybody * scans papers for the piece of info they happen to be searching for, before reading it in any detail. The abstract should contain that, but might not. And nobody * reads the entire abstract anyway.

A clinician scanning a medical paper is looking for patient relevance: should they use the approach described? The statistical details are too intimidating, the preamble is irrelevant, they know the scope of the problem already.

This is not what "should" happen, but it is what actually happens.

The gap between published findings and clinical practice is several years. The peer review and publication process are way out of touch with clinical reality.

On top of this, people find articles using Google and read them on their phones. (In reality, they read summarised opinion pieces found via Google.)

A systematic reviewer may read papers in full. But even they scan papers for inclusion/exclusion criteria first. The deeper the information is buried, the greater the risk of misclassification. I'm not suggesting that TLDRs will fix this, it's just another data point in why we're seeing TLDRs being created.

* "Everybody" and "nobody" here excludes researchers :)