Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: TLDR This – Auto summarize any article or webpage in a click

tldrthis.com

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Show HN: TLDR This – Auto summarize any article or webpage in a click

#51

Post by bhl »

blobster wrote: post #39

Nice landing page. If you Google "summarizer", you will find dozens of similar services for free. The mechanism behind it is very simple. A couple a years ago I built one from scratch in about 2 hours, then I accidentally deleted it and rewrote it in 15 minutes. Here's how most of them work: 1. Split the text into words 2. Rank each word based on how many times it appears in the text. For example, a word that appears…

You can use tf-idf [1] to achieve step 2 and that extra fancy part of excluding commmon articles and prepositions: count the frequency of words in the article, but divide it by the sum of frequencies from past articles.

Text summarization works as a good toy problem, because it leads to two harder problems: 1. text extraction (how to distinguish content from non-content like ads) 2. q&a (given text and a question about the text, how can you produce an answer).

[1] https://en.wikipedia.org/wiki/Tf%E2%80%93idf

Re: Show HN: TLDR This – Auto summarize any article or webpage in a click

#53

Post by txcwpalpha »

Is there any thought put into considering if this type of service is actually beneficial?

Of course on it's face it seems nice that it saves us time. But it's no secret that the reduction of complicated topics into simplified one-liners leads to less understanding and more misinformation spread.

In my opinion, this just makes that problem worse. There is often a reason that texts aren't already shorter. If the author didn't intend for you to read the details of something and instead wanted you to just read bullet points, they would have just made the bullet points themselves.

Re: Show HN: TLDR This – Auto summarize any article or webpage in a click

#56

Post by shiredude95 »

blobster wrote: post #39

Nice landing page. If you Google "summarizer", you will find dozens of similar services for free. The mechanism behind it is very simple. A couple a years ago I built one from scratch in about 2 hours, then I accidentally deleted it and rewrote it in 15 minutes. Here's how most of them work: 1. Split the text into words 2. Rank each word based on how many times it appears in the text. For example, a word that appears…

I built a similar service a while back, with a small modification to the common algorithm.

You can improve contextual summarization by splitting the x sentences into x/n buckets. Then based on the percent of article to be summarized (eg return 60% of the article), pick the sentences ranked in the top 60% of each bucket. Then do this for all the x sentences, ie top 60% across buckets, and combine them together.

This prevents the bias rising from picking a sentence with a lot of critical words.

Re: Show HN: TLDR This – Auto summarize any article or webpage in a click

#57

Post by fortran77 »

If you Google "summarizer", you will find dozens of similar services for free. A couple a years ago I built one from scratch in about 2 hours, then I accidentally deleted it and rewrote it in 15 minutes. Rank each word based on how many times it appears in the text.

Re: Show HN: TLDR This – Auto summarize any article or webpage in a click

#58

Post by mannykannot »

blobster wrote: post #39

Nice landing page. If you Google "summarizer", you will find dozens of similar services for free. The mechanism behind it is very simple. A couple a years ago I built one from scratch in about 2 hours, then I accidentally deleted it and rewrote it in 15 minutes. Here's how most of them work: 1. Split the text into words 2. Rank each word based on how many times it appears in the text. For example, a word that appears…

I agree that the effectiveness is quite surprising, given the simplicity of the analysis. It can go very wrong, however, if a significant negation is overlooked, as in cautionary tales:

... So, don't do what the late Thag Simmons did...

Maybe final paragraphs should be more highly weighted? That's often where the conclusion is.

Re: Show HN: TLDR This – Auto summarize any article or webpage in a click

#59

Post by dstein64 »

blobster wrote: post #39

Nice landing page. If you Google "summarizer", you will find dozens of similar services for free. The mechanism behind it is very simple. A couple a years ago I built one from scratch in about 2 hours, then I accidentally deleted it and rewrote it in 15 minutes. Here's how most of them work: 1. Split the text into words 2. Rank each word based on how many times it appears in the text. For example, a word that appears…

I used a similar algorithm when developing a Chrome extension a few years ago: https://chrome.google.com/webstore/detail/auto-highlight/dnk...

Longer sentences had an inherent advantage, so I controlled for that by reducing sentence scores as a function of the sentence length.

Post reply on HN