Project idea: A HackerNews or reddit client that displays summaries before you click through.
we have done just that, check it out https://news.ycombinator.com/item?id=33748363
I did too! See a link elsewhere in this thread.
Show HN: Summate.it – Quickly summarize web articles with OpenAI
31–40 of 56 posts
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#32This* was a good primer to understand the different flavours of summarization technology for end user.
It would be good if these tools labeled their capabilities as extractive or abstractive:
Extractive: extract the most relevant information from a document.
Abstractive: generate new text that captures the most relevant information.
* https://huggingface.co/docs/transformers/tasks/summarization
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#33Reminds me of the quote "Speed reading is like reading - just without comprehension"
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#34The hardest part is dealing with GPT token limits. You can't summarize long form text, where it's most needed.
there is actually a paper by OpenAI themselves on summarizing long document.
essentially, break a longer text into smaller chunks, and run a multi-stage sequential summarization. each chunk uses a trailing window of previous chunk as context, and run this recursively.
https://arxiv.org/abs/2109.10862
did a rough implementation myself, works well for articles even 20k tokens. but kind slow because all the additional overlapping runs required. (and more costly)
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#35That looks amazing, just wish "expand" would expand a bit more, right now there is not much difference between first output and expanded one.
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#36Reminds me of the quote "Speed reading is like reading - just without comprehension"
[deleted]
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#37soon writers will be writing summaries to get open ai to expand into articles which readers will have open ai summarize so they can read summaries. ais writing for other ais itsallsotiresome.gif
[deleted]
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#38Is there something like this but for pdfs?
Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#39Re: Show HN: Summate.it – Quickly summarize web articles with OpenAI
#40The hardest part is dealing with GPT token limits. You can't summarize long form text, where it's most needed.
I didn't try, but it seems GPT-Index and LangChain use techniques to get around prompting length limits?