Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ArXiv Papers as Audiobooks

github.com

Post reply on HN
Print view View original

11–20 of 43 posts

Re: ArXiv Papers as Audiobooks

#11

Post by ipsum2 »

josh-sematic wrote: post #9

https://www.listening.com/ does this as a service.FWIW I haven’t tried it myself. Edit: looks like they support a few traditional publishers as well.

Whatever they're using for text to speech is rough. Probably using an open source model. The one used in OP (Google's) is a lot more listenable.

Re: ArXiv Papers as Audiobooks

#12

Post by Almondsetat »

Papers are already difficult to process when reading them carefully multiple times, what even is the point of turning them into an audio version? I am genuinely at a loss, unless we are talking about blind people

Re: ArXiv Papers as Audiobooks

#13

Post by reader5000 »

Uehreka wrote: post #4

When I’ve tried listening to YouTube videos explaining, say, Attention Is All You Need, I find that I cannot do it passively at all. The first 10 or so minutes I’m nodding along, folding laundry or doing dishes, then the presenter says something like “by reifying this tensor against the priors I was just talking about, we’re able to—“ and I have to pause, rewind a couple minutes, grab a piece of paper and actually en…

Visual reading of dense papers also leads to failing to understand concepts or distractions.

Re: ArXiv Papers as Audiobooks

#15

Post by julienchastang »

Almondsetat wrote: post #12

Papers are already difficult to process when reading them carefully multiple times, what even is the point of turning them into an audio version? I am genuinely at a loss, unless we are talking about blind people

The YouTube Channel may shed some light. As I understand this, it is not reading the paper, but interpreting or summarizing it with visual cues as to which section it is analyzing.

Re: ArXiv Papers as Audiobooks

#16

Post by beacon294 »

Uehreka wrote: post #4

When I’ve tried listening to YouTube videos explaining, say, Attention Is All You Need, I find that I cannot do it passively at all. The first 10 or so minutes I’m nodding along, folding laundry or doing dishes, then the presenter says something like “by reifying this tensor against the priors I was just talking about, we’re able to—“ and I have to pause, rewind a couple minutes, grab a piece of paper and actually en…

I use a combined approach of pre listening then reading the technical writing later sometimes

Re: ArXiv Papers as Audiobooks

#17

Post by Nowado »

josh-sematic wrote: post #9

https://www.listening.com/ does this as a service.FWIW I haven’t tried it myself. Edit: looks like they support a few traditional publishers as well.

Last time I tried it, app literally just read papers. As in parsed arxiv pdfs text2speech. It was an awful misunderstanding of the medium. Unless it was rebuilt significantly over last months, it's just bad.

Re: ArXiv Papers as Audiobooks

#18

Post by mrkramer »

I had a similar idea but what happens when you stumble upon code, equations, tables, graphs etc.? Can LLM understand that as well?

For example; you are listening to the paper with some text2speech model and then it stumbles open code snippet or table or graph....what should happen next? Should model skip it or prompt you to look at the graph or table or whatever. Or should you write some software that tries to interpret graphs and other non-text content.

Re: ArXiv Papers as Audiobooks

#19

Post by nicklecompte »

neuronexmachina wrote: post #6

The LLM prompts are pretty interesting, e.g.: https://github.com/imelnyk/ArxivPapers/blob/main/gpt/utils.p... > "You are an ArXiv paper audio paraphraser. Your primary goal is to rephrase the original paper content while preserving its overall meaning and structure, but simplifying along the way, and make it easier to understand. In the event that you encounter a mathematical expression, it is essential that you verb…

There's no way the prompt actually works, though. LLMs are not able to reliably "preserve the overall meaning" of things unless they're doing direct technical translation. The problem is going to be even worse with original research, because the LLM will try to summarize according to old ideas from blog posts / etc in its training data, and not the new ideas in the original research. In general document summarization is one of the worst use cases for LLMs, both in terms of its reliability and the difficulty of finding errors - how would you know without reading the paper? I would be surprised if this prompt worked on a single honest[1] paper that was written after the LLM was pretrained.

The bit about translating LaTeX expressions into human-comprehensible math sentences is interesting and AFAIK should work on something like GPT-4. But that's just a case of technical translation. GPT-4 definitely cannot "rephrase the overall paper... simplifying along the way." GPT-4 can't even summarize corporate reports without screwing up facts and figures - why on earth would you try to use it to summarize new scientific research?

Stuff like this is why I'm so concerned about LLMs: this prompt doesn't work, and people using AI for this stuff is just automating ignorance. Very frustrating.

[1] I say "honest" because this prompt would probably do ok on stuff coming out of a paper mill - the problem is carefully stated original ideas. GPT tears original ideas to shreds.

Re: ArXiv Papers as Audiobooks

#20

Post by pulpfictional »

I've been looking for a good way to TTS longer PDFs and EPUBs into recordings so I may listen to them on the go. I'd like to take advantage of high quality TTS models but I'd prefer it to be one I may host myself.

Haven't found the right way yet, I'm considering: https://github.com/MycroftAI/mimic3

Post reply on HN