Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ArXiv Papers as Audiobooks

github.com

Post reply on HN
Print view View original

31–40 of 43 posts

Re: ArXiv Papers as Audiobooks

#31

Post by hagbard_c »

pulpfictional wrote: post #20

I've been looking for a good way to TTS longer PDFs and EPUBs into recordings so I may listen to them on the go. I'd like to take advantage of high quality TTS models but I'd prefer it to be one I may host myself. Haven't found the right way yet, I'm considering: https://github.com/MycroftAI/mimic3

I use Librera Reader [1] for this, it handles both ePub as well as PDF and then some. The quality of the TTS output is dependent on what you have on your (Android) device since that is what it uses. I tend to use Google's TTS with a male UK voice which I tune down (as in deeper voice) and speed up a bit. It mostly works fine, probably better for nonfiction than fiction but that is what I mostly use it for anyway. You can swap between reading on-screen and listening since it keeps position in the document while reading aloud.

You can also have it read into an audio file is so desired which can be listened to later.

[1] https://f-droid.org/en/packages/com.foobnix.pro.pdf.reader/

Re: ArXiv Papers as Audiobooks

#32

Post by 3abiton »

chaxor wrote: post #27
Uehreka wrote: post #4

When I’ve tried listening to YouTube videos explaining, say, Attention Is All You Need, I find that I cannot do it passively at all. The first 10 or so minutes I’m nodding along, folding laundry or doing dishes, then the presenter says something like “by reifying this tensor against the priors I was just talking about, we’re able to—“ and I have to pause, rewind a couple minutes, grab a piece of paper and actually en…

There is definetly a way to make this happen though. Little bit o' whisper, Mixtral in some RAG, and you've got yourself a buddy to talk about the paper while it's reading it to you. Of course everyone will immediately say this is dangerous and it may mislead you by giving wrong explanations, etc etc. and then others will counter with 'it will definitely get better over time' (the best models as products are ~3 years…

But listening to a paper passively is the not the same thing as being mentally prepared to converse with an LLM about a dense topic. I feel the usecases are quite different, and I doubt that there is a middle ground between listening passively and learning a complex topic. But maybe I am missing something.

Re: ArXiv Papers as Audiobooks

#33

Post by bionhoward »

nicklecompte wrote: post #19
neuronexmachina wrote: post #6

The LLM prompts are pretty interesting, e.g.: https://github.com/imelnyk/ArxivPapers/blob/main/gpt/utils.p... > "You are an ArXiv paper audio paraphraser. Your primary goal is to rephrase the original paper content while preserving its overall meaning and structure, but simplifying along the way, and make it easier to understand. In the event that you encounter a mathematical expression, it is essential that you verb…

There's no way the prompt actually works, though. LLMs are not able to reliably "preserve the overall meaning" of things unless they're doing direct technical translation. The problem is going to be even worse with original research, because the LLM will try to summarize according to old ideas from blog posts / etc in its training data, and not the new ideas in the original research. In general document summarization…

Can somebody please pay me a nickel every time someone states a belief LLMs cannot perform some task or another?

Re: ArXiv Papers as Audiobooks

#34

Post by Almondsetat »

chaxor wrote: post #29
Almondsetat wrote: post #21

Earlier quoted context omitted.

I still don't get the purpose. If you have a video to watch it's not an audiobook anymore. Secondly, why not just read the abstract? The paper might contain formulas (need to be carefully read to understand) and data (need to be carefully read to understand). If you strip the paper of its scientific elements then only a series of badly justified steps remain, at which point you might as well just consider the abstrac…

What if you want to hear about the latest arxiv updates while on your morning run? This seems like a fantastic idea for that purpose.

I mean, couldn't you do it with a program that takes an RSS feed, parses the abstract from each paper and puts ot through a run of the mill TTS engine?

Re: ArXiv Papers as Audiobooks

#35

Post by Gooblebrai »

Uehreka wrote: post #4

When I’ve tried listening to YouTube videos explaining, say, Attention Is All You Need, I find that I cannot do it passively at all. The first 10 or so minutes I’m nodding along, folding laundry or doing dishes, then the presenter says something like “by reifying this tensor against the priors I was just talking about, we’re able to—“ and I have to pause, rewind a couple minutes, grab a piece of paper and actually en…

It also really depends on what type of papers. Psychology papers are very accessible to audio format as are, in general, quite comprehensive.

Re: ArXiv Papers as Audiobooks

#36

Post by Y_Y »

bionhoward wrote: post #33
nicklecompte wrote: post #19

Earlier quoted context omitted.

There's no way the prompt actually works, though. LLMs are not able to reliably "preserve the overall meaning" of things unless they're doing direct technical translation. The problem is going to be even worse with original research, because the LLM will try to summarize according to old ideas from blog posts / etc in its training data, and not the new ideas in the original research. In general document summarization…

Can somebody please pay me a nickel every time someone states a belief LLMs cannot perform some task or another?

LLMs can't give you nickels

Re: ArXiv Papers as Audiobooks

#37

Post by chaxor »

3abiton wrote: post #32
chaxor wrote: post #27

Earlier quoted context omitted.

There is definetly a way to make this happen though. Little bit o' whisper, Mixtral in some RAG, and you've got yourself a buddy to talk about the paper while it's reading it to you. Of course everyone will immediately say this is dangerous and it may mislead you by giving wrong explanations, etc etc. and then others will counter with 'it will definitely get better over time' (the best models as products are ~3 years…

But listening to a paper passively is the not the same thing as being mentally prepared to converse with an LLM about a dense topic. I feel the usecases are quite different, and I doubt that there is a middle ground between listening passively and learning a complex topic. But maybe I am missing something.

This is a bit different than the "read a paper" TTS app. I mentioned the idea just to say it's possible and coming. The blend of the two isn't out of the question though.

Think of asking for a reading of a paper wherein you could interject at any time.

System: "This work is presented fromainly 3 groups: Deepmind, University of Pennsylvania, and ETH Zurich - the authors are Matthew Botvinick, Dani Bassett, and Bastian Rieck. They uncover a useful meta-learning program that relies on an AT methodology rooted in the bifiltration of the Ricci curvature of the embeddings and training step, wherein ..." You: "Wait a second - the algebraic topology method - what are the prior works in that area and why would that be the starting point for this paper" System: "It appears that the relevant citations point to Anne Sizemore's work while in Bassett's lab, with a few other key authors such as Guisti. The titles suggest that..."

(...) System: "Now that we've cleared that up a bit (and added it to a research list for further exploration later), to continue on the paper ..."

And so on.

This is very achievable today with a little bit of work. Perhaps not easy to work _super well_ - but likely easy enough to get working to _some degree_. A well polished product that does work super well certajnly isn't out of the question though.

Re: ArXiv Papers as Audiobooks

#38

Post by chaxor »

Almondsetat wrote: post #34
chaxor wrote: post #29

Earlier quoted context omitted.

What if you want to hear about the latest arxiv updates while on your morning run? This seems like a fantastic idea for that purpose.

I mean, couldn't you do it with a program that takes an RSS feed, parses the abstract from each paper and puts ot through a run of the mill TTS engine?

Sure, I've done something similar to that in a few hours.

The free TTS options aren't great still, and "just the abstract" is not the problem. I did full articles, and the hard...er (it was a few hours) part was extracting out the relevant sections of full papers without the 'junk' info (page numbers, superscript citations, 7 pages of authors in any cern paper, etc.

So you get something, but it's often not good.

Re: ArXiv Papers as Audiobooks

#39

Post by bredren »

chaxor wrote: post #37
3abiton wrote: post #32

Earlier quoted context omitted.

But listening to a paper passively is the not the same thing as being mentally prepared to converse with an LLM about a dense topic. I feel the usecases are quite different, and I doubt that there is a middle ground between listening passively and learning a complex topic. But maybe I am missing something.

This is a bit different than the "read a paper" TTS app. I mentioned the idea just to say it's possible and coming. The blend of the two isn't out of the question though. Think of asking for a reading of a paper wherein you could interject at any time. System: "This work is presented fromainly 3 groups: Deepmind, University of Pennsylvania, and ETH Zurich - the authors are Matthew Botvinick, Dani Bassett, and Bastian…

How much would this product be worth to you?

Re: ArXiv Papers as Audiobooks

#40

Post by LeafItAlone »

adi4213 wrote: post #25
pulpfictional wrote: post #20

I've been looking for a good way to TTS longer PDFs and EPUBs into recordings so I may listen to them on the go. I'd like to take advantage of high quality TTS models but I'd prefer it to be one I may host myself. Haven't found the right way yet, I'm considering: https://github.com/MycroftAI/mimic3

If you’re an iOS user, try https://oration.app

Please don’t use HN primarily to promote your product. It is against the community guidelines.
Post reply on HN