Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Aeneas – a Python audio/text aligner

github.com

Post reply on HN
Print view View original

1–10 of 37 posts

Re: Show HN: Aeneas – a Python audio/text aligner

#4

Post by alpe »

psobot wrote: post #2

This is super cool. I'm trying to think of common practical applications for this - would one use this to sync a script with a performance? Could this remove a lot of the work required to manually subtitle movies, TV shows, and YouTube videos?

Thank you.

Indeed several users of aeneas adopted it for producing SRT/TTML files, i.e. captions, for videos, both online and offline --- and many of them start with an existing transcript.

However, please note that there are limitations on the amount of "non speech" that aeneas can tolerate: for example, long spurious portions of audio or sung passages might affect the quality of the alignment.

For details on how aeneas works: https://github.com/readbeyond/aeneas/blob/master/wiki/HOWITW...

Re: Show HN: Aeneas – a Python audio/text aligner

#6

Post by alpe »

braindead_in wrote: post #3

What's the accuracy level of alignment?

aeneas is not based on ASR (i.e., it does not try to "recognize" words and align them with the input text), but on the "older" MFCC + DTW approach.

Hence, it is difficult to give you a precise answer, e.g. in terms of word-error-rate or similar metrics.

For the task aeneas has been designed for --- aligning an ebook and the corresponding audiobook --- and for similar tasks (e.g., captioning videos of lectures or spoken-only content), it generally produces an alignment that is indistinguishable from a manually-produced one.

If you want to see some examples, read+listen one of these audio-ebooks: the alignment has been produced by aeneas: https://www.readbeyond.it/ebooks.html

But of course if you want to align at finer level (word) or a more noisy/non-matching audio, the quality of the alignment can deteriorate.

Re: Show HN: Aeneas – a Python audio/text aligner

#8

Post by braindead_in »

alpe wrote: post #6
braindead_in wrote: post #3

What's the accuracy level of alignment?

aeneas is not based on ASR (i.e., it does not try to "recognize" words and align them with the input text), but on the "older" MFCC + DTW approach. Hence, it is difficult to give you a precise answer, e.g. in terms of word-error-rate or similar metrics. For the task aeneas has been designed for --- aligning an ebook and the corresponding audiobook --- and for similar tasks (e.g., captioning videos of lectures or spok…

Thanks for the explanation. Will it work if there are gaps in the transcript? Eg, the clean verbatim transcript where the ah's and uhm's are left out.

Re: Show HN: Aeneas – a Python audio/text aligner

#9

Post by afarrell »

This is really cool!

Would you like me to make a conda package for this? I can do so for Linux and OSX so that someone who uses python for data science can do `conda install aeneas` and it will install this and it's dependencies into a virtualenv.

I'd do it on windows too, but I don't know of an easy way to get my hands on a windows box. If anyone knows of a service that can give me 30 minutes of CLI access to a windows box, I'd be grateful.

Re: Show HN: Aeneas – a Python audio/text aligner

#10

Post by alpe »

afarrell wrote: post #9

This is really cool! Would you like me to make a conda package for this? I can do so for Linux and OSX so that someone who uses python for data science can do `conda install aeneas` and it will install this and it's dependencies into a virtualenv. I'd do it on windows too, but I don't know of an easy way to get my hands on a windows box. If anyone knows of a service that can give me 30 minutes of CLI access to a wind…

Hi, thank you.

Having it on conda would be great (https://github.com/readbeyond/aeneas/issues/158 ), so if you feel like it, it would be wonderful!

The two points that proved difficult in packaging aeneas (as self-installers and as .deb for Ubuntu/Debian) are:

1. the package must also install/require-as-dep ffmpeg and espeak 2. the package must trigger the compilation of the Python C extensions as described in setup.py

Unfortunately I am a Debian/OS X user (I do not even own a Windows machine right now), but I am told that one can use the Win10 IE VirtualBox images.

Post reply on HN