All these AI text to voice models seem to ignore emotion. It always sounds like a robot.
Generate audiobooks from E-books with Kokoro-82M
41–50 of 255 posts
Re: Generate audiobooks from E-books with Kokoro-82M
#42On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during d…
With this technology, one could produce high quality audio books without having access to high quality narrators by annotating the books with the voice, speed and such things.
I wonder if a standardized markup exists to do so.
Re: Generate audiobooks from E-books with Kokoro-82M
#43On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during d…
I wholeheartedly agree. https://en.m.wikipedia.org/wiki/Stephen_Briggs got me hooked on Terry Pratchett's Discworld series. I loved "Going Postal".
I know someone who listened Terry Pratchett's "Wachen! Wachen!" audiobook on Spotify while living in Germany for few years. It was so well narrated that he also acquired some peculiarities of local dialects used by specific characters in the book. Locals in Bavaria were quite surprised of a foreigner speaking such language.
Re: Generate audiobooks from E-books with Kokoro-82M
#44"was trained on How the hell was it trained on that little data ?
Yeah that surprised me as well - seems low vs what is used on text llms . To be fair 100 hours of speaking is a lot of speaking though
Re: Generate audiobooks from E-books with Kokoro-82M
#45Well there was some hope with ChatGPT that people will go back to being able to process text communication.
Guess it was just a matter of time till someone figured out how to use "AI" to resume encouraging illiteracy.
Re: Generate audiobooks from E-books with Kokoro-82M
#46On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during d…
With this technology, one could produce high quality audio books without having access to high quality narrators by annotating the books with the voice, speed and such things. I wonder if a standardized markup exists to do so.
Don't end to end trained models already do this to some extent? Like raising the pitch towards a question mark, like a human would.
TortoiseTTS has a few examples under prompt engineering on their demo site: https://nonint.com/static/tortoise_v2_examples.html
Re: Generate audiobooks from E-books with Kokoro-82M
#47Well there was some hope with ChatGPT that people will go back to being able to process text communication. Guess it was just a matter of time till someone figured out how to use "AI" to resume encouraging illiteracy.
There was some hope with the rise of equestrianism that people will go back to be able to shoe horses.
Guess it was just a matter of time till someone figured out how to use "cars" to resume encouraging being unable to to a basic farrier job.
Re: Generate audiobooks from E-books with Kokoro-82M
#48On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during d…
> On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well
This (and everything else with AI) isn't saying "you don't need good actors any more". It's saying "if you don't have an audiobook, you can make a mediocre one automatically".
AI (text, images, videos, whatever) doesn't replace the top end, it replaces the entire bottom-to-middle end.
Re: Generate audiobooks from E-books with Kokoro-82M
#49On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during d…
With this technology, one could produce high quality audio books without having access to high quality narrators by annotating the books with the voice, speed and such things. I wonder if a standardized markup exists to do so.
They still wouldn't be high quality. It's just not possible to capture the precise tone of voice in an annotation, and that precision I believe really makes a difference. My experience is that the deeper the narrator understands the text and conveys that understanding, the easier it becomes for me to absorb that information.
Re: Generate audiobooks from E-books with Kokoro-82M
#50On the one hand, this is very convenient. Probably cool for some non-fiction. On the other, some of my favorite audio books all stood out because the narrator was interpreting the text really well, for example by changing the pacing during chaotic moments. Or those audiobooks with multiple narrators and different voices for each character. Not to mention that sometimes the only cue you get for who's speaking during d…
With this technology, one could produce high quality audio books without having access to high quality narrators by annotating the books with the voice, speed and such things. I wonder if a standardized markup exists to do so.
There is SSML for speech markup to indicate various characters of speech like whispers, pronunciation, pace, emphasis, etc.
With LLMs proving to be very good at generating code, it may be reasonable to assume they can get good at generating SSML as well.
Not sure if there is a more direct way to channel the interpretation of the tone/context/emotion etc from prose into generated voice qualities.
If we train some models on ebooks along with their professionally produced human-narrated audiobooks, with enough variety and volume of training data, the models might capture the essence of that human-interpretation of written text? Just maybe?
Amazon with its huge collection of Audible + Kindle library -- if it can do this without violating any rights -- has a huge corpus for this. They already have "whispersync" which is a feature that syncs text in a kindle ebook with words in corresponding audible audiobook.