StyleTTS2 – open-source Eleven-Labs-quality Text To Speech

Post by regularfry » Mon, Nov 20, 2023, 7:19 PM UTC

modeless wrote: post #108

lucubratory wrote: post #104

Earlier quoted context omitted.

How hard on your end does the task of making the chatbot converse naturally look? Specifically I'm thinking about interruptions, if it's talking too long I would like to be able to start talking and interrupt it like in a normal conversation, or if I'm saying something it could quickly interject something. Once you've got the extremely high speed, theoretically faster than real time, you can start doing that stuff ri…

Yes, I implemented the ability to interrupt the chatbot while it is talking. It wasn't too hard, although it does require you to wear headphones so the bot doesn't hear itself and get interrupted. The other way around (bot interrupting the user) is hard. Currently the bot starts processing a response after every word that the voice recognition outputs, to reduce latency. When new words come in before the response is…

Post by stavros » Mon, Nov 20, 2023, 9:02 PM UTC

wingworks wrote: post #105

wczekalski wrote: post #52

Earlier quoted context omitted.

have you tested longer utterances with both ElevenLabs and with StyleTTS? Short audio synthesis is a ~solved problem in the TTS world but things start falling apart once you want to do something like create an audiobook with text to speech.

I can say that the paid service from ElevenLabs can do long form TTS very well. I used it for a while to convert long articles to voice to listen to later instead of reading. It works very well. I only stopped because it gets a little pricey.

Post by rahimnathwani » Mon, Nov 20, 2023, 11:17 PM UTC

lhl wrote: post #223

rahimnathwani wrote: post #181

Earlier quoted context omitted.

Thanks. Following the instructions now. BTW mamba is no longer recommended (for those like me who aren't already using it), and the #mambaforge anchor in the link didn't work.

I switched from conda to mamba a while ago and never looked back (it's probably saved dozens of hours from waiting for conda's slow as molasses package resolution). I'm looking at the latest docs and it doesn't look like there's any deprecation messages or anything (it does warn against installing mamba inside of conda, but that's been the case for a long time): https://mamba.readthedocs.io/en/latest/installation/mam…

  conda update -n base conda
  conda install -n base conda-libmamba-solver
  conda config --set solver libmamba

Post by eXpl0it3r » Tue, Nov 21, 2023, 9:33 PM UTC

beachy wrote: post #86

wg0 wrote: post #82

The quality is really really INSANE and pretty much unimaginable in early 2000s. Could have interesting prospects for games where you have LLM assuming a character and such TTS giving those NPCs voice.

This is a big thing for one area I'm interested in - golf simulation. Currently playing in a golf simulator has a bit of a post-apocalyptian vibe. The birds are cheeping, the grass is rustling, the game play is realistic, but there's not a human to be seen. Just so different from the smacktalking of a real round, or the crowd noise at a big game. It's begging for some LLM-fuelled banter to be added.

Post by ideasman42 » Wed, Nov 22, 2023, 6:03 AM UTC