41–50 of 119 posts
I'm interested in applying TTS to a chat system, and one important feature for that is that there should be as many as possible distinct voices, so that each person would have their own.
Would this, or something else be able to do that?
Interested to see how it performs for Mandarin Chinese speech synthesis, especially with prosody and emotion. The highest quality open source model I've seen so far is EmotiVoice[0], which I've made a CLI wrapper around to generate audio for flashcards.[1] For EmotiVoice, you can apparently also clone your own voice with a GPU, but I have not tested this.[2] [0] https://github.com/netease-youdao/EmotiVoice [1] https:…
What we seem to need is high-quality speech recordings in any language (audiobooks are great) and some recordings for each target language which can be low-quality but need varied prosody/emotions (otherwise everything we generate will sound like an audiobook).
Earlier quoted context omitted.
They don't mention the ability to add custom voices to the speech output, I wonder if that's a feature thatbwould be supported
They do mention voice cloning in the README ("We’ve also added an example of voice cloning based on a reference audio file."), do you have something different in mind?
How tunable is the voice? I'm interested in applying TTS to a chat system, and one important feature for that is that there should be as many as possible distinct voices, so that each person would have their own. Would this, or something else be able to do that?
To check how this works in practice you can check the Google Collab link, at the end we are cloning the voice from a Churchill's speech over radio.
Can it run local only?
I think it should work pretty good with the Apple's MLX framework as well if anyone would be willing to convert it. :)
Can this be run on Mac M1?
Idk if it would out of the box, but it should be possible. I know that Whisper (and some variants) run on both x86 and silicon macs.
this is the best tts ive heard, the voice modulates as you'd expect a human to.
We are constantly working on these models and we push new versions every two months or so. It should get even better soon. :)
I was looking at video on training a custom voice with Piper, following a tutorial at https://www.youtube.com/watch?v=b_we_jma220 , and noticed how the datasets required metadata of the text for the source audio files. This training method by Collabora seems to automate that process and only requires an audio file for training.
This works for any language that is well supported by the OpenAI Whisper model.
I know it's old at this point and doesn't use the fancy new tech, but Mycroft's Mimic 3 is still pretty impressive and is small enough to fit comfortably and generate speech in real time on a raspberry pi [0]. Some of their voices are better than others, but the best of them are definitely equal to the examples of WhisperSpeech given here. [0] https://mycroft.ai/mimic-3/
That said if you have a modern NVidia GPU you should be able to run a voice-bot in real-time with WhisperSpeech.