WhisperSpeech – An open source text-to-speech system built by inverting Whisper
1–10 of 119 posts
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#2I was looking at video on training a custom voice with Piper, following a tutorial at https://www.youtube.com/watch?v=b_we_jma220, and noticed how the datasets required metadata of the text for the source audio files. This training method by Collabora seems to automate that process and only requires an audio file for training.
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#3Can this be run on Mac M1?
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#4I know it's old at this point and doesn't use the fancy new tech, but Mycroft's Mimic 3 is still pretty impressive and is small enough to fit comfortably and generate speech in real time on a raspberry pi [0]. Some of their voices are better than others, but the best of them are definitely equal to the examples of WhisperSpeech given here.
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#5I was looking at video on training a custom voice with Piper, following a tutorial at https://www.youtube.com/watch?v=b_we_jma220 , and noticed how the datasets required metadata of the text for the source audio files. This training method by Collabora seems to automate that process and only requires an audio file for training.
Whisper solves it, that’s its purpose.
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#6Interested to see how it performs for Mandarin Chinese speech synthesis, especially with prosody and emotion. The highest quality open source model I've seen so far is EmotiVoice[0], which I've made a CLI wrapper around to generate audio for flashcards.[1] For EmotiVoice, you can apparently also clone your own voice with a GPU, but I have not tested this.[2]
[0] https://github.com/netease-youdao/EmotiVoice
[1] https://github.com/siraben/emotivoice-cli
[2] https://github.com/netease-youdao/EmotiVoice/wiki/Voice-Clon...
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#7Interested to see how it performs for Mandarin Chinese speech synthesis, especially with prosody and emotion. The highest quality open source model I've seen so far is EmotiVoice[0], which I've made a CLI wrapper around to generate audio for flashcards.[1] For EmotiVoice, you can apparently also clone your own voice with a GPU, but I have not tested this.[2] [0] https://github.com/netease-youdao/EmotiVoice [1] https:…
Have you released your flashcard app?
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#8Interested to see how it performs for Mandarin Chinese speech synthesis, especially with prosody and emotion. The highest quality open source model I've seen so far is EmotiVoice[0], which I've made a CLI wrapper around to generate audio for flashcards.[1] For EmotiVoice, you can apparently also clone your own voice with a GPU, but I have not tested this.[2] [0] https://github.com/netease-youdao/EmotiVoice [1] https:…
Have you released your flashcard app?
It's just an Anki deck.
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#9Interested to see how it performs for Mandarin Chinese speech synthesis, especially with prosody and emotion. The highest quality open source model I've seen so far is EmotiVoice[0], which I've made a CLI wrapper around to generate audio for flashcards.[1] For EmotiVoice, you can apparently also clone your own voice with a GPU, but I have not tested this.[2] [0] https://github.com/netease-youdao/EmotiVoice [1] https:…
Did you try XTTS v2 for Mandarin? I'm curious how it compares with EmotiVoice.
Re: WhisperSpeech – An open source text-to-speech system built by inverting Whisper
#10Aside: is it just me, or is anyone else just as dumbfounded with how quickly literally every aspect of AI and LLMs and Models and blah blah blah is going?
Am I weird in just having my head spin - even though I've also been at leading edge tech before, but this is just me yelling at these new algos on my lawn?