How long till a stand alone OS that makes AI usage its first class citizen?
Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
21–30 of 30 posts
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#22My very first Multimodal AI star on Github. Hope we see more of these in the future.
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#23Ok, I need this but cloning Majel Barrett as the voice of the Enterprise computer.
Trivially done with a minute-long wav file. Simply specify the source sample in your june-va config.json
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#24I currently use Ollama + Openwebui for this. It also has a really serviceable voice mode. And it has many options like RAG integrations, custom models, memories to know you better, vision, a great web interface etc. But I'll have a look at this thing.
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#25Earlier quoted context omitted.
is there anyone to sue you? (how does that work?)
I don't know the details in this case, but it seems plausible that someone still owns the IP and thus might be in a position to initiate legal proceedings.
That same person could update the license, no?
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#26Coqui's XTTSv2 is good for this because it has a streaming mode. I have my own version of this where I got ~500ms end-to-end response latency, which is much faster than any other open source project I've seen. https://github.com/jdarpinian/chirpy These are easy to make and fun to play with and it's awesome to have everything local. But it will take more to build something truly useable. A truly natural conversational…
> There aren't any open source audio-to-audio models yet
I think that's not true. See this for example: https://huggingface.co/facebook/seamless-m4t-v2-large It's not general purpose like GPT4o but translation still seems pretty useful
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#27Coqui's XTTSv2 is good for this because it has a streaming mode. I have my own version of this where I got ~500ms end-to-end response latency, which is much faster than any other open source project I've seen. https://github.com/jdarpinian/chirpy These are easy to make and fun to play with and it's awesome to have everything local. But it will take more to build something truly useable. A truly natural conversational…
> There aren't any open source audio-to-audio models yet I think that's not true. See this for example: https://huggingface.co/facebook/seamless-m4t-v2-large It's not general purpose like GPT4o but translation still seems pretty useful
I don't think SeamlessM4T qualifies as an end-to-end audio-to-audio model. The paper states "the task of speech-to-speech translation in SeamlessM4T v2 is broken down into speech-to-text translation (S2TT) and then text-to-unit conversion (T2U)". And while language translation is an important application as you mention, it's strictly limited to that. It wouldn't understand or produce non-speech audio (e.g. singing, music, environmental sounds, etc) and you can't have a conversation with it.
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#28Looks interesting! Is the latency low enough for it to feel natural? How's the Coqui speech quality?
It supports XTTSv2 which is currently the open-weight state of the art. So, pretty damn good ( https://huggingface.co/coqui/XTTS-v2/blob/main/samples/en_sa... ). Too bad that the project is in limbo after Coqui (the company) folded. The license limits the use of the weights to non-commercial usage unless you buy a commercial license, and there's nobody left to sell you one now.
Honestly, I don't think that sounds as human as piper does, but that's probably a function of the voice model files more than anything, fe, en_US 'amy' sounds artificial, but hfc_female sounds more realistic on the Piper samples.
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#29Today we released our full open source end to end ASR+LLM+TTS dockerized stack at ->
Re: Show HN: Local voice assistant using Ollama, transformers and Coqui TTS toolkit
#30we have made an open source orchestration which enables you to plug in your own TTS/ASR/LLM for end-to-end voice conversations at -> https://github.com/bolna-ai/bolna . We are also working on a complete open source stack for ASR+TTS+LLM and will be releasing it shortly.
Have you thought about support for the wyoming protocol? That would make it pretty much plug&play with home assistant.
As mentioned few days before, we just released the end to end open source stack at https://github.com/bolna-ai/bolna/tree/master/examples/whisp...