good
Ask HN: What's the current best local/open speech-to-speech setup?
51–60 of 78 posts
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#52You should look into the new Nvidia model: https://research.nvidia.com/labs/adlr/personaplex/ It has dual channel input / output and a very permissible license
Oh man that space emergency example had me rolling
Changing my title to "Astronaut" right now... I'll be using that line as well anytime someone asks me to do something.
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#53Re: Ask HN: What's the current best local/open speech-to-speech setup?
#54This is not strictly speech-to-speech, but I quite like it when working with Claude Code or other CLI Agents: STT: Handy [1] (open-source), with Parakeet V3 - stunningly fast, near-instant transcription. The slight accuracy drop relative to bigger models is immaterial when you're talking to an AI. I always ask it to restate back to me what it understood, and it gives back a nicely structured version -- this helps con…
posts like this are why i visit HN daily!!!
thanks for sharing your knowledge; can’t wait to try out your voice plugin
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#55speech to speech is not nearly as good as livekit IMO ("old school" sequence of transcribe, LLM, synthesize). depends on what you're doing of course, but this is just because the LLMs are just way smarter than the speech to speech models which are pretty much the worst (again IMO) at anything beyond basic banter. and livekit is just a framework so you can hook it up with any models in the stack. im not an expert on t…
They work for two entirely different things. The problem with these pipelines is that unless the latency is very low they simply aren't suitable replacements for Alexa etc. For that use case, low latency beats smarts.
The latency is very very low in my experience, it would definitely work well as an Alexa style assistant
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#56It requires a bit of tinkering, but I think pipecat is the way to go. You can plug in pretty much any STT/LLM/TTS you want and go. It definitely supports local models but its up to you to get your hands on those models. Not sure if there's any turnkey setups that are preconfigured for local install where you can just press play and go though. Last I heard E2E speech to speech models are still pretty weak. I've had pr…
I suspect the glued pipeline is going to remain dominant for a while, mostly because the intermediate text layer is structural, not just a byproduct. If you drop the text for a pure E2E model, you suddenly lose the ability to easily inject RAG context or handle complex tool use. I've been building some agent workflows recently and having that text state to pass into something like LangGraph is the only way to reliably control the logic. Without it, you are basically flying blind on the backend.
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#57There was a great post the other day showing low latency end to end using Nvidia models on a single GPU with pipecat
Discussion: https://news.ycombinator.com/item?id=46528045
Article: https://www.daily.co/blog/building-voice-agents-with-nvidia-...
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#58This is not strictly speech-to-speech, but I quite like it when working with Claude Code or other CLI Agents: STT: Handy [1] (open-source), with Parakeet V3 - stunningly fast, near-instant transcription. The slight accuracy drop relative to bigger models is immaterial when you're talking to an AI. I always ask it to restate back to me what it understood, and it gives back a nicely structured version -- this helps con…
Hi, so I'm looking for an stt that can happen on a server/cron, that will use a small local model (I have 4 vCPU threadripper CPU only and 20G ram on the server) and be able to transcribe from remote audio URLs (preferably, but I know that local models probably don't have this feature so will have to do something like curl the audio down to memory or /tmp and then transcribe and then remove the file etc).
Have any thoughts?
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#59This is not strictly speech-to-speech, but I quite like it when working with Claude Code or other CLI Agents: STT: Handy [1] (open-source), with Parakeet V3 - stunningly fast, near-instant transcription. The slight accuracy drop relative to bigger models is immaterial when you're talking to an AI. I always ask it to restate back to me what it understood, and it gives back a nicely structured version -- this helps con…
Wow Handy works impressively well! Excellent UX too (on Windows at least).
Re: Ask HN: What's the current best local/open speech-to-speech setup?
#60Oh... Having a local-only voice assistant would be great. Maybe someone can share the practical side of this. Do you have the GPU running all day at 200W to scan for wake words? Or is that running on the machine you are working on anyway? Is this running from a headset microphone (while sitting at the desk?) or more like a USB speakerphone? Is there an Alexa jailbreak / alternative firmware as a frontend and run this…
I recently trained a stt model which detects about 40 words- the model is less than 50 kiolobytes. it can run on a <$1 chip.