Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Real-time AI Voice Chat at ~500ms Latency

github.com

Post reply on HN
Print view View original

161–170 of 238 posts

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#161

Post by koljab »

pzo wrote: post #146
koljab wrote: post #2

I built RealtimeVoiceChat because I was frustrated with the latency in most voice AI interactions. This is an open-source (MIT license) system designed for real-time, local voice conversations with LLMs. Quick Demo Video (50s): https://www.youtube.com/watch?v=HM_IQuuuPX8 The goal is to get closer to natural conversation speed. It uses audio chunk streaming over WebSockets, RealtimeSTT (based on Whisper), and Realtime…

This looks great will definitely have a look. I'm just wondering if you tested fastRTC from hugging face? I haven't done that curious about speed between this vs fastrtc vs pipecat.

Yes, I tested it. I'm not that sure what they created there. It adds some noticable latency compared towards using raw websockets. Imho it's not supposed to, but it did it nevertheless in my tests.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#162

Post by koljab »

dummydummy1234 wrote: post #73
koljab wrote: post #2

I built RealtimeVoiceChat because I was frustrated with the latency in most voice AI interactions. This is an open-source (MIT license) system designed for real-time, local voice conversations with LLMs. Quick Demo Video (50s): https://www.youtube.com/watch?v=HM_IQuuuPX8 The goal is to get closer to natural conversation speed. It uses audio chunk streaming over WebSockets, RealtimeSTT (based on Whisper), and Realtime…

Have you looked at pipecat, seems to be similar trying to do standardized backend/webrtc turn detection pipelines.

Did not look into that one. Looks quite good, I will try that soon.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#163

Post by jillyboel »

Dlemo wrote: post #159
TeMPOraL wrote: post #147

Earlier quoted context omitted.

FWIW, wake words are a stopgap; if we want to have a Star Trek level voice interfaces, where the computer responds only when you actually meant to call it, as opposed to using the wake word as a normal word in the conversation, the computer needs to be constantly listening. A good analogy here is to think of the computer (assistant) as another person in the room, busy with their own stuff but paying attention to the…

I want that for privacy reasons and for resource reasons. And having this as a small hardware device should not add relevant latency to it.

Privacy isn't a concern when everything is local

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#164

Post by sabellito »

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up.

Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess outside of their control.

"Just use docker". Have you tried using docker on windows? There's a reason I never do dev work on windows.

I spent most of my career in the JVM and Node, and despite the issues, never had to deal with this level of lack of compatibility.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#165

Post by ConfusedDog »

That's a big improvement over Siri tbh (interruption and latency), but Siri's answer generally kind of shorter than this. My general experience with Siri hasn't been great lately. For complex question, it just redirect to ChatGPT with an extra step for me to confirm. Often stops listening when I'm not even finished with my sentence, and gives "I don't know anything about that" way too often.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#166

Post by jokethrowaway »

koljab wrote: post #2

I built RealtimeVoiceChat because I was frustrated with the latency in most voice AI interactions. This is an open-source (MIT license) system designed for real-time, local voice conversations with LLMs. Quick Demo Video (50s): https://www.youtube.com/watch?v=HM_IQuuuPX8 The goal is to get closer to natural conversation speed. It uses audio chunk streaming over WebSockets, RealtimeSTT (based on Whisper), and Realtime…

Neat!

I build something almost identical last week (closed source, not my IP) and I recommend: NeMo Parakeet (even faster than insanely_fast_whisper), F5-TTS (fast + very good quality voice cloning), Qwen3-4B for LLM (amazing quality).

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#167

Post by jhoho »

sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

Let me introduce you to the beautiful world of virtual environments. They save you the headache of getting a full installation to run, especially when using Windows.

I prefer miniconda, but venv also does the job.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#169

Post by atoav »

sabellito wrote: post #164

Every time I see these things, they look cool as hell, I get excited, then I try to get them working on my gaming PC (that has the GPU), I spend 1-2h fighting with python and give up. Today's issue is that my python version is 3.12 instead of =3.9. Installing python 3.11 from the official website does nothing, I give up. It's a shame that the amazing work done by people like the OP gets underused because of this mess…

uv is the way.

https://docs.astral.sh/uv/

Sadly it appears that people in the LLM space aren't really all that good at packaging their software (maybe, on purpose).

Post reply on HN