Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Real-time AI Voice Chat at ~500ms Latency

github.com

Post reply on HN
Print view View original

101–110 of 238 posts

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#101

Post by Bjartr »

joshstrange wrote: post #27
smusamashah wrote: post #4

Saying this as a user of these tools (openai, Google voice chat etc). These are fast yes, but they don't allow talking naturally with pauses. When we talk, we take long and small pauses for thinking or for other reasons. With these tools, AI starts taking as soon as we stop. Happens both in text and voice chat tools. I saw a demo on twitter a few weeks back where AI was waiting for the person to actually finish what…

This 100%, yes! I've found myself putting in filler words or holding a noise "Uhhhhhhhhh" while I'm trying to form a thought but I don't want the LLM to start replying. It's a really hard problem for sure. Similar to the problem of allowing for interruptions but not stopping if the user just says "Right!", "Yes", aka active listening. One thing I love about MacWhisper (not special to just this STT tool) is it's hold…

Somehow need to overlap an LLM with vocal stream processing to identify semantically meaningful transition points to interrupt naturally instead of just looking for any pause or sentence boundary.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#102

Post by alluro2 »

lacoolj wrote: post #66

Call me when the AI can interrupt YOU :)

Apparently, based on other comments that mentioned it, this one can and will if it's confident enough it has sufficient context/information:

https://www.sesame.com/research/crossing_the_uncanny_valley_...

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#103

Post by rylittle »

modeless wrote: post #53
jedberg wrote: post #35

I did some research into this about a year ago. Some fun facts I learned: - The median delay between speakers in a human to human conversation is zero milliseconds. In other words, about 1/2 the time, one speaker interrupts the other, making the delay negative. - Humans don't care about delays when speaking to known AIs. They assume the AI will need time to think. Most users will qualify a 1000ms delay is acceptable…

My take on this is that voice AI has not truly arrived until it has mastered the "Interrupting Cow" benchmark.

When I google '"Interrupting Cow" benchmark' the first result is this comment. What is it?

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#104

Post by foobahhhhh »

kevinsync wrote: post #38
PhunkyPhil wrote: post #24

Earlier quoted context omitted.

This is the system prompt https://github.com/KoljaB/RealtimeVoiceChat/blob/main/code/s... My favorite line: "You ARE this charming, witty, wise girlfriend. Don't explain how you're talking or thinking; just be that person."

I still crack up at the idea of 'personality prompting', mostly because the most engaging and delightful IRL persons who knock us off our guard in a non-threatening way are super natural and possess that "It Factor" that's impossible to articulate lol -- probably because it's multimodal with humans and voice/cadence/vocab/timing/delivery isn't 100% of the attraction. That said, it's not like we have any better altern…

Humans can train for this too

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#105

Post by foobahhhhh »

znpy wrote: post #82

This kind of thing immediately made me think about the 512gb mac studio. If this works as good on that hardware as it does on the recommended nvidia cards, then the $15k is not much the price of the hardware but rather the price of having a full conversational at home, private.

Exciting... because that'll be $1500 at some point. Then $150. Then $15 I.e. on a cheap android old gen phone.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#107

Post by cyjackx »

smusamashah wrote: post #4

Saying this as a user of these tools (openai, Google voice chat etc). These are fast yes, but they don't allow talking naturally with pauses. When we talk, we take long and small pauses for thinking or for other reasons. With these tools, AI starts taking as soon as we stop. Happens both in text and voice chat tools. I saw a demo on twitter a few weeks back where AI was waiting for the person to actually finish what…

I think the problem is that it's also an interesting problem for humans. It's very subjective. Imagine a therapy session, filled with a long pensive pauses. Therapy is one of those things that encourage not interrupting and just letting you talk more, but there's so much subtext and nuance to that. Then make it compared to excited chatter one might have with friends. There's also so much body language that an AI obviously cannot see. At least for now.

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#108

Post by karimf »

koljab wrote: post #2

I built RealtimeVoiceChat because I was frustrated with the latency in most voice AI interactions. This is an open-source (MIT license) system designed for real-time, local voice conversations with LLMs. Quick Demo Video (50s): https://www.youtube.com/watch?v=HM_IQuuuPX8 The goal is to get closer to natural conversation speed. It uses audio chunk streaming over WebSockets, RealtimeSTT (based on Whisper), and Realtime…

Do you have any information on how long each step take? Like how many ms on each step of the pipeline?

I'm curious how fast it will run if we can get this running on a Mac. Any ballpark guess?

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#109

Post by Jordan-117 »

rylittle wrote: post #103
modeless wrote: post #53

Earlier quoted context omitted.

My take on this is that voice AI has not truly arrived until it has mastered the "Interrupting Cow" benchmark.

When I google '"Interrupting Cow" benchmark' the first result is this comment. What is it?

Classic knock-knock joke.

"Knock-knock."

"Who's there?"

"Interrupting cow."

"Interrupting co-"

"MOO!"

Re: Show HN: Real-time AI Voice Chat at ~500ms Latency

#110

Post by freediver »

rylittle wrote: post #103
modeless wrote: post #53

Earlier quoted context omitted.

My take on this is that voice AI has not truly arrived until it has mastered the "Interrupting Cow" benchmark.

When I google '"Interrupting Cow" benchmark' the first result is this comment. What is it?

https://workauthentically.com/interrupting-cow/

"Knock-Knock. Who's there? Interrupting Cow. Interrupting cow who? Moo!

Note that the timing is everything here. You need to yell out your Moo before the other person finishes the Interrupting cow who? portion of the joke, thereby interrupting them. Trust me, it's hilarious! If you spend time with younger kids or with adults who need to lighten up (and who doesn't?!?), try this out on them and see for yourself."

Basically it is about AI interrupting you, and just in the right momment too. Super hard to do from a technical perspective.

Post reply on HN