Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a sub-500ms latency voice agent from scratch

ntik.me

Post reply on HN
Print view View original

151–160 of 168 posts

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#151

Post by 47282847 »

jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

> The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done.

This reminds me of a great diversity training at a previous employer, where we dug into the different expectations of when and how to take your turn in conversation and how that can create a lot of friction just from different cultural/familial habits. In my family, we’re expecting to talk over each other and it’s not offensive at all to do so, whereas some of my friends really get upset if we don’t take clear turns, a mode which would cause high levels of irritation in my family (and still do in me).

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#153

Post by bronco21016 »

nicktikhonov wrote: post #121
bronco21016 wrote: post #50

When someone is able to put something like this together on their own it leaves me feeling infuriated that we can’t have nice things on consumer hardware. At a minimum Siri, Alexa, and Google Home should at least have a path to plugin a tool like this. Instead I’m hacking together conversation loops in iOS Shortcuts to make something like this style of interaction with significantly worse UX.

I feel like you could get pretty far with a raspberry pi and microphone/speaker. I think the hard part is running a model that can detect a "Hey agent" on-device, so that it can run 24/7 and hand off to the orchestrator when it catches a real question/query.

I think you’re right. I’ve been seeing more and more DIY hardware setups popping up. There are even wake work models for hardware as low powered as the ESP32.

In the middle of moving though so probably have to wait before taking on hardware.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#155

Post by arcadianalpaca »

brody_hamer wrote: post #27

> Voice is a turn-taking problem It really feels to me like there’s some low hanging fruit with voice that no one is capitalizing on: filler words and pacing. When the llm notices a silence, it fills it with a contextually aware filler word while the real response generates. Just an “mhmm” or a “right, right”. It’d go so far to make the back and forth feel more like a conversation, and if the speaker wasn’t done spea…

The filler word idea is interesting but I suspect the uncanny valley risk is super high. A mistimed "mhm" from a computer would probably feel way worse than just silence, because now your brain is pattern matching against human conversation and every small timing error stands out more

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#157

Post by garettmd »

jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

> It's because your brain is predicting what they will say while they speak, and processing an answer at the same time. It's also why when they say what you didn't expect, you say, "what?" and then answer half a second later, when your brain corrects.

that's super interesting. do you know of any resources to learn more about this phenomenon?

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#158

Post by BobaFloutist »

jermaustin1 wrote: post #146
sinuhe69 wrote: post #94

Earlier quoted context omitted.

Yeah, I remember the time when we had to use satellites to connect. The long delay was really annoying and so unusual that most people without "training" could not even use the phone for conversation and just wasted the dollars.

A former boss of mine took off to Everest for a month leaving me (a 22 year old, at the time) in charge of the office. I was out to dinner with my now wife when I got a call from a very long phone number I didn't recognize, so I ignored it. I then got another one right after, and picked it up. It was my boss, he needed me to log into his personal email to grab a phone number for the medical insurance he purchased for…

Maybe you bring back radio etiquette and just say "over" at the end of every thought?

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#159

Post by Barbing »

russdill wrote: post #110
Barbing wrote: post #93

Earlier quoted context omitted.

Google seems to be experimenting with this with their AI Mode. They used to be more likely to send 10 blue links in response to complex queries, but now they may instead start you off with slop. (Meanwhile at OpenAI: testing out the free ChatGPT, it feels like they prompted GPT 3.5 to write at length based on the last one or maybe two prompts)

This is more of a "Are all the windows closed upstairs?" "The windows upstairs..." "...are all closed except for the bedroom window" The first portion of the response requires a couple of seconds to play but only a few tens of milliseconds to start streaming using a small model. Currently I just break the small model's response off at whatever point will produce about enough time to spin up the larger model. But all…

Whoa, that thing's fast. Very nice! Bet that's fun to play with, least probably fun the first time you saw it working :)

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#160

Post by digitallyamar »

eru wrote: post #81
nicktikhonov wrote: post #28

Earlier quoted context omitted.

100% - I thought about that shortly after writing this up. One way to make this work is to have a tiny, lower latency model generate that first reply out of a set of options, then aggressively cache TTS responses to get the latency super low. Responses like "Hmm, let me think about that..." would be served within milliseconds.

See also any public speaking who starts every answer to a question from the audience (or in a verbal interview) with something like 'that is a good question!' or "thank you for asking me that!" Same strategy but employed by humans.

"You are absolutely right!"
Post reply on HN