Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a sub-500ms latency voice agent from scratch

ntik.me

Post reply on HN
Print view View original

111–120 of 168 posts

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#111

Post by miki_oomiri »

YesBox wrote: post #80
jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

Why dont voice assistants use a finishing word or sound? People are already trained to say a name to start. Curious why the tech has avoided a cap? “Alexa, what’s tomorrow’s weather [dada]?”

When I speak to an agent, siri, or whatnot, I am always worried that they will assume I'm done talking when I'm thinking. Sometimes I need a many-seconds pause. Even maybe a minute… For Sire and such, I want to ask something simple "Hey Siri, remind me to call dad tomorrow". Easy. But for Claude and such, I want to go on a long monolog (20s, a minute, multi-minutes).

To me, be the best solution would be semantic + keyword + silence.

Hey Agent, blablablabla, thank you.

Hey Agent, blablablabla, please.

Hey Agent, blablablabla, oops cancel.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#112

Post by ktos »

YesBox wrote: post #80
jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

Why dont voice assistants use a finishing word or sound? People are already trained to say a name to start. Curious why the tech has avoided a cap? “Alexa, what’s tomorrow’s weather [dada]?”

"Alexa, what's tomorrow's weather? Over."

"It will be sunny with a high of 10 degrees. Over"

"Thank you. Over and out."

Just add some noise and Push-To-Talk and it will be great for ham radio enthusiasts!

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#113

Post by iso1631 »

miki_oomiri wrote: post #111
YesBox wrote: post #80

Earlier quoted context omitted.

Why dont voice assistants use a finishing word or sound? People are already trained to say a name to start. Curious why the tech has avoided a cap? “Alexa, what’s tomorrow’s weather [dada]?”

When I speak to an agent, siri, or whatnot, I am always worried that they will assume I'm done talking when I'm thinking. Sometimes I need a many-seconds pause. Even maybe a minute… For Sire and such, I want to ask something simple "Hey Siri, remind me to call dad tomorrow". Easy. But for Claude and such, I want to go on a long monolog (20s, a minute, multi-minutes). To me, be the best solution would be semantic + ke…

And suddenly your address book has changed the name from "Dad" to "Tomorow"

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#114

Post by juliendorra »

Nice write up! Even if I think that turn taking is a very simplified model of conversation! There’s collaborative overlapping, while the other continue, there is all the confirmations that the other agree, there’s the phatic messages maintaining the "listening channel open", and there’s even completion (filling a word or a name) that are not turn taking and should not be taken as such, yet that the model should be able to produce and accept. They are probably not modeled well or at all by a turn taking process

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#115

Post by fragmede »

eru wrote: post #82
fragmede wrote: post #53

Earlier quoted context omitted.

it's bad enough how to deal with people that don't think before they speak now we gotta make the computers do it as well‽

Huh, the grandfather was suggestion to have the computer think while you speak. That's different from banning the computer from thinking before they speak, ain't it?

Thinking while I'm speaking means it isn't listening to everything I've said before thinking what to say. If I start my reply with "no, because...", and it's already formulating its response based on the "no" and not what comes after the because, then it's not thinking before it speaks.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#118

Post by aabdi »

nicktikhonov wrote: post #39
jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

This is fascinating, thanks for sharing! I wonder why amazon/google/apple didn't hop on the voice assistant/agent train in the last few years. All 3 have existing products with existing users and can pretty much define and capture the category with a single over-the-air update.

it was too hard~, they all tried real hard and the models just kept failing. The models only got good enough -1.5 years ago~.

I mean its deployed now (Alexa+/gemini). but its expensive as hell. and also kinda useless. Claude cowork/clawbot form factors are better.

Wrong form factor/use case really. People really wanna buy stuff using clawbot.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#119

Post by ismailmaj »

jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

Semantic end of turn being 300ms of silence is horrible because I ended up intentionally um-ing to finish my thoughts before getting answer.

It was difficult to detrain and that made me stop using voice chat with LLMs all together.

Post reply on HN