Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a sub-500ms latency voice agent from scratch

ntik.me

Post reply on HN
Print view View original

131–140 of 168 posts

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#131

Post by tmstieff »

miki_oomiri wrote: post #111
YesBox wrote: post #80

Earlier quoted context omitted.

Why dont voice assistants use a finishing word or sound? People are already trained to say a name to start. Curious why the tech has avoided a cap? “Alexa, what’s tomorrow’s weather [dada]?”

When I speak to an agent, siri, or whatnot, I am always worried that they will assume I'm done talking when I'm thinking. Sometimes I need a many-seconds pause. Even maybe a minute… For Sire and such, I want to ask something simple "Hey Siri, remind me to call dad tomorrow". Easy. But for Claude and such, I want to go on a long monolog (20s, a minute, multi-minutes). To me, be the best solution would be semantic + ke…

I have the same issue. It gives this very weird minor sense of public speaking anxiety where I almost feel the need to write down what I'm about to say, which negates the whole purpose. Only solution I've found is using push-to-talk with some of the system wide STS applications.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#132

Post by stavros »

llbbdd wrote: post #84
stavros wrote: post #66

Earlier quoted context omitted.

I'm puzzled by this conversation, because Amazon did get on the agent bandwagon with Alexa Plus (I have it, it's buggier than regular Alexa and it's all making me throw my Echos away since they can't even play Spotify reliably). Also, my Alexa does advertise stuff to me when I talk to it. It's not Budweiser, but it'll try to upsell me on Amazon services all the time.

I upgraded to Alexa+ and initially hated it but I've kept it because it's sooo much better at some things. This last December I bought a handful of smart plugs for my Christmas lights all around the house, and I did almost all the setup trivially over voice, e.g. fuzzy run-on stuff like this just worked on the first try: - "Alexa, name the new unnamed outlet 'Living Room Lights', and the other unnamed one 'Stair Ligh…

Hm yeah, that's helpful. For me it'll randomly stop or stutter when playing Spotify, it'll randomly not answer commands, it'll refuse to listen and let some other Alexa in another room reply, it's super janky.

I only use it for music, and use two commands, but apparently having this work correctly is too much to ask for these days.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#133

Post by recognity »

The insight about TTFT dominating everything resonates. We're seeing the same pattern in CLI tools — the perceived speed of AI features comes down to how fast you get the first useful output, not total processing time.

Curious about your semantic end-of-turn detection: are you using a separate lightweight model for that, or is it baked into the main LLM inference? That seems like the hardest part to get right without adding latency.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#134

Post by kvirani »

jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

Isn't fact 2 just a now problem though? Will people's latency expectation not change over time, as it gradually goes down?

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#136

Post by mywacaday »

jedberg wrote: post #36

Oh, this is really interesting to me. This is what I worked on at Amazon Alexa (and have patents on). An interesting fact I learned at the time: The median delay between human speakers during a conversation is 0ms (zero). In other words, in many cases, the listener starts speaking before the speaker is done. You've probably experienced this, and you talk about how you "finish each other's sentences". It's because you…

No. 2 is interesting, our national lottery in Ireland has an app that you can scan the barcode on your ticket to check if you have won or not, at some stage they updated the app and the scan picks up the barcode even before you center it on the screen and tells you if you have lost/won instantly, I though it was my IT background that made me uncomfortable with it happening so fast, wonder what other examples like this exist where the result/action being too fast causes doubt with the user?

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#137

Post by numpad0 »

docheinestages wrote: post #18

Does anyone know about a fully offline, open-source project like this voice agent (i.e. STT -> LLM -> TTS)?

This is hardly a novel implementation of [stream responses and chunk on sentences] + [stop on VAD and memory hole the chat log] concept. This takes <1k vibecoded lines to replicate it with an all-local setup.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#139

Post by derangedHorse »

stavros wrote: post #66
jedberg wrote: post #43

Earlier quoted context omitted.

If your Alexa did that, how quickly would you box it up and send it to me. :) I am serious though about having it sent to me: if anyone has an Alexa they no longer want, I'm happy to take it off your hands. I have eight and have never bought one. Having worked there I actually trust the security more than before I worked there. It was basically impossible for me, even as a Principle Engineer, to get copies of the Tex…

I'm puzzled by this conversation, because Amazon did get on the agent bandwagon with Alexa Plus (I have it, it's buggier than regular Alexa and it's all making me throw my Echos away since they can't even play Spotify reliably). Also, my Alexa does advertise stuff to me when I talk to it. It's not Budweiser, but it'll try to upsell me on Amazon services all the time.

Alexa Plus sucks. It takes way too long to respond even when given simple commands. I either had to turn it off or trash my Echo. Luckily there was an option to turn it off, but Amazon is on thin ice with me.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#140

Post by stavros »

derangedHorse wrote: post #139
stavros wrote: post #66

Earlier quoted context omitted.

I'm puzzled by this conversation, because Amazon did get on the agent bandwagon with Alexa Plus (I have it, it's buggier than regular Alexa and it's all making me throw my Echos away since they can't even play Spotify reliably). Also, my Alexa does advertise stuff to me when I talk to it. It's not Budweiser, but it'll try to upsell me on Amazon services all the time.

Alexa Plus sucks. It takes way too long to respond even when given simple commands. I either had to turn it off or trash my Echo. Luckily there was an option to turn it off, but Amazon is on thin ice with me.

I agree, I can't wait for the trial to end.
Post reply on HN