Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I built a sub-500ms latency voice agent from scratch

ntik.me

Post reply on HN
Print view View original

21–30 of 168 posts

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#22

Post by woodson »

nicktikhonov wrote: post #16
modeless wrote: post #14

IMO STT -> LLM -> TTS is a dead end. The future is end-to-end. I played with this two years ago and even made a demo you can install locally on a gaming GPU: https://github.com/jdarpinian/chirpy , but concluded that making something worth using for real tasks would require training of end-to-end models. A really interesting problem I would love to tackle, but out of my budget for a side project.

If you're of that opinion, you'll enjoy the new stuff coming out from nvidia: https://research.nvidia.com/labs/adlr/personaplex/

You mean Moshi (https://github.com/kyutai-labs/moshi)? Since Personaplex is just a finetuned Moshi model.

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#25

Post by mountainriver »

woodson wrote: post #22
nicktikhonov wrote: post #16

Earlier quoted context omitted.

If you're of that opinion, you'll enjoy the new stuff coming out from nvidia: https://research.nvidia.com/labs/adlr/personaplex/

You mean Moshi ( https://github.com/kyutai-labs/moshi )? Since Personaplex is just a finetuned Moshi model.

Yeah except moshi doesn’t sound good at all

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#27

Post by brody_hamer »

> Voice is a turn-taking problem

It really feels to me like there’s some low hanging fruit with voice that no one is capitalizing on: filler words and pacing. When the llm notices a silence, it fills it with a contextually aware filler word while the real response generates. Just an “mhmm” or a “right, right”. It’d go so far to make the back and forth feel more like a conversation, and if the speaker wasn’t done speaking; there’s no talking over the user garbage. (Say the filler word, then continue listening.)

Re: Show HN: I built a sub-500ms latency voice agent from scratch

#28

Post by nicktikhonov »

brody_hamer wrote: post #27

> Voice is a turn-taking problem It really feels to me like there’s some low hanging fruit with voice that no one is capitalizing on: filler words and pacing. When the llm notices a silence, it fills it with a contextually aware filler word while the real response generates. Just an “mhmm” or a “right, right”. It’d go so far to make the back and forth feel more like a conversation, and if the speaker wasn’t done spea…

100% - I thought about that shortly after writing this up. One way to make this work is to have a tiny, lower latency model generate that first reply out of a set of options, then aggressively cache TTS responses to get the latency super low. Responses like "Hmm, let me think about that..." would be served within milliseconds.
Post reply on HN