Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Noise cancellation improves turn-taking for AI Voice Agents

krisp.ai

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Noise cancellation improves turn-taking for AI Voice Agents

#21

Post by pzo »

Ajedi32 wrote: post #15
polygot wrote: post #11

How much better is the turn taking relative to two humans, when, for example, ordering a pizza? Human to human interaction has a non-zero false turn taking false positive score in my experience.

Latency (such as you get when communicating over the phone) makes turn taking much more difficult. Even in-person it's still a non-zero false positive rate though.

exactly when you are speaking face to face and you see person you have another visual cue from someone lips and face expression realtime and you know if someone stopped speaking or just taking some time to gather thoughts or trying to find proper word in their own head (e.g. non native speaker).

Re: Noise cancellation improves turn-taking for AI Voice Agents

#22

Post by pzo »

> low algorithmic latency of just 15 milliseconds

I guess overall latency will be higher since processing will have to go to their server than back to our server then back from our server to STT provider and back to as then back to LLM provider and back to us and last part to TTS provider and back to the user.

It's so weird that e.g. OpenAI doesn't provide a way to make very simple voice pipeline STT + LLM + TTS executed totally on their servers, this would reduce latency significantly.

The pipeline with this server side audio processing right now looks mostly would have to look like that:

user phone -> our server -> krisp server -> our server -> OpenAI STT -> our server -> OpenAI LLM -> our server -> OpenAI TTS -> our server -> back to user.

Then you have to hope that user and all servers are hosten in the same region

Re: Noise cancellation improves turn-taking for AI Voice Agents

#23

Post by edmundsauto »

pzo wrote: post #22

> low algorithmic latency of just 15 milliseconds I guess overall latency will be higher since processing will have to go to their server than back to our server then back from our server to STT provider and back to as then back to LLM provider and back to us and last part to TTS provider and back to the user. It's so weird that e.g. OpenAI doesn't provide a way to make very simple voice pipeline STT + LLM + TTS exec…

It depends on what you're optimizing against, I think. I'd guess OpenAI prioritizes retention, so they don't give up the recent massive user base growth. If you retain users, you get them to continue using the product (data flywheel), and you can iterate fast enough - you will eventually be successful.

Re: Noise cancellation improves turn-taking for AI Voice Agents

#24

Post by [deleted] »

KeplerBoy wrote: post #20
kazinator wrote: post #16

How about having the AI keep talking until you say a specific word like "stop". Background noise will rarely produce a false positive for that word.

Because that's a completely different problem. By doing that you'd reduce the problem to transcription and have a worse UX.

[deleted]

Re: Noise cancellation improves turn-taking for AI Voice Agents

#27

Post by mort96 »

a1371 wrote: post #26

Recently I was practicing QA with ChatGPT, I wondered if I say "stay silent for 4 seconds before talking" will it work? lo and behold, it didn't.

Does it even have control over that? Isn't ChatGPT's voice mode just speech to text and text to speech wrapped around a text model? Unless it specifically has access to pragmas like "stay silent for 4 seconds" which gets communicated to the text to speech part, it's hard to imagine that it'd even have the ability to stay silence for that amount of time.

Re: Noise cancellation improves turn-taking for AI Voice Agents

#28

Post by tossandthrow »

mort96 wrote: post #27
a1371 wrote: post #26

Recently I was practicing QA with ChatGPT, I wondered if I say "stay silent for 4 seconds before talking" will it work? lo and behold, it didn't.

Does it even have control over that? Isn't ChatGPT's voice mode just speech to text and text to speech wrapped around a text model? Unless it specifically has access to pragmas like "stay silent for 4 seconds" which gets communicated to the text to speech part, it's hard to imagine that it'd even have the ability to stay silence for that amount of time.

You can instruct the speech

Re: Noise cancellation improves turn-taking for AI Voice Agents

#29

Post by dgellow »

Off topic: I hate so much what Krisp (the desktop app) has become. It was the perfect background noise cancelling tool, and now is asking permissions to record your whole screen and audio, for AI meeting companion features I couldn’t care about one bit. And they make it such a frustrating pain to opt-out, with constant modals tricking you in enabling all those AI features.

Does anyone know an alternative that achieves similar level of background noise cancelling?

Re: Noise cancellation improves turn-taking for AI Voice Agents

#30

Post by YetAnotherNick »

kazinator wrote: post #16

How about having the AI keep talking until you say a specific word like "stop". Background noise will rarely produce a false positive for that word.

They are solving the opposite problem, ie detecting when to start the AI. Stopping the AI is significantly easier problem.
Post reply on HN