Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Noise cancellation improves turn-taking for AI Voice Agents

krisp.ai

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Noise cancellation improves turn-taking for AI Voice Agents

#31

Post by bartman »

dgellow wrote: post #29

Off topic: I hate so much what Krisp (the desktop app) has become. It was the perfect background noise cancelling tool, and now is asking permissions to record your whole screen and audio, for AI meeting companion features I couldn’t care about one bit. And they make it such a frustrating pain to opt-out, with constant modals tricking you in enabling all those AI features. Does anyone know an alternative that achieve…

I've dropped Krisp after they transitioned from their 'old' app to the 'new' app in the most confusing way that I've ever seen. We were paying for a business subscription and suddenly the 'new' app only seemed to worked with a new type of subscription, the 'old' app did not get updates to support the newest macOS versions for months, ...

Since then we've been relying solely on Zoom's noise cancelling features and haven't been missing anything. They really improved massively over the years.

While it's not a drop in solution if you routinely need to join calls with other conferencing software, I haven't missed Krisp's after 'switching' 2 years ago.

If you're on Windows, NVIDIA Broadcast's AI Noise Removal has performed similarily well for me while gaming.

Re: Noise cancellation improves turn-taking for AI Voice Agents

#32

Post by fakedang »

dgellow wrote: post #29

Off topic: I hate so much what Krisp (the desktop app) has become. It was the perfect background noise cancelling tool, and now is asking permissions to record your whole screen and audio, for AI meeting companion features I couldn’t care about one bit. And they make it such a frustrating pain to opt-out, with constant modals tricking you in enabling all those AI features. Does anyone know an alternative that achieve…

Is this the same Krisp.AI on the front page?

Re: Noise cancellation improves turn-taking for AI Voice Agents

#34

Post by cubefox »

That's nice, but the main problem with current voice turn-taking is different. It's that these systems don't know when it is their turn to speak.

When a human speaks to another, the second person will listen and interpret and guess when the first person is finished talking. For voice agents it doesn't work that way at all.

The text-to-speech system just seems to have a hardcoded "pause" detector, e.g. 2 seconds, and if 2 seconds of silence are ever detected, the "end of message" token is sent and the LLM will start talking. Even if you were just collecting your thoughts and weren't finished at all.

So the semantic content of what you are saying is completely ignored for turn-taking and no analysis takes place which would determine whether the user is likely to have said everything they wanted to say.

Instead of the rigid pause detector, it would actually make more sense for the end-of-message token to be sent when you explicitly say a specific phrase, like literally "over". Which was of course common in half-duplex radio where only one person could transmit. LLMs are half-duplex too: they can't listen and talk at the same time.

Re: Noise cancellation improves turn-taking for AI Voice Agents

#36

Post by phrotoma »

dgellow wrote: post #29

Off topic: I hate so much what Krisp (the desktop app) has become. It was the perfect background noise cancelling tool, and now is asking permissions to record your whole screen and audio, for AI meeting companion features I couldn’t care about one bit. And they make it such a frustrating pain to opt-out, with constant modals tricking you in enabling all those AI features. Does anyone know an alternative that achieve…

It _was_ both the first and the most useful ML powered app I'd ever used and I couldn't get my credit card out fast enough after trying it. Now it won't leave me alone about new features I don't give a fuck about.

Re: Noise cancellation improves turn-taking for AI Voice Agents

#37

Post by dgellow »

fakedang wrote: post #32
dgellow wrote: post #29

Off topic: I hate so much what Krisp (the desktop app) has become. It was the perfect background noise cancelling tool, and now is asking permissions to record your whole screen and audio, for AI meeting companion features I couldn’t care about one bit. And they make it such a frustrating pain to opt-out, with constant modals tricking you in enabling all those AI features. Does anyone know an alternative that achieve…

Is this the same Krisp.AI on the front page?

Yes. They switched from the best desktop noise cancelling software to a horrible mess of AI features supposed to help you with your meetings.

The app now really wants you to use their task tracking, transcripts, summaries, etc. But they have pretty frustrating UX, require invasive permissions. Every other day you get nagged into enabling features you already said no to 20 times

Re: Noise cancellation improves turn-taking for AI Voice Agents

#38

Post by dgellow »

bartman wrote: post #31
dgellow wrote: post #29

Off topic: I hate so much what Krisp (the desktop app) has become. It was the perfect background noise cancelling tool, and now is asking permissions to record your whole screen and audio, for AI meeting companion features I couldn’t care about one bit. And they make it such a frustrating pain to opt-out, with constant modals tricking you in enabling all those AI features. Does anyone know an alternative that achieve…

I've dropped Krisp after they transitioned from their 'old' app to the 'new' app in the most confusing way that I've ever seen. We were paying for a business subscription and suddenly the 'new' app only seemed to worked with a new type of subscription, the 'old' app did not get updates to support the newest macOS versions for months, ... Since then we've been relying solely on Zoom's noise cancelling features and hav…

Yes, I really like NVIDIA broadcast. Unfortunately I work with macOS.

I will look again at zoom, last time I tried their noise canceling it would cut my sentences

Re: Noise cancellation improves turn-taking for AI Voice Agents

#40

Post by krisoft »

cubefox wrote: post #34

That's nice, but the main problem with current voice turn-taking is different. It's that these systems don't know when it is their turn to speak. When a human speaks to another, the second person will listen and interpret and guess when the first person is finished talking. For voice agents it doesn't work that way at all. The text-to-speech system just seems to have a hardcoded "pause" detector, e.g. 2 seconds, and…

> Instead of the rigid pause detector, it would actually make more sense for the end-of-message token to be sent when you explicitly say a specific phrase, like literally "over".

That doesn’t sound very conversational at all. Instead one could train the network to recognise the appropriate turn-taking points.

The simple way to do that is to make the model output a “listen a bit more” token when it is not yet their turn to talk. You can use real life recorded conversations to build up the initial training set, and then add more data where clashes happen (where tha AI and the speaker speaks at the same time over each other.)

More complicated would be a system where the model is periodically fed the audio chunk so far, and the model predicts what the speaker is likely going to say and based on that when it is appropriate to respond and with wath. And then a smaller, faster, local model can be used to verify if what was said matches the prediction, and if so outputs the generated response. If there is a mismatch it engages the more expensive model to come up with a new prediction.

If you engineer this right you can reuse the state vector from save points and save a bit of compute that way.

Asking the user to say “over” at the end of their turn is the most heavy handed solution. Recognising the flow of a conversation is just pattern recognition. That is what machine learning is good at.

Post reply on HN