Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

tavus.io

Post reply on HN
Print view View original

31–40 of 50 posts

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#31

Post by bpanahij »

nextaccountic wrote: post #4

> Non-verbal cues are invisible to text: Transcription-based models discard sighs, throat-clearing, hesitation sounds, and other non-verbal vocalizations that carry critical conversational-flow information. Sparrow-1 hears what ASR ignores. Could Sparrow instead be used to produce high quality transcription that incorporate non-verbal cues? Or even, use Sparrow AND another existing transcription/ASR thing to augment…

This is a very good idea. We currently have a model in our perception system (Raven-1) that performs this partially. It uses audio to understand tone and augment the transcription we send to the conversational LLM. That seems to have an impact on the conversational style of the replicas output, in a good way. We’re still evaluating that model and will post updates when we have better insights.

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#34

Post by bpanahij »

mentalgear wrote: post #12

Metric | Sparrow-1 Precision 100% Recall 100% Common ...

The response timing in the chart in the blog post shows that even with perfect precision/recall Sparrow-1 also has the fastest true positive response times.

The turn taking models were evaluated in a controlled environment with no additional cascaded steps: LLM, TTS, Phx. This matters to get apples to apples comparison: without the rest of the pipeline variability influencing the measurements.

The video conversation examples are sparrow-1 within the full pipeline. These responses aren’t as fast as sparrow itself because the LLM, TTS, facial rendering, and network transport also take time. Without Sparrow-1 they would be slower. Sparrow-1 enables the responses being as fast as they are, and with a faster CVI pipeline configuration the responses can be as fast as 430ms in my testing.

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#35

Post by bpanahij »

krautburglar wrote: post #13

Such things were doing a good-enough job scamming the elderly as it is--even with the silence-based delays.

That’s unfortunate and certainly not what I spend my time dreaming about. My favorite use case for the elderly is as a sort of companion for sharing their story for future generations. One of our partners uses our technology to help elderly. But yeah, this kind of technology makes AI feel more natural, so we should be aware of that and make sure it’s used for good.

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#36

Post by butlike »

ttul wrote: post #9

I tried talking to Claude today. What a nightmare. It constantly interrupts you. I don’t mind if Claude wants to spend ten seconds thinking about its reply, but at least let ME finish my thought. Without decent turn-taking, the AI seems impolite and it’s just an icky experience. I hope tech like this gets widely distributed soon because there are so many situations in which I would love to talk with a model. If only…

Am I not allowed to cut you off if you're ramble-y and incoherent?

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#37

Post by BizarroLand »

butlike wrote: post #36
ttul wrote: post #9

I tried talking to Claude today. What a nightmare. It constantly interrupts you. I don’t mind if Claude wants to spend ten seconds thinking about its reply, but at least let ME finish my thought. Without decent turn-taking, the AI seems impolite and it’s just an icky experience. I hope tech like this gets widely distributed soon because there are so many situations in which I would love to talk with a model. If only…

Am I not allowed to cut you off if you're ramble-y and incoherent?

Its rude if you're a human, and entirely unacceptable if you are a computer.

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#39

Post by Taikonerd »

ttul wrote: post #9

I tried talking to Claude today. What a nightmare. It constantly interrupts you. I don’t mind if Claude wants to spend ten seconds thinking about its reply, but at least let ME finish my thought. Without decent turn-taking, the AI seems impolite and it’s just an icky experience. I hope tech like this gets widely distributed soon because there are so many situations in which I would love to talk with a model. If only…

Agreed. I tried using Gemini's voice interface in their app. It went like this:

===

ME: "OK, so, I have a question about the economics of medicine. Uh..." [pauses to gather thoughts to ask question]

GEMINI: "Sure! Medical economics is the field of..."

===

And it's aggravated by the fact that all the LLMs love to give you page-long responses before it's your turn to talk again!

Re: Show HN: Sparrow-1 – Audio-native model for human-level turn-taking without ASR

#40

Post by code_brian »

MrDunham wrote: post #29
ttul wrote: post #9

I tried talking to Claude today. What a nightmare. It constantly interrupts you. I don’t mind if Claude wants to spend ten seconds thinking about its reply, but at least let ME finish my thought. Without decent turn-taking, the AI seems impolite and it’s just an icky experience. I hope tech like this gets widely distributed soon because there are so many situations in which I would love to talk with a model. If only…

I love Anthropic's models but their realtime voice is absolutely terrible. Every time I use it there is at least once that I curse at it for interrupting me. My main use case for OpenAI/ChatGPT at this point is realtime voice chats. OpenAI has done a pretty great job w/ realtime (their realtime API is pretty fantastic out of the box... not perfect, but pretty fantastic and dead simple setup). I can have what feels li…

I use Claude Code for everything, and I love Anthropic's models. I don't know why, but it wasn't until reading this that I realized: I can use Sparrow-1 with Anthropic's models within CVI. Adding this to my todo list.
Post reply on HN