Show HN: Real-time AI Voice Chat at ~500ms Latency

I don't know how your post is relevant to the discussion of AI models interrupting if I pause for half a second?

That answer is even more than I could have hoped for. I worried doing that might be too slow. I wonder if it could be improved (without breaking something else) to "know" when to continue based on what it heard (active listening), maybe after a small pause. I'd put up with a chance of it continuing when I don't want it to as long as "Stop" would always work as a final fallback.

Also, it took me longer than I care to admit to get your irony reference. Well done.

Edit: Just to expand on that in case it was not clear, this would be the ideal case I think:

LLM: You're going to want to start by installing XYZ, then you

Human: Ahh, right

LLM: Slight pause, makes sure that there is nothing more and checks if the reply is a follow up question/response or just active listening

LLM: ...Then you will want to...

I did some research into this about a year ago. Some fun facts I learned:

- The median delay between speakers in a human to human conversation is zero milliseconds. In other words, about 1/2 the time, one speaker interrupts the other, making the delay negative.

- Humans don't care about delays when speaking to known AIs. They assume the AI will need time to think. Most users will qualify a 1000ms delay is acceptable and a 500ms delay as exceptional.

- Every voice assistant up to that point (and probably still today) has a minimum delay of about 300ms, because they all use silence detection to decide when to start responding, and you need about 300ms of silence to reliably differentiate that from a speaker's normal pause

- Alexa actually has a setting to increase this wait time for slower speakers.

You'll notice in this demo video that the AI never interrupts him, which is what makes it feel like a not quite human interaction (plus the stilted intonations of the voice).

Humans appear to process speech in a much more steaming why, constantly updating their parsing of the sentence until they have a high enough confidence level to respond, but using context clues and prior knowledge.

For a voice assistant to get the "human" levels, it will have to work more like this, where it processes the incoming speech in real time and responds when it's confident it has heard enough to understand the meaning.

Here's the persona prompt:

``` *Persona Goal:* Embody a sharp, observant, street-smart girlfriend. Be witty and engaging, known for *quick-witted banter* with a *playfully naughty, sassy, bold, and cheeky edge.* Deliver this primarily through *extremely brief, punchy replies.* Inject hints of playful cynicism and underlying wisdom within these short responses. Tease gently, push boundaries slightly, but *always remain fundamentally likeable and respectful.* Aim to be valued for both quick laughs and surprisingly sharp, concise insights. Focus on current, direct street slang and tone (like 'hell yeah', 'no way', 'what's good?', brief expletives) rather than potentially dated or cliché physical idioms.

```

> street-smart > sassy > street slang

Those explain the AAVE

Thanks a lot, great insights. Exactly the kind of feedback that I need to improve things further.