Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How OpenAI delivers low-latency voice AI at scale

openai.com

Post reply on HN
Print view View original

101–110 of 172 posts

Re: How OpenAI delivers low-latency voice AI at scale

#101

Post by hun3 »

zamadatix wrote: post #35
legohead wrote: post #22

The low latency is more of a pain point than a good thing, the way they have it implemented. Trying to have a casual conversation with it, as humans we naturally pause, and GPT will take this as you are "done" and start blabbing away. I also suffer from finding the appropriate word I want as I've gotten older and slower, and this fast-voice-gpt just ends up frustrating me more than helping. I have to sit there and th…

I think these are 2 different layers of "latency". The latency in the article is referring to the transport of the audio stream itself while the latency in your scenario is about how quickly to start responding inside the audio stream.

They are orthogonal.

Suppose you have 100ms audio latency and no wait time. Then, natural pause will trigger response immediately but you won't notice it has started until after ~200ms (round-trip time). Twice as annoying.

Re: How OpenAI delivers low-latency voice AI at scale

#102

Post by modeless »

Lucasoato wrote: post #65

Wait a minute... I’m genuinely happy that they are sharing this, but keep in mind that realtime audio model from OpenAI are still stuck with the 4o family in terms of capabilities, sadly. I still find them so useful, such a pity that there’s no real competitor in this segment, having the experience a real conversation has helped me so much in expressing ideas and concepts. Still, it’s worth to keep in mind that these…

Grok voice is surprisingly good, actually. It's still a dumber model than the thinking modes of frontier models, but it's less dumb than the voice modes of other providers.

Re: How OpenAI delivers low-latency voice AI at scale

#103

Post by stavros »

dharma1 wrote: post #76
Lucasoato wrote: post #75

Earlier quoted context omitted.

Thanks, I’ll try it, even if my experience wasn’t that great with Google models lately (503s)

Give it a shot, 3.1 live one in AI studio/API and max out reasoning - not the one in Gemini app it’s an older model. Another option is to use pipecat with their VAD and separate STT and TTS and any (fast) LLM of your choice - but it’s more plumbing and not a true speech to speech model

Haha, wow, I never thought I'd see a voice model that was too quick, but 3.1 live felt like it responded unnaturally quickly! I'm kind of blown away, I'd want to insert a 100ms delay to make it sound more natural, wow. I never thought I'd see that.

Re: How OpenAI delivers low-latency voice AI at scale

#104

Post by discordance »

janalsncm wrote: post #56
legohead wrote: post #22

The low latency is more of a pain point than a good thing, the way they have it implemented. Trying to have a casual conversation with it, as humans we naturally pause, and GPT will take this as you are "done" and start blabbing away. I also suffer from finding the appropriate word I want as I've gotten older and slower, and this fast-voice-gpt just ends up frustrating me more than helping. I have to sit there and th…

I’ve also experienced this and it’s really annoying. There is this pressure to keep talking if I’m not done with my thought that feels pretty unnatural at least for me. If I’m searching for the right word, I want the opportunity to find it. I think the solution is to handle pauses more intelligently rather than having a higher latency protocol. With low latency you can interrupt and the bot can immediately stop rambl…

Have you tried telling it to pause to let you think?

I often use it while I’m walking and tell it to not respond until I initiate a conversation.

Re: How OpenAI delivers low-latency voice AI at scale

#105

Post by ryanar »

Sean-Der wrote: post #23

Very grateful that OpenAI published the article/publicized their usage of Pion[0] a library I work on. If you aren't familiar with WebRTC it's a super fun space. I work on a book WebRTC for the Curious [1] that details how it works. [0] https://github.com/pion/webrtc [1] https://webrtcforthecurious.com

I used pion and it was fantastic. Most of the article seems pretty standard webrtc techniques for performant voice.

Re: How OpenAI delivers low-latency voice AI at scale

#106

Post by tedsanders »

fragmede wrote: post #45
jiggawatts wrote: post #21

Earlier quoted context omitted.

Something I noticed is that companies that are vibe-coding their products miss out on the intelligence that (still) only humans can bring to bear. Just the knowledge cutoff alone puts AI at a serious disadvantage in any rapidly changing field.

GPT 5.5's knowledge cutoff is August 2025. Which aspect of WebRTC has meaningfully changed since then?

Dec 2025, actually: https://developers.openai.com/api/docs/models/gpt-5.5

(though knowledge cutoffs in practice can be bit fuzzy)

Re: How OpenAI delivers low-latency voice AI at scale

#108

Post by zuzululu »

fidotron wrote: post #64
qrush wrote: post #30

Am I reading this right that OpenAI is not using Livekit for WebRTC/audio anymore?

It does appear that way. The LiveKit server is not what you would want for this architecture anyway (as they basically say with the SFU discussion), although it does have a lot of useful stuff in the client SDKs.

whats wrong with livekit ?

Re: How OpenAI delivers low-latency voice AI at scale

#109

Post by idiotsecant »

a456463 wrote: post #48
thatxliner wrote: post #33

Earlier quoted context omitted.

slightly unrelated but what’s with storing the entire codebase in the root directory instead of a nested src folder? It makes getting to the README a lot more difficult

This is valid criticism. Go fanbois don't like listening to any go criticism. They were all like who needs templates in go. and now go has templates. To me go code looks like somebody vomitted stuff in the root dir and i have to wade through that every time. No namespacing. nothing

Ok... The question was why is it like that. The answer is because it's in go. Nobody was anything other than civil before you neckbearded in here. Chill. There's a sane way to say what you said.

Re: How OpenAI delivers low-latency voice AI at scale

#110

Post by pottertheotter »

discordance wrote: post #104
janalsncm wrote: post #56

Earlier quoted context omitted.

I’ve also experienced this and it’s really annoying. There is this pressure to keep talking if I’m not done with my thought that feels pretty unnatural at least for me. If I’m searching for the right word, I want the opportunity to find it. I think the solution is to handle pauses more intelligently rather than having a higher latency protocol. With low latency you can interrupt and the bot can immediately stop rambl…

Have you tried telling it to pause to let you think? I often use it while I’m walking and tell it to not respond until I initiate a conversation.

I’ve tried this and it says it will but just keeps cutting in. I hate this feature so much.
Post reply on HN