I hate the voice ai though, it's so much dumber
Fwiw - I found the advanced AI voice feature to be actually detrimental. It's good if you just want a single sentence answer. I've turned it off though when I want a more detailed, structured, considered answer.
How OpenAI delivers low-latency voice AI at scale
11–20 of 172 posts
Re: How OpenAI delivers low-latency voice AI at scale
#12> Voice AI only feels natural if conversation moves at the speed of speech […] At OpenAI’s scale, that translates into three concrete requirements: Global reach for more than 900 million weekly active users Surely the number refers to the total users of ChatGPT overall, and the fraction of those who use voice features is considerably smaller, is it not? That’s the kind of thing that influences business decisions like…
Re: How OpenAI delivers low-latency voice AI at scale
#13- openai is wrong. almost of the issues they described are issues with libwebrtc, not with webrtc, kubernetes, network architecture, etc. the clue was when they said "the conventional one-port-per-session WebRTC model."
- there are no alternatives worth trying. everything else open source in the ecosystem, like pion, coturn, stunner, are too immature.
- libwebrtc is the only game in town.
- they haven't discovered libwebrtc feature flags or how it works with candidates, which directly fix a bunch of latency issues they are discovering. a correct feature flag can instantly reduce latency for free, compared to pay for twilio network traversal style solutions
- 99% of low latency voice END USERS will be in a network situation that can eliminate relays, transceivers, etc. it is totally first class on kubernetes. but you have to know something :)
this is the first time i'm experiencing gell mann amnesia with openai! look those guys are brilliant, but there is hardly anyone in the world who is doing this stuff correctly.
Re: How OpenAI delivers low-latency voice AI at scale
#14if anyone is looking to get into this. pipecat is a great open-source repo and community. https://github.com/pipecat-ai/pipecat
I wish I had known about Pipecat a lot sooner. I found out about it a few weeks back, and since Gemma 4 launched, I've been building my own entirely local voice assistant using Gemma 4 + Kokoro TTS + Whisper from scratch - https://github.com/pncnmnp/strawberry . Pipecat's smart turn model is really good for VAD - https://huggingface.co/pipecat-ai/smart-turn-v3
Re: How OpenAI delivers low-latency voice AI at scale
#15Re: How OpenAI delivers low-latency voice AI at scale
#16lol, definitely didn't need to know there's 900M weekly users for this post. I mean yeah, there's a lot of users and they serve globally, that's relevant. But this is just pulling out your biggest stat because you can. How many voice users you have would actually be relevant and interesting but, to baselessly speculate on motivation here, might be a number that doesn't add as much fuel to an upcoming IPO as reminded people that you're almost at a billion users does.
Re: How OpenAI delivers low-latency voice AI at scale
#17> Voice AI only feels natural if conversation moves at the speed of speech […] At OpenAI’s scale, that translates into three concrete requirements: Global reach for more than 900 million weekly active users Surely the number refers to the total users of ChatGPT overall, and the fraction of those who use voice features is considerably smaller, is it not? That’s the kind of thing that influences business decisions like…
Yeah, that's why they've used "reach" - the total number of users who could be exposed to the feature regardless of engagement.
Re: How OpenAI delivers low-latency voice AI at scale
#18Should I or shouldn't I be glad to see zero mention on Codex.