Live data from Hacker News

How OpenAI delivers low-latency voice AI at scale

openai.com

11–20 of 172 posts

Re: How OpenAI delivers low-latency voice AI at scale

#11

I hate the voice ai though, it's so much dumber

Fwiw - I found the advanced AI voice feature to be actually detrimental. It's good if you just want a single sentence answer. I've turned it off though when I want a more detailed, structured, considered answer.

Interestingly, that kind of parallels the real world too: if you want a quick and high level answer, talk to someone in person; if you want something detailed and info-dense, get them to write it down.

Re: How OpenAI delivers low-latency voice AI at scale

#12
post #5

> Voice AI only feels natural if conversation moves at the speed of speech […] At OpenAI’s scale, that translates into three concrete requirements: Global reach for more than 900 million weekly active users Surely the number refers to the total users of ChatGPT overall, and the fraction of those who use voice features is considerably smaller, is it not? That’s the kind of thing that influences business decisions like…

Yeah, that's why they've used "reach" - the total number of users who could be exposed to the feature regardless of engagement.

Re: How OpenAI delivers low-latency voice AI at scale

#13
what i learned from making a webrtc+kubernetes game streaming product:

- openai is wrong. almost of the issues they described are issues with libwebrtc, not with webrtc, kubernetes, network architecture, etc. the clue was when they said "the conventional one-port-per-session WebRTC model."

- there are no alternatives worth trying. everything else open source in the ecosystem, like pion, coturn, stunner, are too immature.

- libwebrtc is the only game in town.

- they haven't discovered libwebrtc feature flags or how it works with candidates, which directly fix a bunch of latency issues they are discovering. a correct feature flag can instantly reduce latency for free, compared to pay for twilio network traversal style solutions

- 99% of low latency voice END USERS will be in a network situation that can eliminate relays, transceivers, etc. it is totally first class on kubernetes. but you have to know something :)

this is the first time i'm experiencing gell mann amnesia with openai! look those guys are brilliant, but there is hardly anyone in the world who is doing this stuff correctly.

Re: How OpenAI delivers low-latency voice AI at scale

#14
post #10
post #6

if anyone is looking to get into this. pipecat is a great open-source repo and community. https://github.com/pipecat-ai/pipecat

I wish I had known about Pipecat a lot sooner. I found out about it a few weeks back, and since Gemma 4 launched, I've been building my own entirely local voice assistant using Gemma 4 + Kokoro TTS + Whisper from scratch - https://github.com/pncnmnp/strawberry . Pipecat's smart turn model is really good for VAD - https://huggingface.co/pipecat-ai/smart-turn-v3

What do you have going on the hardware side? I want to plug this into hass but don’t know what hardware I need for reasonable latency

Re: How OpenAI delivers low-latency voice AI at scale

#16
> Global reach for more than 900 million weekly active users

lol, definitely didn't need to know there's 900M weekly users for this post. I mean yeah, there's a lot of users and they serve globally, that's relevant. But this is just pulling out your biggest stat because you can. How many voice users you have would actually be relevant and interesting but, to baselessly speculate on motivation here, might be a number that doesn't add as much fuel to an upcoming IPO as reminded people that you're almost at a billion users does.

Re: How OpenAI delivers low-latency voice AI at scale

#17
post #5

> Voice AI only feels natural if conversation moves at the speed of speech […] At OpenAI’s scale, that translates into three concrete requirements: Global reach for more than 900 million weekly active users Surely the number refers to the total users of ChatGPT overall, and the fraction of those who use voice features is considerably smaller, is it not? That’s the kind of thing that influences business decisions like…

Yeah, that's why they've used "reach" - the total number of users who could be exposed to the feature regardless of engagement.

[deleted]

Re: How OpenAI delivers low-latency voice AI at scale

#18

Should I or shouldn't I be glad to see zero mention on Codex.

Shouldn’t, I think - advanced voice is a surprisingly slick feature, and if you’re someone who feels that they can think and speak more naturally than when they think and type, AI voice transcription is kind of huge.
Post reply on HN