Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How OpenAI delivers low-latency voice AI at scale

openai.com

Post reply on HN
Print view View original

21–30 of 172 posts

Re: How OpenAI delivers low-latency voice AI at scale

#21

Post by jiggawatts »

doctorpangloss wrote: post #13

what i learned from making a webrtc+kubernetes game streaming product: - openai is wrong. almost of the issues they described are issues with libwebrtc, not with webrtc, kubernetes, network architecture, etc. the clue was when they said "the conventional one-port-per-session WebRTC model." - there are no alternatives worth trying. everything else open source in the ecosystem, like pion, coturn, stunner, are too immat…

Something I noticed is that companies that are vibe-coding their products miss out on the intelligence that (still) only humans can bring to bear. Just the knowledge cutoff alone puts AI at a serious disadvantage in any rapidly changing field.

Re: How OpenAI delivers low-latency voice AI at scale

#22

Post by legohead »

The low latency is more of a pain point than a good thing, the way they have it implemented. Trying to have a casual conversation with it, as humans we naturally pause, and GPT will take this as you are "done" and start blabbing away.

I also suffer from finding the appropriate word I want as I've gotten older and slower, and this fast-voice-gpt just ends up frustrating me more than helping. I have to sit there and think out the whole sentence in my head before I say anything -- not very natural.

Re: How OpenAI delivers low-latency voice AI at scale

#23

Post by Sean-Der »

Very grateful that OpenAI published the article/publicized their usage of Pion[0] a library I work on. If you aren't familiar with WebRTC it's a super fun space. I work on a book WebRTC for the Curious [1] that details how it works.

[0] https://github.com/pion/webrtc

[1] https://webrtcforthecurious.com

Re: How OpenAI delivers low-latency voice AI at scale

#24

Post by Sean-Der »

charisma123 wrote: post #20

If a transceiver crashes during a stream, how is the active session recovered? Does the system automatically re-establish the context in a new WebRTC session?

It doesn't today, but you could with sometime like this [0]. You can save/suspend all WebRTC state and bring it back with the next process.

[0] https://github.com/pion/webrtc-zero-downtime-restart

Re: How OpenAI delivers low-latency voice AI at scale

#25

Post by Sean-Der »

AnthOlei wrote: post #14
pncnmnp wrote: post #10

Earlier quoted context omitted.

I wish I had known about Pipecat a lot sooner. I found out about it a few weeks back, and since Gemma 4 launched, I've been building my own entirely local voice assistant using Gemma 4 + Kokoro TTS + Whisper from scratch - https://github.com/pncnmnp/strawberry . Pipecat's smart turn model is really good for VAD - https://huggingface.co/pipecat-ai/smart-turn-v3

What do you have going on the hardware side? I want to plug this into hass but don’t know what hardware I need for reasonable latency

Check out [0]. You can do 'Voice AI' on small/cheap hardware. It's the most fun you can have in the space ATM :) It's been a while, but posted a demo here [1]

[0] https://github.com/pipecat-ai/pipecat-esp32

[1] https://www.youtube.com/watch?v=6f0sUEUuruw

Re: How OpenAI delivers low-latency voice AI at scale

#28

Post by Sean-Der »

doctorpangloss wrote: post #13

what i learned from making a webrtc+kubernetes game streaming product: - openai is wrong. almost of the issues they described are issues with libwebrtc, not with webrtc, kubernetes, network architecture, etc. the clue was when they said "the conventional one-port-per-session WebRTC model." - there are no alternatives worth trying. everything else open source in the ecosystem, like pion, coturn, stunner, are too immat…

Did you use libwebrtc on the backend? When you say `libwebrtc` is the only game in town are you talking about clients or servers?

Even for clients you have things like libpeer that libwebrtc can't hit.

Post reply on HN