The low latency is more of a pain point than a good thing, the way they have it implemented. Trying to have a casual conversation with it, as humans we naturally pause, and GPT will take this as you are "done" and start blabbing away. I also suffer from finding the appropriate word I want as I've gotten older and slower, and this fast-voice-gpt just ends up frustrating me more than helping. I have to sit there and th…
How OpenAI delivers low-latency voice AI at scale
41–50 of 172 posts
Re: How OpenAI delivers low-latency voice AI at scale
#42Very grateful that OpenAI published the article/publicized their usage of Pion[0] a library I work on. If you aren't familiar with WebRTC it's a super fun space. I work on a book WebRTC for the Curious [1] that details how it works. [0] https://github.com/pion/webrtc [1] https://webrtcforthecurious.com
slightly unrelated but what’s with storing the entire codebase in the root directory instead of a nested src folder? It makes getting to the README a lot more difficult
import ("github.com/go-sql-driver/mysql")
so it's standard to have the library files in the root directory.Re: How OpenAI delivers low-latency voice AI at scale
#43Re: How OpenAI delivers low-latency voice AI at scale
#44Re: How OpenAI delivers low-latency voice AI at scale
#45what i learned from making a webrtc+kubernetes game streaming product: - openai is wrong. almost of the issues they described are issues with libwebrtc, not with webrtc, kubernetes, network architecture, etc. the clue was when they said "the conventional one-port-per-session WebRTC model." - there are no alternatives worth trying. everything else open source in the ecosystem, like pion, coturn, stunner, are too immat…
Something I noticed is that companies that are vibe-coding their products miss out on the intelligence that (still) only humans can bring to bear. Just the knowledge cutoff alone puts AI at a serious disadvantage in any rapidly changing field.
Re: How OpenAI delivers low-latency voice AI at scale
#46Earlier quoted context omitted.
What do you have going on the hardware side? I want to plug this into hass but don’t know what hardware I need for reasonable latency
Check out [0]. You can do 'Voice AI' on small/cheap hardware. It's the most fun you can have in the space ATM :) It's been a while, but posted a demo here [1] [0] https://github.com/pipecat-ai/pipecat-esp32 [1] https://www.youtube.com/watch?v=6f0sUEUuruw
Re: How OpenAI delivers low-latency voice AI at scale
#47OpenAI uses Go for the networking implementation for the relays and the services, which makes a ton of sense, instead of something immature as TypeScript / Node or whatever. Yet another reason to not consider anything else like that for low-latency networking. Golang (or even Rust and C++) is unmatched for this use-case.
Can golang do zero copy networking nowadays? In the past golang was terrible at this kind of thing due to allocations and copies of all relayed data.
Re: How OpenAI delivers low-latency voice AI at scale
#48Very grateful that OpenAI published the article/publicized their usage of Pion[0] a library I work on. If you aren't familiar with WebRTC it's a super fun space. I work on a book WebRTC for the Curious [1] that details how it works. [0] https://github.com/pion/webrtc [1] https://webrtcforthecurious.com
slightly unrelated but what’s with storing the entire codebase in the root directory instead of a nested src folder? It makes getting to the README a lot more difficult
To me go code looks like somebody vomitted stuff in the root dir and i have to wade through that every time. No namespacing. nothing
Re: How OpenAI delivers low-latency voice AI at scale
#49Very grateful that OpenAI published the article/publicized their usage of Pion[0] a library I work on. If you aren't familiar with WebRTC it's a super fun space. I work on a book WebRTC for the Curious [1] that details how it works. [0] https://github.com/pion/webrtc [1] https://webrtcforthecurious.com
Re: How OpenAI delivers low-latency voice AI at scale
#50The low latency is more of a pain point than a good thing, the way they have it implemented. Trying to have a casual conversation with it, as humans we naturally pause, and GPT will take this as you are "done" and start blabbing away. I also suffer from finding the appropriate word I want as I've gotten older and slower, and this fast-voice-gpt just ends up frustrating me more than helping. I have to sit there and th…