I just need their API to be faster. 15-30 seconds per request using 4o-mini isn't good enough for responsive applications.
The new Realtime Websocket API appears to send back responses within less than a second. It might be just what you want.
OpenAI DevDay 2024 live blog
71–80 of 99 posts
Re: OpenAI DevDay 2024 live blog
#72> The Realtime API improves this by streaming audio inputs and outputs directly, enabling more natural conversational experiences. It can also handle interruptions automatically, much like Advanced Voice Mode in ChatGPT. > Under the hood, the Realtime API lets you create a persistent WebSocket connection to exchange messages with GPT-4o. The API supports function calling(opens in a new window), which makes it possibl…
It's incredible that people are talking about the downfall of software engineering - now, at many companies, hundreds of call center roles will be replaced by a few engineering roles. With image fine-tuning, now we can replace radiologists with software engineers, etc. etc.
Replacing call center roles with this is something I can see happening with the realtime api + voice output.
Radiologists, I'm not sure what we need is just image model finetuning + LLMs to get there.
Re: OpenAI DevDay 2024 live blog
#73> The Realtime API improves this by streaming audio inputs and outputs directly, enabling more natural conversational experiences. It can also handle interruptions automatically, much like Advanced Voice Mode in ChatGPT. > Under the hood, the Realtime API lets you create a persistent WebSocket connection to exchange messages with GPT-4o. The API supports function calling(opens in a new window), which makes it possibl…
It's incredible that people are talking about the downfall of software engineering - now, at many companies, hundreds of call center roles will be replaced by a few engineering roles. With image fine-tuning, now we can replace radiologists with software engineers, etc. etc.
People have been trying to replace radiologists for several years now. Maybe they'll get there, but it doesn't seem to be easy.
Re: OpenAI DevDay 2024 live blog
#74I just need their API to be faster. 15-30 seconds per request using 4o-mini isn't good enough for responsive applications.
The new Realtime Websocket API appears to send back responses within less than a second. It might be just what you want.
yes and you can use it in text-text mode if you want. a key benefit is for turn-based usages (where you have running back and forth between user and assistant) you only need to send the incremental new input message for each generation. this is better than "prompt caching" on the chat completions API, which is basically a pricing optimization, as it's actually a technical advantage that uses less upstream bandwidth.
Re: OpenAI DevDay 2024 live blog
#75Earlier quoted context omitted.
I just had an evil thought: once AIs are fast enough, it would be possible to create a “dynamic” user interface on the fly using an AI. Instead of Java or C# code running in an event loop processing mouse clicks, in principle we could have a chat bot generate the UI elements in a script like WPF or plain HTML and process user mouse and keyboard input events! If you squint at it, this is what chat bots do now, except…
That actually sounds pretty entertaining. Especially if there is dynamic user input, like text box input
Other than being borderline impossible to secure, it “should just work” once the AIs get smart enough.
Fine-tuning the model based on example pages and responses might be all that’s required for a sufficient level of consistency.
An immediate use-case might be prototyping in-place.
If you have an existing site, you can capture the request-response pairs and train the AI on it, annotated with the spec docs. Then tell it to implement some new functionality and it should be able to. Just route a subset of the site to the AI instead of the normal controllers.
One could “design” new components and functionality in English and try it instantly with no compilation or deployment steps!
Re: OpenAI DevDay 2024 live blog
#76> The Realtime API improves this by streaming audio inputs and outputs directly, enabling more natural conversational experiences. It can also handle interruptions automatically, much like Advanced Voice Mode in ChatGPT. > Under the hood, the Realtime API lets you create a persistent WebSocket connection to exchange messages with GPT-4o. The API supports function calling(opens in a new window), which makes it possibl…
It's incredible that people are talking about the downfall of software engineering - now, at many companies, hundreds of call center roles will be replaced by a few engineering roles. With image fine-tuning, now we can replace radiologists with software engineers, etc. etc.
What's the role of the software engineer besides setting this up?
Your example makes me think it will merely moves QA into essentially providing countless cases and then updating them over time to improve the AIs data.
And is it really gonna be cheaper than human support?
What's gonna happen when we will find out (see the impossibility to reach a human when interacting with many companies alredy) this is gonna bring (maybe, eventually) costs down, and revenue too because pissed off customers will move elsewhere.
Re: OpenAI DevDay 2024 live blog
#77I've never seen a company publishing consistently groundbreaking features at such a speed like this one. I really wonder how their teams work. It's unprecedented at what i've seen in 15 years software
[deleted]
Re: OpenAI DevDay 2024 live blog
#78I understand the Realtime API voice novelty, and the techonological achievement it is, but I don't see it from the product point of view. It looks like one of those startups finding a solution before knowing the problem. The two examples shown in the DevDay are the things I don't really want to do in the future. I don't want to talk to anybody, and I don't want to wait for their answer in a human form. That's why I o…
I would love a work assistant, some sort of secretary idk I can talk to while I code.
"What are today's most important tasks? Anything I forgot before I log off? Can you write John to check the blocking PR? Let's fix this bug together".
Re: OpenAI DevDay 2024 live blog
#79Interesting choice of a 24kHz sample rate for PCM audio. I wonder if the model was trained on 24kHz audio, rather than the usual 8/16kHz for ML models.
Re: OpenAI DevDay 2024 live blog
#80I understand the Realtime API voice novelty, and the techonological achievement it is, but I don't see it from the product point of view. It looks like one of those startups finding a solution before knowing the problem. The two examples shown in the DevDay are the things I don't really want to do in the future. I don't want to talk to anybody, and I don't want to wait for their answer in a human form. That's why I o…
You're right, having a voice conversation for any reason is just so passe these days. They should stop adding microphones to phones and everything. So old-fashioned and inefficient. And who wants to ever have to actually talk to someone or some AI to ask for anything? I'm sure our vocal cords will evolve away soon. They are so primitive. Vestigial organs.
I love having voice conversations with friends, family, and people I care of. Not with businesses.