They just finished a demo on twitch. Pretty crazy!
https://www.twitch.tv/videos/1114111652
Starts at 15:45.
OpenAI Codex
11–20 of 181 posts
Re: OpenAI Codex
#12I thought OpenAI was originally supposed to be some kind of for-the-good, non-profit institution studying AI and its safe use in particular with an effort to make it more accessible and available to all through more open collaboration. This is cool research, sure; but what happened to making models available for use by others instead of just through some opaque APIs? Maybe I'm just remembering wrong or conflating Ope…
No, they did some good, they've done a few things to personally help me. They created OpenAI Gym which is a great help when doing reinforcement learning research and defined the standard interface for reinforcement learning libraries for a generation. But they not longer maintain OpenAI Gym.
They also created Spinning Up [0], one of the best resources I've found for learning reinforcement learning. Their teaching resources are detailed but relatively brief and are focused on implementing the algorithms, even if some of the "proofs" are neglected. But they no longer maintain Spinning Up.
So yes, originally they were for-the-good, but lately I've noticed them moving away from that in more ways than one. It seems they learned one cool trick with language sequence modelling, and they have a lot of compute, and this is all they do now.
Re: OpenAI Codex
#13I thought OpenAI was originally supposed to be some kind of for-the-good, non-profit institution studying AI and its safe use in particular with an effort to make it more accessible and available to all through more open collaboration. This is cool research, sure; but what happened to making models available for use by others instead of just through some opaque APIs? Maybe I'm just remembering wrong or conflating Ope…
That was the marketing message. They became for-profit in 2019 and took investment from Microsoft. Many people were skeptical before that because the main investors were mostly known for for-profit ventures.
Re: OpenAI Codex
#14A warning to devs building on OpenAI APIs: We spent months developing a chatbot using GPT3 for our game and released a video showcasing it: https://www.youtube.com/watch?v=nnuSQvoroJo&t=264s Afterwards OpenAI then added GPT3 chatbot guidelines disallowing basically anything like this. We were in communication with them beforehand, but they decided later that any sort of free form chatbot was dangerous. What they allo…
That's a really interesting demo. What makes the responses so laggy? Does the model take that long to generate text? You can also experiment with things like repeating the user question or adding pauses like "hmm let's see" to make it less noticeable at least some of the time.
Too bad they asked you to pull it. What's the danger they are worried about? Annoying thing from their press releases is how seriously they take their GPT3 bot impact on humans. Despite all the hype, it's difficult to see the end of humanity by GPT3 bots any time soon. Honestly they need to rename themselves - can't see what's open about openai.
Re: OpenAI Codex
#15I thought OpenAI was originally supposed to be some kind of for-the-good, non-profit institution studying AI and its safe use in particular with an effort to make it more accessible and available to all through more open collaboration. This is cool research, sure; but what happened to making models available for use by others instead of just through some opaque APIs? Maybe I'm just remembering wrong or conflating Ope…
You're remembering correctly. OpenAI transitioned from non-profit to for-profit in 2019, took about $1 billion from Microsoft (there has been speculation that this was mostly in the form of Azure credits), and announced that Microsoft would be their preferred partner for commercializing OpenAI technologies: https://openai.com/blog/microsoft/
Re: OpenAI Codex
#16A warning to devs building on OpenAI APIs: We spent months developing a chatbot using GPT3 for our game and released a video showcasing it: https://www.youtube.com/watch?v=nnuSQvoroJo&t=264s Afterwards OpenAI then added GPT3 chatbot guidelines disallowing basically anything like this. We were in communication with them beforehand, but they decided later that any sort of free form chatbot was dangerous. What they allo…
That's a really interesting demo. What makes the responses so laggy? Does the model take that long to generate text? You can also experiment with things like repeating the user question or adding pauses like "hmm let's see" to make it less noticeable at least some of the time. Too bad they asked you to pull it. What's the danger they are worried about? Annoying thing from their press releases is how seriously they ta…
It's laggy since it needs to do speech to text, gpt3 text response, then text to speech. Not sure what adds the most latency actually.
They only allow gpt3 chatbots if the chatbot is designed to speak only about a specific subject, and literally never says anything bad/negative (and we have to keep logs to make sure this is the case). Which is insane. Their reasoning to me was literally a 'what if' the chatbot "advised on who to vote for in the election". As if a chatbot in the context of a video game saying who to vote for was somehow dangerous
I understand the need to keep GPT3 private. There is a lot of possibility for deception using it. But they are so scared of their chatbot saying a bad thing and the PR around that they've removed the possibility of doing anything useful with it. They need to take context more into account - a clearly labeled chatbot in a video game is different than a Twitter bot
Re: OpenAI Codex
#17I really want to just play with this tech- it’s frightening but also the future, but I’m still waiting to be accepted on the GitHub copilot waitlist. I wonder how long this will take for people who don’t know someone who knows someone…
that's not the future, these large language models have no understanding of language, they repeat the most frequently occurring patterns like parrots. They miss this whole thing called semantics.
Re: OpenAI Codex
#18A warning to devs building on OpenAI APIs: We spent months developing a chatbot using GPT3 for our game and released a video showcasing it: https://www.youtube.com/watch?v=nnuSQvoroJo&t=264s Afterwards OpenAI then added GPT3 chatbot guidelines disallowing basically anything like this. We were in communication with them beforehand, but they decided later that any sort of free form chatbot was dangerous. What they allo…
This stunning. Imagine being able to practice your foreign language lessons this way.
Re: OpenAI Codex
#19I don't understand what is going on, why are people even spending time on this? I think this and copilot and etc are solving a non problem of "we will remove the boring part of programming" by generating a bunch of code, so now it's even more boring to read it and check if it actually does what you want.
In the same time zero of the developers I interviewed know how a linked list is laid out in memory, or what is the pro/con of continuous memory layouts, or even how a cpu works actually.
Maybe those things are not needed anymore, but I see their code... I think it will be better if they know them.
Re: OpenAI Codex
#20I'm still surprised by the approach. I mean, great that it works this well -- but program synthesis is one of those rare domains where you can observe exactly what the outcome is after you generate something. You can see execution traces, variable values, what the JIT produced, etc. And all of this is relatively cheap -- often executing a code snippet should be far cheaper than an extra pass through a giant DNN right? So it's fascinating to me that they train entirely from dealing with code as text.
Imagine learning to develop recipes, not by ever cooking or eating or even seeing food, but only reading a giant library of cookbooks. Or learning to compose music but never hearing or playing anything -- only seeing scores.