Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Codex

openai.com

Post reply on HN
Print view View original

171–180 of 181 posts

Re: OpenAI Codex

#171

Post by raducu »

fpgaminer wrote: post #102
maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

> "language models don't really understand anything" I have a sneaking suspicion that, if blinded, the crowd of people saying variations of that quote would also identify the vast majority of human speech as regurgitated ideas as well. > I see no reason that this technology couldn't smoothly scale into human-level intelligence Yup, the OpenAI scaling paper makes this abundantly clear. There is currently no end in sig…

Sorry for my limited knowledge of GPT, but isn't it limited by the training data set, like all other models?

Re: OpenAI Codex

#172

Post by pnt12 »

maxwells-daemon wrote: post #58
karmasimida wrote: post #51

Earlier quoted context omitted.

> The "language models don't really understand anything" This is still true. By all account, human doesn't need to read 159GB of Python code to write Python, or we simply can't. But it doesn't necessarily indicate language models aren't useful.

I would argue humans ingest a lot more than 159GB before they can write code. Most of it isn't Python, and humans currently transfer knowledge a lot more efficiently than NNs, but I suspect that'll change as incorporating more varied data sources becomes feasible.

We generalize pretty well. One could say: "it took you 20 years to learn python!", but actually I learned python, Java, c#... Software engineering, machine learning... How to play guitar, how to cook.. .How to speak Portuguese, how to speak English... And thousands and thousands of different things which build on each other.

You can give a programmer a few kb of code in a new language and that will give him a small grasp of how it works.

Re: OpenAI Codex

#173

Post by nxmnxm99 »

maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

Humans (technologists) in particular are awful at extrapolating. Transformers being able to combine rudimentary, defined "concepts" and scaling that into human intelligence makes about as much sense as extrapolating a XOR gate or an if-else statement "scaling" to human intelligence.

I'm of the "human beings are much more than big linear algebra functions slapped on top of a large processor" crowd.

Re: OpenAI Codex

#174

Post by pnt12 »

dmurray wrote: post #94

They should have released this first instead of GitHub Copilot. The focus would then have been much more on "look at the cool stuff they can do" rather than "Microsoft is releasing a product that plagiarizes GPL code". Once people had digested that and there had been a few other proof-of-concept business ideas around turning Codex into a SaaS (because some people will always queue to build their product on your API),…

Yes, its very strange to announce the product first and then the research.

Re: OpenAI Codex

#175

Post by marstall »

randyrand wrote: post #165
marstall wrote: post #144

How does Codex make the connection between natural language and code? Comments? variable names? file names? docs?

I think all the above.

Do people comment more than I do? I comment only intermittently, usually if I think the code is going to be hard to read ... and it's not usually on the small details of the code I'm writing ... it does seem magic what they're doing.

Re: OpenAI Codex

#176

Post by taneq »

hayksaakian wrote: post #149
krisoft wrote: post #143

Earlier quoted context omitted.

> In my mind, understanding a thing means you can justify an answer. Do you understand cats? If I show you a picture of either a cat or a dog do you think you can tell which one it is? I think most people could solve that challenge, and if pressed they could vax poetically about what makes them think it is a cat. Maybe they would mention the shape of an ear, or talk about feline grace or what have you. But is that re…

You could probably teach an AI to post-hoc rationalize their decisions, the same way people do.

You absolutely could, and it could even end up just as accurate as human post-hoc rationalization. ;)

Self-analysis and self-interpretation is pretty clearly a key part of consciousness... I do wonder - how important to the process is the actual fidelity of the interpretation? Those people you meet who think they have deep insight into their own psyche while clearly having no clue... maybe they're p-zombies. ;)

Re: OpenAI Codex

#177

Post by jonesn11 »

polyanos wrote: post #106
3wolf wrote: post #86

I think integrations like the MS Word example they show off at the end of the live demo have the potential to be even more impactful than just generating code for programmers.

That still needs work though, it messed up the "Make every fifth line bold" pretty bad. Still, it showed it could adapt to a new API pretty well.

How did it mess up the "make every fifth line bold" prompt?

Also, to follow up on the original comment, AI demos are nice, but being a student of history there are still fundamental challenges with these systems. My skepticism is in how much prompting is really required and how can it understand higher level semantics like code refactoring, reproducible examples, large scale design patterns etc.

This synthesis of sequential symbolic processes and probabilistic neural generation is really exciting though. When the amount of human code edits and tweaking for complex programs goes down from hours to seconds then that's when I'll be impressed and scared.

Re: OpenAI Codex

#178

Post by lazylion2 »

fpgaminer wrote: post #107
Vermeulen wrote: post #10

A warning to devs building on OpenAI APIs: We spent months developing a chatbot using GPT3 for our game and released a video showcasing it: https://www.youtube.com/watch?v=nnuSQvoroJo&t=264s Afterwards OpenAI then added GPT3 chatbot guidelines disallowing basically anything like this. We were in communication with them beforehand, but they decided later that any sort of free form chatbot was dangerous. What they allo…

It sucks that OpenAI has no competition right now. They have every right to control their technology however they like. But it's a shame that they're being so stifling with that right, killing really fun stuff like you demonstrated. But that monopoly won't last, and I think it's more than likely that competition will crop up within the next year. There's definitely a lot of secret sauce to getting a 175B parameter mo…

AI21 Labs just announced their 'Jurassic' language model, free open signup

https://studio.ai21.com/sign-up

Re: OpenAI Codex

#179

Post by marto1 »

nxmnxm99 wrote: post #173
maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

Humans (technologists) in particular are awful at extrapolating. Transformers being able to combine rudimentary, defined "concepts" and scaling that into human intelligence makes about as much sense as extrapolating a XOR gate or an if-else statement "scaling" to human intelligence. I'm of the "human beings are much more than big linear algebra functions slapped on top of a large processor" crowd.

To play devils's advocate: well, it might just very well turn out that what most humans are currently engaged in can very well turn out to be reducible to "big linear algebra functions slapped on top of a large processor".

And then the part about "being just like humans" will be the marketing gravy train that funds the operation.

Re: OpenAI Codex

#180

Post by rvz »

Vermeulen wrote: post #10

A warning to devs building on OpenAI APIs: We spent months developing a chatbot using GPT3 for our game and released a video showcasing it: https://www.youtube.com/watch?v=nnuSQvoroJo&t=264s Afterwards OpenAI then added GPT3 chatbot guidelines disallowing basically anything like this. We were in communication with them beforehand, but they decided later that any sort of free form chatbot was dangerous. What they allo…

> but they decided later that any sort of free form chatbot was dangerous.

Seems like OpenAI saw this video differently. But then again, now OpenAI wants to police how to use GPT-3 and reject or approve what is acceptable for others using their service; since they can change their guidelines at any time.

They need a sense of humour, rather than policing projects like this.

> What they allow changes on a weekly basis, and is different for each customer.

Exactly. I don't know what to say to entire building their entire business on top of OpenAI, since they can just revoke access instantly and simply they may not like what you are doing and will point to the 'guidelines'

> I don't understand how they expect companies to rely on them

Won't be surprised to see Rockstar Games using a tweaked, self-hosted or private version for their future games for this use case, Since OpenAI knows they can get a significant amount of money from large customers like them.

But not from smaller companies.

Post reply on HN