Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Codex

openai.com

Post reply on HN
Print view View original

51–60 of 181 posts

Re: OpenAI Codex

#51

Post by karmasimida »

maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

> The "language models don't really understand anything"

This is still true. By all account, human doesn't need to read 159GB of Python code to write Python, or we simply can't.

But it doesn't necessarily indicate language models aren't useful.

Re: OpenAI Codex

#53

Post by vimy »

parksy wrote: post #38
throwaway128327 wrote: post #19

I don't understand what is going on, why are people even spending time on this? I think this and copilot and etc are solving a non problem of "we will remove the boring part of programming" by generating a bunch of code, so now it's even more boring to read it and check if it actually does what you want. In the same time zero of the developers I interviewed know how a linked list is laid out in memory, or what is the…

This is just nascent technology leading toward something like this: "Computer, I want to play a game." "Okay, what will the game be?" "I want to be a starship captain, give me a cool space ship I can explore the galaxy with" "Okay... like this?" "Not quite, make the galaxy more realistic, with real stars and planets. Also make it 3d. I want to be the captain inside the ship." "How about now?" "Cool, and there should…

Also know as the holodeck from Star Trek.

Re: OpenAI Codex

#54

Post by wantsanagent »

abeppu wrote: post #20

I'm still surprised by the approach. I mean, great that it works this well -- but program synthesis is one of those rare domains where you can observe exactly what the outcome is after you generate something. You can see execution traces, variable values, what the JIT produced, etc. And all of this is relatively cheap -- often executing a code snippet should be far cheaper than an extra pass through a giant DNN right…

FWIW execution guided code synthesis is a thing. Get a few possible outputs and ditch those that don't pass a parser as an example. At least in the SQL generation realm this is well worth the time it takes to tack onto a large language model.

Re: OpenAI Codex

#55

Post by mxwsn »

nradov wrote: post #42
throwaway128327 wrote: post #19

I don't understand what is going on, why are people even spending time on this? I think this and copilot and etc are solving a non problem of "we will remove the boring part of programming" by generating a bunch of code, so now it's even more boring to read it and check if it actually does what you want. In the same time zero of the developers I interviewed know how a linked list is laid out in memory, or what is the…

It seems like they're going in totally the wrong direction. If program content is predictable based on patterns (low entropy) then that's a sign that our programming languages are too low level. If we want to improve developer productivity then the solution is the same as it always has been: create higher level languages which abstract away all the repetitive patterns.

Tools are relatively low level compared to any single use case or field because they should universally support all uses cases or fields. The more narrow your field or use case is, the fewer resources there are to create a higher level language that abstracts away the details that aren't important for your area, but are important to other areas. In this manner, Codex has enormous potential.

Re: OpenAI Codex

#56

Post by jdonaldson »

maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

I think intelligence as defined as "mapping inputs into goal states" is pretty well handled by models, and the models may be able to pick and choose states that are sufficient for achieving the goals.

However, the intelligence that's created by language models is very schizophrenic, and the human-level reflective intelligence that it displays is at best a bit of Frankenstein's monster (an agglomeration of utterances from other people that it uses to form sentences that form opinions of itself or its world).

I think that modeling will help us learn more about human intelligence, but we're going to have to do a lot better than just training models blindly on huge amounts of text.

Re: OpenAI Codex

#57

Post by mxwsn »

GistNoesis wrote: post #50

Can I use this to write solidity contracts ?

That has got to be one of the worst possible use cases one could imagine. In page 33 of the appendix, the authors note that nearly 40% of RSA encryption keys created by Codex are clearly insecure.

Re: OpenAI Codex

#58

Post by maxwells-daemon »

karmasimida wrote: post #51
maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

> The "language models don't really understand anything" This is still true. By all account, human doesn't need to read 159GB of Python code to write Python, or we simply can't. But it doesn't necessarily indicate language models aren't useful.

I would argue humans ingest a lot more than 159GB before they can write code. Most of it isn't Python, and humans currently transfer knowledge a lot more efficiently than NNs, but I suspect that'll change as incorporating more varied data sources becomes feasible.

Re: OpenAI Codex

#59

Post by karmasimida »

f0e4c2f7 wrote: post #11

They just finished a demo on twitch. Pretty crazy! https://www.twitch.tv/videos/1114111652 Starts at 15:45.

It is simultaneously impressive and underwhelming for me.

I mean yes this is a super impressive demo, but it didn't go beyond my expectation. I really want to see whether this model can write a correct binary search method without seeing one before.

Or even correctly using the binary search, does it understand concept like index boundaries?

Re: OpenAI Codex

#60

Post by hackinthebochs »

karmasimida wrote: post #51
maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

> The "language models don't really understand anything" This is still true. By all account, human doesn't need to read 159GB of Python code to write Python, or we simply can't. But it doesn't necessarily indicate language models aren't useful.

Considering the sum total of data and computation that goes in to creating an intelligent human mind, including the forces of natural selection in creating our innate structure and dispositions, it's not obvious that any conclusions can be drawn from the fact that so much data and compute goes into training these models.
Post reply on HN