Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Codex

openai.com

Post reply on HN
Print view View original

151–160 of 181 posts

Re: OpenAI Codex

#151

Post by andreyk »

maxwells-daemon wrote: post #45

The "language models don't really understand anything" corner is getting smaller and smaller. In the last few months we've seen pretty definitive evidence that transformers can recombine concepts ([1], [2]) and do simple logical inference using contextual information ([3], "make the score font color visible"). I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of peop…

"I see no reason that this technology couldn't smoothly scale into human-level intelligence, yet lots of people seem to think it'll require a step change or is impossible."

I am a big fan of LMs and am not in the don't really understand crowd, but here are a couple of reasons:

1. Large language models such as GPT or Codex still have several major architectural limitations. They lack the ability to make use of long-term memory, since they have a fairly limited amount of info they can take as input; GPT 3 is great at short stories, but can't go beyond that, and it's hard to prime it with a lot of information as you would eg a new employee. There is some work on this, but afaik not very much and it's very much unsolved.

2. Large language models have only gotten this good by ingesting massive amounts of data and scaling up compute. Yet, this growth comes with diminish returns for every order of magnitude. So it just not being to scale either the data or the compute needs sufficiently (with existing hardware architectures) is a very plausible reason.

3. Large language models 'have it easy' because they only deal with one modality (text). Humans intelligence on the other hand is multimodal - we can process vision inputs, sound, touch, etc. sound, etc. simultaneously and share concepts between these modalities. And we likewise output motor commands that result in motion, text. So far it's not too obvious how to achieve this - OpenAI took a step with DALL-E, but that was by just mining a massive amount of image-text pairs, and it's not obvious this is easy for other modalities, in particular for motor control.

4. Human-level intelligence is often framed as having system 1 (reactive output) and system 2 (longer term reasoning not in response to immediate stimuli) - this is not at all present in language models.

5. related to above two, at least some of human intelligence is derived from reinforcement learning (optimizing a policy that is multi-step with a delayed reward). This is much harder than the plain self-supervised learning of LMs.

And probably there are a bunch more like these. So while I do think these sorts of models represent a lot of progress, there are many reasons to be doubtful that just 'scale it up' will work to get much further.

Re: OpenAI Codex

#152

Post by 6gvONxR4sf7o »

krisoft wrote: post #143
6gvONxR4sf7o wrote: post #69

Earlier quoted context omitted.

> The "language models don't really understand anything" corner is getting smaller and smaller. In my mind, understanding a thing means you can justify an answer. Like a student showing their work and being able to defend it. An answer with a proof understands the answer with respect to the proof it provides. E.g. to understand an answer with regards to first order logic, it'll have to be able to defend a logical ded…

> In my mind, understanding a thing means you can justify an answer. Do you understand cats? If I show you a picture of either a cat or a dog do you think you can tell which one it is? I think most people could solve that challenge, and if pressed they could vax poetically about what makes them think it is a cat. Maybe they would mention the shape of an ear, or talk about feline grace or what have you. But is that re…

The point I’m trying to make (poorly) is that i don’t think a one size fits all definition of “understanding” is useful. It’s more useful to define understanding with respect to how you can justify a thing you know.

So for the case of cats, I will understand cats at a different level from a cat biologist. I can point to features that seem catlike, and they can talk about all sorts of other scientific things that make a cat a cat.

With respect to that sciency kind of understanding, I don’t understand cats. With respect to a much looser ‘point at the features’ kind of understanding, I do understand cats.

Re: OpenAI Codex

#153

Post by naresh_xai »

hayksaakian wrote: post #149
krisoft wrote: post #143

Earlier quoted context omitted.

> In my mind, understanding a thing means you can justify an answer. Do you understand cats? If I show you a picture of either a cat or a dog do you think you can tell which one it is? I think most people could solve that challenge, and if pressed they could vax poetically about what makes them think it is a cat. Maybe they would mention the shape of an ear, or talk about feline grace or what have you. But is that re…

You could probably teach an AI to post-hoc rationalize their decisions, the same way people do.

That’s basically the gist of explainable AI

Re: OpenAI Codex

#154

Post by naresh_xai »

beering wrote: post #118
6gvONxR4sf7o wrote: post #69

Earlier quoted context omitted.

> The "language models don't really understand anything" corner is getting smaller and smaller. In my mind, understanding a thing means you can justify an answer. Like a student showing their work and being able to defend it. An answer with a proof understands the answer with respect to the proof it provides. E.g. to understand an answer with regards to first order logic, it'll have to be able to defend a logical ded…

> In my mind, understanding a thing means you can justify an answer. What if the language model can generate a step-by-step explanation in the form of text? [0] There's no guarantee that the reasoning was used to come up with the answer in the first place, and no proof that the reasoning isn't just the product of "a really fancy markov chain generator", but would you accept it? We're really walking into Searle's Chin…

Umm, no there are clear verification methods for Explainable AI techniques today. One way to check the justification would be if things which were important in the justification were removed in some sense, then would the output change signficantly. Sort of like a sensitivity test for justification.

Re: OpenAI Codex

#155

Post by naresh_xai »

randallsquared wrote: post #148
simsla wrote: post #128

Earlier quoted context omitted.

The problem with this (very popular) argument is that you can't give a CS course to a baby and expect them to get at programming. By the time we see our first line of code, most of us have seen a ridiculous amount of data. We've been trained in problem solving, logical reasoning, maths, natural language processing, ... Hell, we've been trained as pattern matchers since we've been born. By my account, humans actually…

Taking 11Mbps [1] as the raw uncompressed incoming data, and assuming 16 hours of waking environment consumption on average (likely high for children), a 13yo has taken in less than 400 TB of information (I used 11 * 60 * 60 * 16 * 365 * 13 / 8.) That's... surprisingly low. [1] https://www.britannica.com/science/information-theory/Physio...

Are we still limiting to visual cues and not the auditory,smell,taste,touch data which we get exposed to?

Re: OpenAI Codex

#156

Post by tyingq »

"Converting Python to Ruby with OpenAI Codex"

Oof. Looking forward to maintaining some future ports done with this tool.

Re: OpenAI Codex

#157

Post by Cybiote »

krisoft wrote: post #143
6gvONxR4sf7o wrote: post #69

Earlier quoted context omitted.

> The "language models don't really understand anything" corner is getting smaller and smaller. In my mind, understanding a thing means you can justify an answer. Like a student showing their work and being able to defend it. An answer with a proof understands the answer with respect to the proof it provides. E.g. to understand an answer with regards to first order logic, it'll have to be able to defend a logical ded…

> In my mind, understanding a thing means you can justify an answer. Do you understand cats? If I show you a picture of either a cat or a dog do you think you can tell which one it is? I think most people could solve that challenge, and if pressed they could vax poetically about what makes them think it is a cat. Maybe they would mention the shape of an ear, or talk about feline grace or what have you. But is that re…

Take entomologists, bird watchers or those who identify mushrooms. In each, there are instances that are fiendishly difficult to tell apart. If you ask an expert for advice, they'll tell what features to look for and where, sometimes not even on the item itself and some requiring specialists tools.

While explanations are far from sufficient to instantly close the gap to expertise, they provide a massive boost that you might not otherwise have found on your own. The justification comes from the fact that their explanations are a reliable source of increased performance in making fine-grained distinctions. It's further demonstrated by answers to questions like "If they are so difficult to tell apart, why make these distinctions?" or "How did they come to be so similar?".

Re: OpenAI Codex

#158

Post by spyder »

krisoft wrote: post #134
maxwells-daemon wrote: post #81

Earlier quoted context omitted.

Look at the "math test" video. Given the question: "Jane has 9 balloons. 6 are green and the rest are blue. How many balloons are blue?" The model outputs: "jane_balloons = 9; green_balloons = 6; blue_balloons = jane_balloons - green_balloons; print(blue_balloons)" That seems like a good justification of a (very simple) step-by-step reasoning process!

I wonder what would it have outputted if we would remove the “ and the rest are blue” part from the question. Would not surprise me if an innatentive human student would answer that with the same code. After all school “trains” people to expect such challenges to be solveable. A more attenive human might say “we can’t know” or provide an upper limit to the number of potential blue balloons.

Related article: Teaching GPT-3 to Identify Nonsense

https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/

Re: OpenAI Codex

#159

Post by librexpr »

polyanos wrote: post #103
vincnetas wrote: post #80

Earlier quoted context omitted.

Also curious what this ai would produce when provided with contradictory requests. Because often there are multiple requirements which on theyr own sounds reasonable but when you try to fit all requirements in one system, things get nasty.

It is only able to translate small instructions into code. I think it will take a while to get to a situation where you can just give it a list of requirements and it spits a working program. Hell it messed up when they gave it the instruction "make every fifth line bold" in their Word api part of the demo, where it made the first line of every paragraph (which is only 4 lines long in total) bold instead of every fif…

It didn't mess up the instruction "make every fifth line bold". The blank spaces between each "paragraph" are empty lines, so it counted them too. I think this is perfectly reasonable behavior, it's what I would have done absent further instructions too.

You can see it in the generated code on the bottom right during that part of the demo. It loops over the lines and bolds them when index % 5 == 0.

Edit: I guess with the 1-based indexing of natural languages, the code actually bolds lines number 1, 6, etc. So arguably it should have done index % 5 == 4 instead, to bold lines number 5, 10, etc. But funnily enough, if it had done that, it would have bolded all the empty lines, so it would have seemed like it didn't do anything.

Re: OpenAI Codex

#160

Post by modeless »

kevinqi wrote: post #124
ruuda wrote: post #110

Earlier quoted context omitted.

> but they will likely take advantage of the AST In the end, a more general approach with more compute, always wins over applying domain knowledge like taking advantage of the AST. This is called “the bitter lesson”. http://www.incompleteideas.net/IncIdeas/BitterLesson.html

"the bitter lesson" is a very interesting, thank you! However, I wonder if AST vs. text analysis is fully comparable to the examples given in the post. Applying human concepts for chess, go, image processing, etc. failed over statistical methods, but I don't think AST vs. text is exactly the same argument. IMO, using an AST is simply a more accurate representation of a program and doesn't necessarily imply an attempt…

I mean, the AST doesn't help at all with comments which are potentially the most valuable part of the code to an AI like this. Formatting is also ignored by the AST but may play a role in understanding, just as it can for humans.

The model can clearly already generate large amounts of code with no syntax errors in one shot. It's probably better at that than I am, I always need to fix something after typing a bunch of code without calling the compiler. I think that instead of adding a bunch of language-specific AST stuff it would be far better to simply give the model the ability to iterate on its solution the way humans do, to fix any syntax errors or logic bugs discovered by the compiler or at runtime. That could potentially work in a generic way for any language. It seems like the obvious next step, though figuring out how to train it is not obvious.

Post reply on HN