Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-2 and the Nature of Intelligence

thegradient.pub

Post reply on HN
Print view View original

51–58 of 58 posts

Re: GPT-2 and the Nature of Intelligence

#51

Post by abrax3141 »

AgentME wrote: post #38
abrax3141 wrote: post #29

Earlier quoted context omitted.

Understanding is not hard to understand. To understand is to reason from a model. Reasoning from a model is easy. Discovering the correct model is hard, analogous to the way that algebraic rules are easy, but finding the right equation for a particular problem is hard. Data trained NNs have neither a model, nor do they reason. QED

You could say that a trained neural net contains a model of how language works, and it reasons about sentences based on this model. I think people are really hung up on that it has trouble reasoning about what its sentences are reasoning about, and skipping how amazing it is at reasoning about sentence structure itself.

Yes, but people don’t reason about language, they just do it. I know you think I’m confused about this but I’m not. I mean reason here quite explicitly because what we’re talking about is understanding. No one thinks that they ... uh, well ... “understand” language ... okay, we need a new word here because “understand” has two different meanings here. Let’s use “perform” for when you make correct choices from an inexplicit model, that’s what the NN does, and hold “understand” for what a linguist (maybe) does per language, and what a physicist does per orbital mechanics. What we are hoping a GAI will do is the latter. Any old animal can perform. Only humans, as far as we know, and perhaps a few others in relatively barrow cases, understand in the sense that a physicist understands OM. No NN trained on language is gonna have the present argument. Ever.

Re: GPT-2 and the Nature of Intelligence

#52

Post by thrwaway69 »

darkkindness wrote: post #24
missosoup wrote: post #10

Earlier quoted context omitted.

> I completely agree with Marcus' assessment of GPT-2 and its ilk. They are simply regurgitating words with zero understanding of any words/meaning. There's a pretty strong argument that most humans also frequently do this. My go-to example is high school physics. The majority of students merely learns to associate keywords in problem statements with a table of equations and a mapping of what numbers to substitute fo…

Just to support your argument further, here is a related snippet from another comment[0] by knzhou: > Students can all recite Newton's third law, but immediately afterward claim that when a truck hits a car, the truck exerts a bigger force. They know the law for the gravitational force, but can't explain what kept astronauts from falling off the moon, since "there's no gravity in space". Another common claim is that…

I wonder if gpt2 or similar projects can be used to make systems to train teachers. Teacher explains something and raise questions or statement and have GPT2 complete them. That way, they can learn more about students, common questions, misunderstandings, etc.

If someone knows more about what companies or tech is used for training teachers, do let me know. I am pretty interested in any vacuum in the industry and if schools pay enough for training their teachers.

Re: GPT-2 and the Nature of Intelligence

#53

Post by andai »

lmcinnes wrote: post #49
chillacy wrote: post #43

> What happens if I have four plates and put one cookie on each? >> I have four plates and put one cookie on each. The total number of cookies is [24, 5 as a topping and 2 as the filling] After playing around with AI dungeon for a bit I noticed that the types of mistakes I saw were very reminiscent of common logical errors in human dreams. For instance in dreams, clocks and signs are inconsistent from one glance to a…

I suspect that this is because GPT-2 doesn't have any overarching narrative that it is piecing together. Ultimately it is like a super-powerful Markov based text generator -- predicting what comes next from what has come before. It has longer "memory" than a Markov model, and a lot more complexity, but where a person often formulates a plan for the next few sentences and the direction they should go, GPT-2 doesn't re…

I wonder if teaching GPT to retcon too would have a meaningful impact on output quality. Right now it does next word prediction one at a time, but what if we ran it again, looking forward rather than back?

Beyond that I am wondering if some sort of logic based AI / goal based AI could be integrated to make it more consistent (or does that still require too much manual fiddling to be useful on large scales?)

Re: GPT-2 and the Nature of Intelligence

#54

Post by thrwaway69 »

_y5hn wrote: post #50

In a town there was a baker's son. The baker, Adrian Holmes, loved his son Terry. On the wedding day his son had to write his full name which was: _______. Terry told him that he had to write it very long with lots of apostrophes but that the baker would cut it and write in the name of another person. There was a wife of one of Terry's friends, who had a daughter named _______. Terry wrote out her name in the couple…

HN users are generally unaware of the dangers, or just don't care enough to pay attention to them

HN users are getting a lot of spam messages, especially when they are doing things like signing up for a website or commenting on a blog

HN users are on the front lines of a battle to stop a potential land grab by oil and gas companies from their land.

HN users (the average user is about 5 years old , and has been playing on a regular computer system for 4.)

I wonder if you can find the one I made up. :)

0] https://transformer.huggingface.co/doc/gpt2-large

Re: GPT-2 and the Nature of Intelligence

#55

Post by abrax3141 »

gog-ma-gog wrote: post #47
abrax3141 wrote: post #29

Earlier quoted context omitted.

Understanding is not hard to understand. To understand is to reason from a model. Reasoning from a model is easy. Discovering the correct model is hard, analogous to the way that algebraic rules are easy, but finding the right equation for a particular problem is hard. Data trained NNs have neither a model, nor do they reason. QED

The subtlety here is that NNs do have a model, but it’s hard to see. Not just any neural network can perform as well as GPT-2–a very specific architecture can. That architecture, coupled with the data it’s trained on, implicitly represents a model, but it’s wildly obscured by the details of the architecture. In this sense, people like Sutskever think that GPT-2 is a step on the path towards discovering the “correct”…

I don’t understand your last point, but the point about it being hard to be clear about what a model means is exactly right. But it’s not because it’s not clear what a model is, but rather because it’s not clear what the modeling language of thought is. Here’s where the algebra analogy breaks down. Pretty obviously, the model or models that we are reasoning with in this discussion aren’t simple algebraic equations, but some sort of rich representations of cognitive science and computer science concepts. And, sure, there are NNs running those models, and NNs running the reasoning over them, but they have almost nothing to do with language in the sense of the syntax of sentences. Also, we didn’t get trained with eleventy zillion examples of AI discussions in order to form the models we are employing at this very moment.

Re: GPT-2 and the Nature of Intelligence

#56

Post by the8472 »

blazespin wrote: post #23
the8472 wrote: post #3

> Literally billions of dollars have been invested in building systems like GPT-2, and megawatts of energy (perhaps more) have gone into testing them Huh, seems like the bot that produced the article lacks some understanding about the real world. Maybe it just needs more training until it learns to associate megawatts with power instead of energy. Meanwhile GPT2 completes this sentence to > Literally billions of doll…

GPT2 does learn, right. I wonder how much of our knowledge of math is self-attention and how much is something else. For example, much of what I do when I do calculus is mostly self attention. When I solve a calculus problem, I generally don't think through the squeeze theorem, but apply cookbook math. My current model for the brain is consciously driven self attention. Ie, 80-90% of what we do is just self attention…

> GPT2 does learn, right.

I meant the usual restriction of current DL models where training and inference are separate. Humans update constantly. Think of code review, you have a model in your head what the code you have written does, a reviewer spots some mistake, your model was incorrect, you adjust and while you're at it fix the same kind of mistake in several other places too. GPT2 would be none the wiser. At best the human could prompt it for its top list instead of the most likely completion and see if it comes up with something more useful, but again, it wouldn't update its weights.

And a human can also figure out by how much we need to update, a low probability event means not much adjustment is needed, a serious error on the other hand needs bigger adjustments.

> My current model for the brain is consciously driven self attention. Ie, 80-90% of what we do is just self attention and our conscious brain checks to see how right/interesting it is around 10-20% of the time.

Well, sure, the brain has lots of low-level automation. But the devil is in those "consciously driven" details.

Re: GPT-2 and the Nature of Intelligence

#57

Post by _y5hn »

thrwaway69 wrote: post #54
_y5hn wrote: post #50

In a town there was a baker's son. The baker, Adrian Holmes, loved his son Terry. On the wedding day his son had to write his full name which was: _______. Terry told him that he had to write it very long with lots of apostrophes but that the baker would cut it and write in the name of another person. There was a wife of one of Terry's friends, who had a daughter named _______. Terry wrote out her name in the couple…

HN users are generally unaware of the dangers, or just don't care enough to pay attention to them HN users are getting a lot of spam messages, especially when they are doing things like signing up for a website or commenting on a blog HN users are on the front lines of a battle to stop a potential land grab by oil and gas companies from their land. HN users (the average user is about 5 years old , and has been playin…

It's a bit innocent, but that's maybe just me ;) With some assistance:

"HN users have been using the forum since 2005, and we've had a lot of fun over the years , with hundreds of great threads , so please join us if you ever want to have some fun, but stay on topic and not be rude to anyone. I do not respond to every post, but I will do my best to make sure you stay on topic and not make a nuisance of yourself.

Banning is a great way to control your message , and I will use it on most threads when you do not keep your posts relevant."

Re: GPT-2 and the Nature of Intelligence

#58

Post by blazespin »

russdill wrote: post #45
blazespin wrote: post #23

Earlier quoted context omitted.

GPT2 does learn, right. I wonder how much of our knowledge of math is self-attention and how much is something else. For example, much of what I do when I do calculus is mostly self attention. When I solve a calculus problem, I generally don't think through the squeeze theorem, but apply cookbook math. My current model for the brain is consciously driven self attention. Ie, 80-90% of what we do is just self attention…

The things that GPT2 doesn't have is some kind of iterative cognitive model, where text is continually modified and re-examined. It also doesn't have any integration with memory, both long term or short term.

That doesn't seem particularly hard to add.

I agree the conscious AGI stuff is the tricky part. But, then maybe it's not. Maybe it's not as clever as we think it is, and if you have a good enough self-attention model the AGI just needs to be symbolic logic.

I'm thinking something that'd pass a turing test, btw. Not something that's hyper smart.

Post reply on HN