Rodney Brooks on GPT-4 • Hacker News Board

Re: Rodney Brooks on GPT-4

#181

Post by Sai_ » Sun, May 21, 2023, 5:31 AM UTC

notShabu wrote: post #177

fwlr wrote: post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

Has there been any research in the possibility that much of what we say/speak (including this convo) is actually just stochastic parroting? e.g. instead of the stochastic parrots mimicking intelligence maybe intelligence doesn't exist, it's just stochastic parrots of various levels of sophistications organized into a hierarchy. "Intelligence" is necessarily socially defined with the more complex parrots being unpredi…

This is an interesting question.

To paraphrase as per my understanding of your comment, is intelligence an emergent property of being able to interact with each other through language?

Say I speak gibberish (to you) which is actually me explaining to you the theory of relativity, would you consider me intelligent?

Re: Rodney Brooks on GPT-4

#182

Post by cf141q5325 » Sun, May 21, 2023, 5:34 AM UTC

jiggawatts wrote: post #110

This is a terrible article written by someone who doesn't seem to have even tried GPT 4. Their only example references GPT 3.5, for example, and then they waffle on about only vaguely related topics such as level 5 self-driving. This quote in particular stood out as ignorant: “What the large language models are good at is saying what an answer should sound like, which is different from what an answer should be.” That…

Great posts. I think its an error caused by a mistake often made on the topic, the assumption that side effects we see now are some fundamental problem and not just an artifact of the way systems are trained and used. And of how we (mal)function.

Especially tightly embracing the cognitive bias of how special and wonderful our intelligence is. After all we have that fancy squishy brain which we assume to be essential. As far as i can tell the only visible bottlenecks when looking into the future come into view once you start debating intelligence vs emulating intelligence. And if thats really the metric some honest introspection about the nature of human intelligence might be in order.

Not sure how much of that is done purposefully to not get too much urgency in figuring out outer alignment on a societal level. Just as its no wonder that we havent figured out how to deal with fake news while at the same time insisting on malinformation existing, its really no wonder that we cant figure out AI alignment while not having solved human alignment. Nobody should be surprised that the cause of problems might be sitting in front of the machine.

Re: Rodney Brooks on GPT-4

#183

Post by tornato7 » Sun, May 21, 2023, 5:34 AM UTC

dragonwriter wrote: post #148

fwlr wrote: post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

> One way to resolve this surprise is to find some reason to believe these strange abilities are fundamentally not an understanding of the world. Thus stochastic parrots, this article, Yan LeCun and Chomsky, etc. I mean, it is kind of obvious that predicting text based on a large corpus of text written by people with a model of the world will, if it works at all, look like having a model of the world. The question is…

The one thing that has convinced me that chatGPT has built a real world model is asking it how to stack different objects. You can ask it for the safest way to stack a pillow, an egg, a nail, a book and an action figure. Even get more complicated. GPT-4 will, most of the time, correctly reason through how to stack these items to form a stable structure without breaking. That tells me it knows intimately the physical properties of all of these objects and how they interact with each other.

Re: Rodney Brooks on GPT-4

#184

Post by jltsiren » Sun, May 21, 2023, 5:35 AM UTC

jiggawatts wrote: post #123

sheeshkebab wrote: post #5

Earlier quoted context omitted.

Anytime I ask these things something (bard, gpt etc), 33% of the answer is genius, 33% misleading garbage, 33% filler stuff that’s neither here or there The problem is distinguishing between these parts requires me to be be an expert in the area I’m inquiring about - and then why the heck do I need to ask some idiot bot for answers to questions that I already know an answer to? I don’t know who finds these things use…

Bard is a brain-damaged-but-literate idiot compared to GPT 3, which is still dumber than the typical human. Try GPT 4 for a week. I've found it to be more like 50% immediately useful, 25% very impressive, and 25% where it's not wrong but I have to poke it a few times with different prompts to coax out the specific answer I'm looking for. That's better than most humans that I collaborate with at work. Literally half o…

> Literally half of humans -- in a professional IT setting -- can't understand simplified, clear english in emails. Similarly, in my experience about half can't follow simple A -> B logic.

There are alternate hypotheses.

People have preferences. When it appears that someone does not understand something, they may be pretending they don't understand it, or they may simply be ignoring it. Maybe they are trying to avoid an unpleasant task, or maybe they find dealing with a specific person unpleasant and not worth the effort.

In my experience, people are far more capable and competent when they feel comfortable and are interested in the task.

Re: Rodney Brooks on GPT-4

#185

Post by nopinsight » Sun, May 21, 2023, 5:36 AM UTC

fwlr wrote: post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

There is in fact compelling evidence that language models can learn some world models.

Here's a research paper on this and a blog post by the lead author summarizing the results.

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. https://arxiv.org/abs/2210.13382

Do Large Language Models learn world models or just surface statistics? https://thegradient.pub/othello/

-----

An argument based on common sense can also be made: Any system that possesses a wide range of capabilities, most of which it was not specifically trained to perform, cannot possibly perform all these tasks so well solely by making probabilistic guesses.

(Humans, too, were not directly shaped by natural and sexual selection to possess all of our cognitive capacities.)

Re: Rodney Brooks on GPT-4

#186

Post by hattmall » Sun, May 21, 2023, 5:36 AM UTC

zainhoda wrote: post #172

thomastjeffery wrote: post #107

Earlier quoted context omitted.

> My current (tentative) resolution of the surprise is that language encoded way more information about reality than we thought it did. I think you are close to the mark, but you have been subtly mislead: language is not the data we are working with. We are working with text . Once you fix that particular failure of word choice, everything else becomes much more clear: text contains much more information than languag…

There’s an exercise that some people do when learning programming, which is to write down the steps to make a sandwich. Then the teacher follows the exact instructions to make a sandwich and most people don’t put enough detail for a computer to follow (I.e. open the fridge etc) and the teacher will run around bumping into things. That used to be a teaching exercise to show people the amount of precision required when…

I don't get the "magic" people are seeing. It makes sense.

>LLMs have somehow learned to fill in the blanks

It's not somehow, it's because they have read a ton of books, documents, etc and can make enough links between cheese and refrigerator and follow that back to know that a refrigerator needs to be opened.

I have seen a lot of very clever AI examples using the latest tools, but I haven't seen anything that seems difficult to deconstruct.

Re: Rodney Brooks on GPT-4

#187

Post by galaxyLogic » Sun, May 21, 2023, 5:37 AM UTC

beardedwizard wrote: post #105

krainboltgreene wrote: post #90

Earlier quoted context omitted.

> it's "read" a significant fraction of everything we've ever written about chess I doubt heavily that a significant fraction of chess's writings are even available in digital format, much less inside of CommonCrawl and correctly trained on.

Why would common crawl NOT contain a chess manual? the rules are explained in detail on wikipedia. the simplest conclusion is that it has indeed been trained on a chess manual and is good at predicting what the next word in a chess manual is. it is not synthesizing anything.

I haven't seen this mentioned but can LLMs actually play chess?

I'm sure they have read rules of chess online, but if you ask them to play chess with you, what happens? Can they apply the rules? Can they apply them intelligently and win the game?

My point is that even though LLMs "know" what the rules of chess are, they don't really "understand" them, unless they can use them to play the game and play it well.

Re: Rodney Brooks on GPT-4

#188

Post by keyle » Sun, May 21, 2023, 5:38 AM UTC

fwlr wrote: post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

> or just put a few industries out of a job.

Yeah no big deal. Happens in history all the time, within a year.

Re: Rodney Brooks on GPT-4

#189

Post by groestl » Sun, May 21, 2023, 5:38 AM UTC

anybodyz wrote: post #115

fwlr wrote: post #74

Earlier quoted context omitted.

> A human being who did that without ever playing a game would also start out better than a typical novice. I am quite skeptical of these arguments along the lines of “imagine a human read everything written on the topic…”. What humans are doing when they read something is not what neural nets are doing when they read something. Humans are (idealistically) doing something like Feynman’s description of how he reads (o…

What LLM's are doing when they imagine playing chess is what we do when we stand up after sitting on the floor, or what we do when we see a few million individual samples of color and light intensity and realize there's an apple and a knife in front of us. I think what is almost impossible for most people to understand is that AI's do not need to be structured like the human brain and use the crutches we use to solve…

Also, there is no reason to believe that playing chess in our head is anything else but us pattern matching a mental process on a higher level, recognizing a simulation there, and feeding that info back into the loop below. Nature provided us with a complex, layered and circular architecture of the brain, but the rest is pretty much training that structure. And we know that different architectures with similar outcome are possible, since there are vast variations across our own species, and other species as well, with essentially the same capabilities.

Re: Rodney Brooks on GPT-4

#190

Post by Nimitz14 » Sun, May 21, 2023, 5:42 AM UTC

jiggawatts wrote: post #110

This is a terrible article written by someone who doesn't seem to have even tried GPT 4. Their only example references GPT 3.5, for example, and then they waffle on about only vaguely related topics such as level 5 self-driving. This quote in particular stood out as ignorant: “What the large language models are good at is saying what an answer should sound like, which is different from what an answer should be.” That…

> Go ask GPT 4 -- not 3.5 -- what it thinks about elephants flying to the moon. Then, and only them go write a snarky IEEE article.

It's hard to know what things have been seen in the training data and are only therefore correct. And GPT4 is large enough that it can generalize from learning that x doesn't make sense that y also doesn't make sense. Does that mean it *understands*? Maybe. But it doesn't have persistent state and can't do math. It's definitely not yet what we think of when we say AGI.