Live data from Hacker News

Rodney Brooks on GPT-4

spectrum.ieee.org

271–280 of 412 posts

Re: Rodney Brooks on GPT-4

#271
post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

Has there been any research in the possibility that much of what we say/speak (including this convo) is actually just stochastic parroting? e.g. instead of the stochastic parrots mimicking intelligence maybe intelligence doesn't exist, it's just stochastic parrots of various levels of sophistications organized into a hierarchy. "Intelligence" is necessarily socially defined with the more complex parrots being unpredi…

BF Skinner, “Verbal Behavior”. Critiquing that book is what put Chomsky on the map.

Re: Rodney Brooks on GPT-4

#272
One of my goals in life before dementia sets in for real is to devise some model, perhaps a conceptual framework which will allow us to escape the clutches of habitual simplification, a subset of which is dichotomous thinking, which in turn leads to the inevitable painting of strawmen as a way to prove our point (among other things). How sweet it would be to shortcut all the mandatory twists and turns of discourse that follows: "this is a mischaracterization of x", "not all x are y", "x and z are really not opposites, but overlapping", "this is a spectrum with a bell curve, not an either/or" etc. But of course, we all do this, not because we can't think clearly, but because we have an agenda, or maybe more frequently: want to trash talk the stance of an opponent because of what the proliferation of that stance might lead to, and so on. Taking such into account should be an integral part of the conceptual framework, obviously.

In the case at hand, one could easily argue that people in the debate are creating false dichotomies: LLMs are either stochastic parrots OR algorithms with an understanding, when in reality they are both (and also something else completely), but acknowledging such would likely require that one doesn't have an axe to grind, a stake in the field or what you might call it. It would require extending some "philosophers charity" to an opponent, that maybe has tried to undercut one's work for decades, in a field steeped in fierce and bitter competition for a name, like academia. Or, in case one has a business in the field, it would require maybe saying something that puts your core business idea in the crosshairs of legislators, or something else that doesn't serve your long term business interests.

Which brings us to this important aspect of this "conceptual framework against simplification" already briefly touched upon, namely identifying the bias of the participants in the debate. My impression is that naming bias has largely gone out of fashion, which is a pity because it is really a necessary part of understanding an argument: it rarely explains it all (that would be a grave simplification), but it is really a vital part of understanding an argument. And a difficult one, because people will go to extreme lengths to hide their agenda. And the current conceptual framework for unravelling bias has largely been occupied by the fact-checking industry: i.e. things are either true or false, and once you are cleared (like most mainstream media) then bias is not questioned. But we can be assured, there is always some bias, and it is usually relevant to name it (if one can see it), even if it infuriates the named party.

Just sayin'.

Re: Rodney Brooks on GPT-4

#273

No world model ? A world model is so obvious papers like these are more confirmation than surprise https://arxiv.org/abs/2305.11169 https://arxiv.org/abs/2210.13382 There a certain sentiment that AGI however you wish to define it won't infact be a "We'll know it when we see it" situation but rather a "AGI will arrive long before consensus reaches its AGI". LLMs have made me believe this will 100% be the case, either…

I find interesting that Turing foresaw all this with the chinese room/turing test thing. If computers can emulate human language to an indiscernible degree of difference from what other humans can do, it is essentially reasoning, for some definitions of reasoning. But how much of it reflects our own understanding of reasoning and efforts in fine tuning? If I show a modern imax movie to a stone age hunter and they confuse it for reality, is the movie reality? People ran from footage of a steam locomotive driving towards the camera. Does that mean the train was real?

Re: Rodney Brooks on GPT-4

#275
post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

The resolution is actually fairly simple. It's an incredibly brilliant stochastic parrot with some limited reasoning capabilities.

Some folks will try to say it cannot reason, but they are wrong, there is extensive proof of that.

The only question is how limited are its reasoning capabilities. After spending extensive time on openai/evals, having submitted 3 of my own, and doing a lot of tests, I would argue that an average person of average IQ could out think GPT4 - as long as the stochastic parrot aspect wasn't a factor.

Re: Rodney Brooks on GPT-4

#276

Earlier quoted context omitted.

The issue with all these experts is they still think it's human nature to be able to fully understand the world before they speak about it. On the contrary it's human nature (and all animal nature) to figure out how to navigate the world without fully understanding or having a complete model of it. All you need is a working model that affects the facets of the world you need to deal with. I still remember in the 90s…

> The issue with all these experts is they still think it's human nature to be able to fully understand the world before they speak about it None of the experts think this.

Especially Rodney Brooks.

Re: Rodney Brooks on GPT-4

#277

One of my goals in life before dementia sets in for real is to devise some model, perhaps a conceptual framework which will allow us to escape the clutches of habitual simplification, a subset of which is dichotomous thinking, which in turn leads to the inevitable painting of strawmen as a way to prove our point (among other things). How sweet it would be to shortcut all the mandatory twists and turns of discourse th…

The limitations of language make adversarial debate a powerful tool for advancing our understanding. It might be construed as unfortunate, but it brought us this far.

Maybe when we learn higher dimensional ways of communicating we can get better tools for constructing common knowledge.

Re: Rodney Brooks on GPT-4

#278
post #33

> The large language models are a little surprising. I’ll give you that. I think this is the key point about LLMs that kind of explains the wide and polarized views on whether it understands or parrots, whether it can think or is the precursor to thinking or is a dead-end, whether it will catastrophically destroy the world, or “merely” make it steadily worse with bullshit, or just put a few industries out of a job. A…

The resolution is actually fairly simple. It's an incredibly brilliant stochastic parrot with some limited reasoning capabilities. Some folks will try to say it cannot reason, but they are wrong, there is extensive proof of that. The only question is how limited are its reasoning capabilities. After spending extensive time on openai/evals, having submitted 3 of my own, and doing a lot of tests, I would argue that an…

It's very hard to know because we don't know and can't experiment with its training data. So - it may be doing first principle reasoning, or it may be doing token substitution vs. some known example that it's seen before and is matching to.

Re: Rodney Brooks on GPT-4

#279
post #278

Earlier quoted context omitted.

The resolution is actually fairly simple. It's an incredibly brilliant stochastic parrot with some limited reasoning capabilities. Some folks will try to say it cannot reason, but they are wrong, there is extensive proof of that. The only question is how limited are its reasoning capabilities. After spending extensive time on openai/evals, having submitted 3 of my own, and doing a lot of tests, I would argue that an…

It's very hard to know because we don't know and can't experiment with its training data. So - it may be doing first principle reasoning, or it may be doing token substitution vs. some known example that it's seen before and is matching to.

The dictionary definition of reasoning says nothing about how the thinking is done, only that it's sensible and logical, which is exactly what GPT4 is. Limited, yes, but it reasons.

Re: Rodney Brooks on GPT-4

#280

Earlier quoted context omitted.

> My current (tentative) resolution of the surprise is that language encoded way more information about reality than we thought it did. I think you are close to the mark, but you have been subtly mislead: language is not the data we are working with. We are working with text . Once you fix that particular failure of word choice, everything else becomes much more clear: text contains much more information than languag…

There’s an exercise that some people do when learning programming, which is to write down the steps to make a sandwich. Then the teacher follows the exact instructions to make a sandwich and most people don’t put enough detail for a computer to follow (I.e. open the fridge etc) and the teacher will run around bumping into things. That used to be a teaching exercise to show people the amount of precision required when…

Even then, you can see some of the "pedantic" cases when it comes to actually understanding the nature of the the connections between those concepts. For example, it's very easy to get it to reverse shorter/taller or younger/older of clearly defined relationships.
Post reply on HN