Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rodney Brooks on GPT-4

spectrum.ieee.org

Post reply on HN
Print view View original

371–380 of 412 posts

Re: Rodney Brooks on GPT-4

#371

Post by krainboltgreene »

ramraj07 wrote: post #192
krainboltgreene wrote: post #168

Earlier quoted context omitted.

Chomsky is an expert of a few things, but I doubt that's what you actually mean to ask. Regardless his editorial matches how scientists think of the human mind and how OpenAI's own creators describe GPT's design.

Citation please, for where you're saying the two parties agree on?

I've not said anything contentious or secret you can literally read the article in question above and also read the OpenAI website: https://openai.com/research/instruction-following:

> This is in part because GPT-3 is trained to predict the next word on a large dataset of Internet text, rather than to safely perform the language task that the user wants.

Re: Rodney Brooks on GPT-4

#372

Post by espe »

catgoose wrote: post #332
espe wrote: post #289

Earlier quoted context omitted.

text and language intersect. in some ways, text is a superset of language, mostly due to social, or what is also called pragmatic, factors that complement semantics. also, the semantics/syntax interface is everything else than clear cut, at least in natural human languages.

I think about this as shape and color. No one ever saw a shape that wasn’t colored and likewise there are no colored things that do not have a shape. Also, displaying text without a font is not possible. Text is the surface of the ocean where waves emerge, and while they have their own properties and may seem to naively have agency, they are an expression of the underlying ocean.

nicely put! many aspects of text at least historically have much to do with its materiality (also in a cognitive development sense, learning how to write etc.). what we can think about nowadays is that text and speech might not be a necessary materiality of language. language might depend more on conceptual systems. more like a substrate of intelligence and that might as well be nonhuman (to stay on topic).

Re: Rodney Brooks on GPT-4

#373

Post by DiogenesKynikos »

RandomLensman wrote: post #315
biorach wrote: post #307

Earlier quoted context omitted.

> For example, a bunch of 7 dimensional objects that can only be stacked a certain way. That's a ridiculous example.

Why? You need make sure that a solution requires true understanding and isn't in the training set. If it can reason properly, it shouldn't have a problem with such a problem.

How well do humans reason about 7-dimensional objects?

I'm already impressed if a computer can reason flexibly about 3-dimensional objects.

Re: Rodney Brooks on GPT-4

#374

Post by tornato7 »

RandomLensman wrote: post #270
tornato7 wrote: post #261

Earlier quoted context omitted.

The point is that ChatGPT undeniably built a world model good enough to understand the physical and three-dimensional properties of these items pretty well, and it gives me a somewhat workable way to stack them, despite never having seen that in its training data.

You cannot conclude that from the output - the training data will likely contain a lot stacking things. Everyday objects also might have some stacking properties that make these questions easy to answer even with semi-random answers. Plus, some stuff clearly makes no sense or is ignored (like the gummy worms in the center, forgetting about the succulent in some cases). If you want to test world modeling, give it obje…

Please tell me how you would pose the question of a bunch of seven-dimensional objects that can only be stacked in a certain way.

Re: Rodney Brooks on GPT-4

#375

Post by dragonwriter »

pixl97 wrote: post #358
croes wrote: post #344

Earlier quoted context omitted.

I don't know if and how fast GPT learns from user input.

It doesn't learn directly from user input. Instead there are two options. Taking the user input and putting it in the training corpus and reweighting the neural net. Or, using the user input as up/down votes on the RLHF to alter the output of the weights that already exist.

> It doesn't learn directly from user input.

Depending on what “it” is, it does through in-context learning, though that’s, obviously, limited to the context window.

Re: Rodney Brooks on GPT-4

#376

Post by RandomLensman »

DiogenesKynikos wrote: post #373
RandomLensman wrote: post #315

Earlier quoted context omitted.

Why? You need make sure that a solution requires true understanding and isn't in the training set. If it can reason properly, it shouldn't have a problem with such a problem.

How well do humans reason about 7-dimensional objects? I'm already impressed if a computer can reason flexibly about 3-dimensional objects.

Humans having the right mathematical tooling do ok.

Re: Rodney Brooks on GPT-4

#377

Post by onethought »

atq2119 wrote: post #247
onethought wrote: post #170

Earlier quoted context omitted.

> considering the scale of the matter, i.e. human extinction. There is literally no evidence that this is the scale of the matter. Has AI ever caused anything to go extinct? Where did this hypothesis (and that's all it is) come from? Terminator movies? It's very frustrating watching experts and the literal founder of lesswrong reacting to pure make believe. There is no disernable/convincing path from GPT4 -> Human Ex…

> Has AI ever caused anything to go extinct? We know from human history that intelligence tends to cause extinctions. AI just hasn't been around long enough, nor been intelligent enough yet. Though, if you count corporations as artificial intelligences, as some suggest, then yes, AIs have in fact already contributed to extinctions.

… This is literally non logical reasoning. If we redefine AI to mean something it’s never been defined as… unfortunately logic has left the chat at that point

Re: Rodney Brooks on GPT-4

#378

Post by onethought »

kristiandupont wrote: post #197
onethought wrote: post #170

Earlier quoted context omitted.

> considering the scale of the matter, i.e. human extinction. There is literally no evidence that this is the scale of the matter. Has AI ever caused anything to go extinct? Where did this hypothesis (and that's all it is) come from? Terminator movies? It's very frustrating watching experts and the literal founder of lesswrong reacting to pure make believe. There is no disernable/convincing path from GPT4 -> Human Ex…

Nuclear bombs have also never caused anything to go extinct. That's no reason not to be cautious. The path is pretty clear to me. An AI that can recreate an improved version of itself will cause an intelligence explosion. That is a mathematical tautology though it could turn out that it would plateau at some point due to physical limitations or whatever. And the situation then becomes: at some point, this AI will be…

But GPT4 can’t access or change its model weights… so crisis averted?

Re: Rodney Brooks on GPT-4

#379

Post by saltcured »

espe wrote: post #289
gmt2027 wrote: post #218

Earlier quoted context omitted.

Text is an instance of language. Think of it as the difference between the python language and a large collection of python programs. The language describes syntactic and semantic rules, the collection is a sampling of possible programs that encodes a significant amount of information about the world. You could learn a lot about the laws of nature, the internet, even human society and laws by examining all the python…

text and language intersect. in some ways, text is a superset of language, mostly due to social, or what is also called pragmatic, factors that complement semantics. also, the semantics/syntax interface is everything else than clear cut, at least in natural human languages.

That relationship seems backwards to me...

Any text corpus is a subset of the language, under the normal definition that a language is the set of all possible sentences (or a set of rules to recognize or generate that set of possibilities). This text subset has an intrinsic bias as to which sentences were selected to represent real language use, which would be significant as a training set for an ML model.

So, perhaps you are saying that the text corpus carries more "world" information than the language, because of the implications you can draw from this selection process? The full language tells us how to encode meaning into sentences, but not what sentences are important to a population who uses language to describe their world. So, if we took a fuzz-tester and randomly generated possible texts to train a large language model, we would no longer expect it to predict use by an actual population. It would probably be more like a Markov chain model, generating bizarre gibberish that merely has valid syntax.

And, this is also seems to apply if you train the model on a selection from one population but then try to use the mode to predict a different population. Wouldn't it be progressively less able to predict usage as the populations have less overlap in their own biased use of language?

Re: Rodney Brooks on GPT-4

#380

Post by blazespin »

moffkalast wrote: post #317
blazespin wrote: post #275

Earlier quoted context omitted.

The resolution is actually fairly simple. It's an incredibly brilliant stochastic parrot with some limited reasoning capabilities. Some folks will try to say it cannot reason, but they are wrong, there is extensive proof of that. The only question is how limited are its reasoning capabilities. After spending extensive time on openai/evals, having submitted 3 of my own, and doing a lot of tests, I would argue that an…

That's probably an accurate assessment, the question is mainly if the reasoning can be improved to a notable extent and how much on the current architecture. I myself assumed that we're pretty close to the end of the S curve when first using 3.5-turbo and figured that hallucinations will be pretty hard to overcome, but with GPT 4 being such a massive improvement on all metrics I'm no longer as sure. GPT 5 will probab…

Exactly, that's the trillion (quadrillion?) dollar question. Have we hit asymptotic reasoning capability or not?

There are interesting things to do with synthetic data, so stochastic parrot might not yet be hitting asymptote.

Post reply on HN