Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

241–250 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#241

Post by hulitu »

atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

> if the intelligence of LLMs proves to be analogous to our own in some deep way

First, they have to implement "intelligence" for LLMs, then we can compare. /s

Re: Chomsky on what ChatGPT is good for (2023)

#242

Post by hackinthebochs »

wat10000 wrote: post #216
hackinthebochs wrote: post #174

Earlier quoted context omitted.

LLMs are modelling the world, not just "predicting the next token". They are certainly not akin to parrots. Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. [1] https://arxiv.org/abs/2405.15943 [2] https://x.com/OwainEvans_UK/status/1894436637054214509 [3] https://www.anthropic.com/research/tracing-thoughts-language...

Maybe it takes some world modeling to do it as well as they do, but ultimately they are just predicting the next token. These things are not mutually exclusive.

The issue is whether they are "just" predicting the next token. When people say they are stochastic parrots, they are denying any of these further capabilities. Modelling is facet of understanding and so to discover that LLMs model the world should strongly raise your credence that they do understand.

Re: Chomsky on what ChatGPT is good for (2023)

#243

Post by CamperBob2 »

Groxx wrote: post #202
geysersam wrote: post #184

Earlier quoted context omitted.

How can you make that claim? Have you ever used an LLM that hasn't encountered high school algebra in it's training data? I don't think so.

I have at least encountered many LLMs with many school's worth of algebra knowledge, but fail miserably at algebra problems. Similarly, they've ingested human-centuries or more of spelling bee related text, but can't reliably count the number of Rs in strawberry. (yes, I understand tokenization is to blame for a large part of this. perhaps that kind of limitation applies to other things too? )

Similarly, they've ingested human-centuries or more of spelling bee related text, but can't reliably count the number of Rs in strawberry

Sigh

Re: Chomsky on what ChatGPT is good for (2023)

#245

Post by 0xbadcafebee »

geysersam wrote: post #180
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

The terms are too unclear here. Can you define what it means to "be able to parse human language"? I'm sure contemporary chatbots score higher on typical reading comprehension tests than most humans. You're certainly correct that llms "only" react to stimuli with a trained response, but I guess anything that isn't consciousness necessarily fits that description

Good point, thanks for calling that out. I'm honestly not sure myself! On further reflection, it's probably a matter of degrees?

So for example, a soldier is trained, and then does what it is told. But the soldier also has a deep trough of contextual information and "decision weights" which can change its decisions, often in ways it wasn't trained for. Or perhaps to put it another way: it is capable of operating outside the parameters it was given, "if it feels like it", because the information the soldier processes at any given time may make it not follow its training.

A dog may also disobey an order after being trained, but it has a much smaller range of information it works off of, and fewer things influence its decision-making process. (genetics being a big player in the decision-making process, since they were literally bred to do what we want/defend our interests)

So perhaps a chat AI, a dog, and a soldier, are just degrees along the same spectrum. I remember reading something about how we can get AI to be about as intelligent as a 2-year-old, and that dogs are about that smart. If that's the case (and I don't know that it is; I also don't know if chat AI is actually capable of "disobeying", much less "learning" anything it isn't explicitly trained to learn), then the next question I'd have is, why isn't the AI able to act and think like a dog yet?

If we put an AI in a robot dog body and told it to act like a dog, would it? Or would it only act the way that we tell it dogs act like? Could/would it have emergent dog-like traits and spawn new dog lineages? Because as far as I'm aware, that's not how AI works yet; so to me, that would mean it's not actually doing the things we're talking about above (re: dogs/soldiers)

Re: Chomsky on what ChatGPT is good for (2023)

#246

Post by 0xbadcafebee »

Xss3 wrote: post #167
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

Dogs out of all creatures probably actually do have some parsing going on for human language. They learn it like we do, picking up context from the environment, actions, tone, facial expressions, body language, etc. You can say 'what's that' in many different ways and a clever dog will react differently for each, even if it's the first time it's heard you say 'what's that?' In a scared tone it'll still react differen…

Oh sure, dogs definitely learn, they have about a 2 year old's level of intellect if I remember correctly? I just meant as example to the train/response thing. I believe the issue is their brains just aren't going to continue "leveling up" the way a human's does as it grows. My assumption is that this is because the AI isn't actually "understanding and thinking", but just acting according to its training. A program following source code, rather than rewriting its own source code, as it were.

Re: Chomsky on what ChatGPT is good for (2023)

#247

Post by csomar »

loveparade wrote: post #201
atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and…

> it has made me question whether such a thing even exists

I was reading a reddit post the other day where the guy lost his crypto holdings because he input his recovery phrase somewhere. We question the intelligence of LLMs because they might open a website, read something nefarious, and then do it. But here we have real humans doing the exact same thing...

> I guess humans really aren't so special after all

No they are not. But we are still far from getting there with the current LLMs and I suspect mimicking the human brain won't be the best path forward.

Re: Chomsky on what ChatGPT is good for (2023)

#248

Post by krige »

csomar wrote: post #247
loveparade wrote: post #201

Earlier quoted context omitted.

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and…

> it has made me question whether such a thing even exists I was reading a reddit post the other day where the guy lost his crypto holdings because he input his recovery phrase somewhere. We question the intelligence of LLMs because they might open a website, read something nefarious, and then do it. But here we have real humans doing the exact same thing... > I guess humans really aren't so special after all No they…

> But here we have real humans doing the exact same thing...

I'd wager that a motivation in designing these systems it so they do not make these mistakes. Otherwise what's the point, really.

Re: Chomsky on what ChatGPT is good for (2023)

#249

Post by guappa »

Workaccount2 wrote: post #213
hyperadvanced wrote: post #186

Earlier quoted context omitted.

Reason about: sure. Independently solve novel ones without extreme amounts of guidance: I have yet to see it. Granted, for most language and programming tasks, you don’t need the latter, only the former.

99.9% of humans will never solve a novel problem. It's a bad benchmark to use here

But they will solve a problem novel to them, since they haven't read all of the text that exists.
Post reply on HN