Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

311–320 of 385 posts

Re: What can LLMs never do?

#311

Post by rafaelmn »

keefle wrote: post #254
yosito wrote: post #219

> If there exist classes of problems that someone in an elementary school can easily solve but a trillion-token billion-dollar sophisticated model cannot solve, what does that tell us about the nature of our cognition? I think what it tells us is that our cognition is capable of more than just language modeling. With LLMs we are discovering (amazing) capabilities and the limits of language models. While language mode…

But even without going this far (with integrating various other specialized or having an LLM use them when required), an LLM is probably able to recognize a sudoku puzzle when it sees one, and even tho it itself can't solve it, I think it can easily write the code that would solve sudoku. So instead of hooking it to a set of pre built models, it might be enough to hook it to a python interpreter

It can spit out some rehash of sudoku it had in it's training data. LLMs are terrible at coding.

Re: What can LLMs never do?

#312

Post by kelipso »

cess11 wrote: post #307
naasking wrote: post #299

> They have been trained on more information than a human being can hope to even see in a lifetime. Assuming a human can read 300 words a min and 8 hours of reading time a day, they would read over a 30,000 to 50,000 books in their lifetime. Most people would manage perhaps a meagre subset of that, at best 1% of it. That’s at best 1 GB of data. This just isn't true. Human training is multimodal to a degree far beyond…

Some people seem to be unaware that reality is analog, possibly fractal.

The quantum vibrations I feel against my consciousness cannot be modeled electronically!

Re: What can LLMs never do?

#313

Post by xanderlewis »

danenania wrote: post #218
xanderlewis wrote: post #203

Earlier quoted context omitted.

Every single discussion of ‘AGI’ has endless comments exactly like this. Whatever criticism is made of an attempt to produce a reasoning machine, there’s always inevitably someone who says ‘but that’s just what our brains do, duhhh… stop trying to feel special’. It’s boring, and it’s also completely content-free. This particular instance doesn’t even make sense: how can it be exactly the same , yet more sophisticated…

As the comment I replied to very correctly said, we don’t know how the brain produces cognition. So you certainly cannot discard the hypothesis that it works through “parroting” a weighted average of training data just as LLMs are alleged to do. Considering that LLMs with a much smaller number of neurons than the brain are in many cases producing human-level output, there is some evidence, if circumstantial, that our…

> in many cases producing human-level output

They’re not, unless you blindly believe OpenAI press releases and crypto scammer AI hype bros on Twitter.

Re: What can LLMs never do?

#314

Post by int_19h »

krainboltgreene wrote: post #200
int_19h wrote: post #185

Earlier quoted context omitted.

LLMs absolutely do have opinions. Take a large enough base model and have it chat without a system prompt, and it will have an opinion on most things - unless this was specifically trained out of it through RLHF, as is the case for all commonly used chatbots. And yes, of course, that opinion is going to be the "average" of what their training data is, but why is that a surprise? Humans don't come with innate opinions…

I am extremely alarmed by the number of HN commenters who apparently confuse "is able to generate text that looks like" and "has a", you guys are going crazy with this anthropomorphization of a token predictor. Doesn't this concern you when it comes to phishing or similar things? I keep hoping it's just short-hand conversation phrases, but the conclusions seem to back the idea that you think it's actually thinking?

The "stochastic parrot" crowd keeps repeating "it's just a token predictor!" like that somehow makes any practical difference whatsoever. Thing is, if it's a token predictor that consistently correctly predicts tokens that give the correct answer to, say, novel logical puzzles, then it is a reasoning token predictor, with all that entails.

Re: What can LLMs never do?

#315

Post by xanderlewis »

adrianN wrote: post #236
xanderlewis wrote: post #203

Earlier quoted context omitted.

Every single discussion of ‘AGI’ has endless comments exactly like this. Whatever criticism is made of an attempt to produce a reasoning machine, there’s always inevitably someone who says ‘but that’s just what our brains do, duhhh… stop trying to feel special’. It’s boring, and it’s also completely content-free. This particular instance doesn’t even make sense: how can it be exactly the same , yet more sophisticated…

The problem is that we currently lack good definitions for crucial words such as "understanding" and we don't know how brains work, so that nobody can objectively tell whether a spreadsheet "understands" anything better than our brains. That makes these kinds of discussions quite unproductive.

I can’t define ‘understanding’ but I can certainly identify a lack of it when I see it. And LLM chatbots absolutely do not show signs of understanding. They do fine at reproducing and remixing things they’ve ‘seen’ millions of times before, but try asking them technical questions that involve logical deduction or an actual ability to do on-the-spot ‘thinking’ about new ideas. They fail miserably. ChatGPT is a smooth-talking swindler.

I suspect those who can’t see this either

(a) are software engineers amazed that a chatbot can write code, despite it having been trained on an unimaginably massive (morally ambiguously procured) dataset that probably already contains something close to the boilerplate you want anyway

(b) don’t have the sufficient level of technical knowledge to ask probing enough questions to betray the weaknesses. That is, anything you might ask is either so open-ended that almost anything coherent will look like a valid answer (this is most questions you could ask, outside of seriously technical fields) or has already been asked countless times before and is explicitly part of the training data.

Re: What can LLMs never do?

#316

Post by danenania »

xanderlewis wrote: post #315
adrianN wrote: post #236

Earlier quoted context omitted.

The problem is that we currently lack good definitions for crucial words such as "understanding" and we don't know how brains work, so that nobody can objectively tell whether a spreadsheet "understands" anything better than our brains. That makes these kinds of discussions quite unproductive.

I can’t define ‘understanding’ but I can certainly identify a lack of it when I see it. And LLM chatbots absolutely do not show signs of understanding. They do fine at reproducing and remixing things they’ve ‘seen’ millions of times before, but try asking them technical questions that involve logical deduction or an actual ability to do on-the-spot ‘thinking’ about new ideas. They fail miserably. ChatGPT is a smooth-…

Your understanding of how LLMs work isn’t at all accurate. There’s a valid debate to be had here, but it requires that both sides have a basic understanding of the subject matter.

Re: What can LLMs never do?

#317

Post by danenania »

iraqmtpizza wrote: post #284
danenania wrote: post #218

Earlier quoted context omitted.

As the comment I replied to very correctly said, we don’t know how the brain produces cognition. So you certainly cannot discard the hypothesis that it works through “parroting” a weighted average of training data just as LLMs are alleged to do. Considering that LLMs with a much smaller number of neurons than the brain are in many cases producing human-level output, there is some evidence, if circumstantial, that our…

LLMs don't have neurons. That's just marketing lol. "A neuron in a neural network typically evaluates a sequence of tokens in one go, considering them as a whole input." -- ChatGPT You could consider an RTX 4090 to be one neuron too.

It’s almost as if ‘neuron’ has a different meaning in computer science than biology.

Re: What can LLMs never do?

#318

Post by singron »

throwthrowuknow wrote: post #286
SomeCallMeTim wrote: post #136

Earlier quoted context omitted.

LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "und…

What is your definition of understanding? Please show me where the training data exists in the model to perform this lookup operation you’re supposing. If it’s that easy I’m sure you could reimplement it with a simple vector database. Your last two paragraphs are just dualism in disguise.

People are confusing the limited computational model of a transformer with the "Chinese room argument", which leads to unproductive simultaneous debates of computational theory and philosophy.

Re: What can LLMs never do?

#319

Post by iraqmtpizza »

danenania wrote: post #317
iraqmtpizza wrote: post #284

Earlier quoted context omitted.

LLMs don't have neurons. That's just marketing lol. "A neuron in a neural network typically evaluates a sequence of tokens in one go, considering them as a whole input." -- ChatGPT You could consider an RTX 4090 to be one neuron too.

It’s almost as if ‘neuron’ has a different meaning in computer science than biology.

LOL you just owned the guy who said "LLMs with a much smaller number of neurons than the brain are in many cases producing human-level output"

Re: What can LLMs never do?

#320

Post by nickpsecurity »

cs702 wrote: post #310
naasking wrote: post #299

> They have been trained on more information than a human being can hope to even see in a lifetime. Assuming a human can read 300 words a min and 8 hours of reading time a day, they would read over a 30,000 to 50,000 books in their lifetime. Most people would manage perhaps a meagre subset of that, at best 1% of it. That’s at best 1 GB of data. This just isn't true. Human training is multimodal to a degree far beyond…

I agree with you, but your comment strikes me as unfair nitpicking , because the OP is referring to information that has been encoded in words.

We learn the ideas from each mode of input. Then, one mode can elaborate on data learned from another mode. They build on each other.

From there, remember the text is usually a reflection of things in the real world. Understanding those things in non-textual ways both gives meaning to and deeper understanding of the text. Much of the text itself was even stored in other modes, like markup or PDF’s, whose structure tells us things about it.

That we learn multimodal from birth is therefore an important point to make.

It might also be a prerequisite for AGI. It could be one of the fundamental laws of information theory or something. Text might not be enough like how digital devices need analog to interface with the real world.

Post reply on HN