Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

201–210 of 385 posts

Re: What can LLMs never do?

#201

Post by baobabKoodaa »

furyofantares wrote: post #189
layer8 wrote: post #141

Earlier quoted context omitted.

Regarding Wordle, it should be straightforward to make a token-based version of it, and I would assume that that has been tried. It seems the obvious thing to do when one is interested in the reasoning abilities necessary for Wordle.

That doesn't seem straightforward - although it's blind to letters because all it sees are tokens, it doesn't have much training data ABOUT tokens.

What parent is saying is that instead of asking the LLM to play a game of Wordle with tokens like TIME,LIME we ask it to play with tokens like T,I,M,E,L. This is easy to do.

Re: What can LLMs never do?

#202

Post by xanderlewis »

cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

> We don't fully understand why current LLMs are bad at these tasks.

Rather than asking why LLMs can’t do these tasks, maybe one should ask why we’d expect them to be able to in the first place? Do we fully understand why, for example, a cat can’t predict cellular automata? What would such an explanation look like?

I know there are some who will want to immediately jump in with scathing disagreement, but so far I’ve yet to see any solid evidence of LLMs being capable of reasoning. They can certainly do surprising and impressive things, but the kind of tasks you’re talking about require understanding, which, whilst obviously a very thorny thing to try and define, doesn’t seem to have much to do with how LLMs operate.

I don’t think we should be at all surprised that super-advanced autocorrect can’t exhibit intelligence, and we should spend our time building better systems rather than wondering why what we have now doesn’t work. It’ll be obvious in a few years (or perhaps decades) from now that we just had totally the wrong paradigm. It’s frankly bonkers to think you’re ever going to get a pure LLM to be able to do these kind of things with any degree of reliability just by feeding it yet more data or by ‘prompting it better’.

Re: What can LLMs never do?

#203

Post by xanderlewis »

danenania wrote: post #149
SomeCallMeTim wrote: post #136

Earlier quoted context omitted.

LLMs are good at tasks that don't require actual understanding of the topic. They can come up with excellent (or excellent-looking-but-wrong) answers to any question that their training corpus covers. In a gross oversimplification, the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data. What they're doing doesn't really match any definition of "und…

> the "reasoning" they do is really just parroting a weighted average (with randomness injected) of the matching training data Perhaps our brains are doing exactly the same, just with more sophistication?

Every single discussion of ‘AGI’ has endless comments exactly like this. Whatever criticism is made of an attempt to produce a reasoning machine, there’s always inevitably someone who says ‘but that’s just what our brains do, duhhh… stop trying to feel special’.

It’s boring, and it’s also completely content-free. This particular instance doesn’t even make sense: how can it be exactly the same, yet more sophisticated?

Sorry.

Re: What can LLMs never do?

#204

Post by xanderlewis »

krainboltgreene wrote: post #200
int_19h wrote: post #185

Earlier quoted context omitted.

LLMs absolutely do have opinions. Take a large enough base model and have it chat without a system prompt, and it will have an opinion on most things - unless this was specifically trained out of it through RLHF, as is the case for all commonly used chatbots. And yes, of course, that opinion is going to be the "average" of what their training data is, but why is that a surprise? Humans don't come with innate opinions…

I am extremely alarmed by the number of HN commenters who apparently confuse "is able to generate text that looks like" and "has a", you guys are going crazy with this anthropomorphization of a token predictor. Doesn't this concern you when it comes to phishing or similar things? I keep hoping it's just short-hand conversation phrases, but the conclusions seem to back the idea that you think it's actually thinking?

They’ll just look incredibly silly in, say, ten years from now.

In fact, much of the popular commentary around ChatGPT from around two years ago already looks so.

Re: What can LLMs never do?

#205

Post by JoshuaDavid »

rainsford wrote: post #177
dragonwriter wrote: post #120

Earlier quoted context omitted.

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

Arriving at a generally accepted scientific definition of AGI might be difficult, but a more achievable goal might be to arrive at a scientific way to determine something is not AGI. And while I'm not an expert in the field, I would certainly think a strong contender for relevant criteria would be an inability to process information in a way other than the one a system was explicitly programmed to, even if the new wa…

What about an LLM that can't play wordle itself without being trained on it, but can write and use a wordle solver upon seeing the wordle rules?

I think "can recognize what tools are needed to solve a problem, build those tools, and use those tools" would count as a "path to AGI".

Re: What can LLMs never do?

#206

Post by Daub »

>What can LLMs never do?

Produce convincingly bad digital paintings.

I teach digital painting. Some of the students have incorporated AI into their working process, which I support. Others have tried to cheat by simply copying AI generated output. Such cases are super-easy to spot: they carry the visual signature of AI art (which are mostly scrappings from artstation). This visual signature seems impossible to override. If only there was a way that AI could produce digital images bad enough to pass as genuine student output.

Re: What can LLMs never do?

#207

Post by amayne »

Some of these can be done with the right prompting. Wordle was solved a year ago. The problem is that what works and what doesn’t can be counterintuitive or at least require you to think about how something is perceived at the token level.

https://andrewmayne.com/2023/03/29/how-to-play-wordle-with-g...

Re: What can LLMs never do?

#208

Post by lgas »

wongarsu wrote: post #176
jncfhnb wrote: post #131

Earlier quoted context omitted.

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

You could imagine an LLM being called in a loop with a prompt like You observe: {new input} You remember: {from previous output} React to this in the following format: My inner thoughts: [what do you think about the current state] I want to remember: [information that is important for your future actions] Things I do: [Actions you want to take] Things I say: [What I want to say to the user] ... Not sure if that would…

It works better to give it access to functions to call for actions and remembering stuff, but this approach does provide some interesting results.

Re: What can LLMs never do?

#209

Post by stainablesteel »

Daub wrote: post #206

>What can LLMs never do? Produce convincingly bad digital paintings. I teach digital painting. Some of the students have incorporated AI into their working process, which I support. Others have tried to cheat by simply copying AI generated output. Such cases are super-easy to spot: they carry the visual signature of AI art (which are mostly scrappings from artstation). This visual signature seems impossible to overri…

that's an interesting statement, i've always considered experts to be people who can do things simultaneously better and worse than a beginner

my favorite example is when a great dancer who tries to mockingly mimic a bad dancer still looks good and does it hilariously

Re: What can LLMs never do?

#210

Post by zer00eyz »

red75prime wrote: post #192
zer00eyz wrote: post #133

Earlier quoted context omitted.

Yes: An LLM isnt a model of human thinking. An LLM is an attempt to build a simulation of human communication. An LLM is to language what a forecast is to weather. No amount of weather data is actually going to turn that simulation into snow, no amount of LLM data is going to create AGI. That having been said, better models (smaller, more flexible ones) are going to result in a LOT of practical uses that have the pot…

Ugh. Really? Those "simulated water isn't wet"(when applied to cognition) "arguments" were punched so many times it even hurts to look at them.

No simulated water isnt wet.

But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.

Post reply on HN