Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

321–330 of 385 posts

Re: What can LLMs never do?

#321

Post by lostmsu »

zer00eyz wrote: post #210
red75prime wrote: post #192

Earlier quoted context omitted.

Ugh. Really? Those "simulated water isn't wet"(when applied to cognition) "arguments" were punched so many times it even hurts to look at them.

No simulated water isnt wet. But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.

This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.

Re: What can LLMs never do?

#322

Post by Xenoamorphous »

empath-nirvana wrote: post #162
Xenoamorphous wrote: post #18

There’s many things they can’t do. Even a simple rule like “ensure that numbers from one to ten are written as words and numbers greater ten as digits in the given text” fails for me for so many examples even if it works for many others; few shot, chain of thought, many versions of the prompt, it doesn’t matter. Sometimes LLMs will even change the number to something else, even with temp set to 0. And then there’s th…

The reason it can't do that is that, for example, "twenty" and "20" are nearly identical in the vector embedding space and it can't really distinguish them that well in most contexts. That's true for generally any task that relies on sort of "how the words look" vs "what the words mean". Any kind of meta request is going to be very difficult for an LLM, but a multi-modal GPT model should be able to handle it.

Thanks, I’ll try the multimodal one.

Re: What can LLMs never do?

#323

Post by Eisenstein »

nathan_compton wrote: post #263
dragonwriter wrote: post #120

Earlier quoted context omitted.

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

"Since there is no objective definition of AGI or test for it, there’s no basis for any meaningful speculation on what can or cannot achieve it; discussions about it are quasi-religious, not scientific." This is such a weird thing to say. Essentially _all_ scientific ideas are, at least to begin with, poorly defined. In fact, I'd argue that almost all scientific ideas remain poorly defined with the possible exception…

What is the rough definition, then?

Re: What can LLMs never do?

#324

Post by _wire_ »

thom wrote: post #70

I have no beef with the actual content or conclusions, but it’s a shame the article is framed the way it is, because I don’t think we can rigorously define the goalposts for what qualifies as a future LLM. It could just as easily have been titled “Exciting avenues of research for future LLMs!” but we’re all so jaded despite the frankly astonishing progress of recent years.

>jaded in spite of astonishing progress

We've all seen something that looks amazing, but few seem to know what we're looking at.

I am unsettled by what I see as a division of thought between extolling AI's amazing effects on one hand and mysterious regards for how it works and its limits on the other.

Noting Arthur Clarke's dictum that 'technology can be sufficiently advanced to be indistinguishable from magic,' AI enthusiasm looks like a Feynman cargo cult.

But technology arousing magical thinking with little discussion of principle of operation and limits is common enough.

This was very much the case with arrival of personal computing: there's something the device is intended to do but most people aren't sure what that is. The devices crashed enough and went out of date so fast you felt ok for not understanding them.

It was even worse with the mobile+web as so much change happened so fast that a generation has been dumbstruck: look at USA politics.

I was looking at old episodes of the Computer Chronicles from early 90s on YT and by that time the show had close to a thousand episodes, but they could barely explain the significance of Windows 3.0 and the Pentium. As to what to expect from this stuff, they didn't even try it was mindless rambling and upsell interspersed with stern warnings from the Software Publishers Association that mucking with the code is a Federal offense. The show's guests all had something to sell with a half life of 3–6 months. For the Pentium episode they show a PC lab with nerds in Dockers (khaki pants) studiously examining how many fans it might take to keep a lanman server from overheating and crashing. Many were amazed by it all.

Also available on YT are old ATT videos, including an introduction to UNIX with Kernighan & Richie. The presentation entirely focuses on the power of the shell. They seemed much more reserved and competent in retrospect, but in its time they looked like a pure priesthood.

Maybe the arrival AI stuff is not so different from the arrival of personal computing?

But AI is basically just one app, and I get the feeling that the scene is far more enigmatic to the point that even the people building the kit don't really know why it does what it does, and no one seems to have a clear idea of what correct functioning means.

Re: What can LLMs never do?

#325

Post by coffeebeqn »

dragonwriter wrote: post #135
ccppurcell wrote: post #128

Earlier quoted context omitted.

I would argue that the G in AGI means it can't require better prompting.

That would like saying that because humans’ output can be better or worse based on better or worse past experience (~prompting, in that it is the source of the equivalent of “in-context learning”), humans lack general intelligence.

This is more like the distinction of a Jr and Sr dev. One needs the tasks the be pre-chewed and defined “good prompts” while the latter can deal with very ambiguous problems

Re: What can LLMs never do?

#326

Post by philipwhiuk »

amayne wrote: post #207

Some of these can be done with the right prompting. Wordle was solved a year ago. The problem is that what works and what doesn’t can be counterintuitive or at least require you to think about how something is perceived at the token level. https://andrewmayne.com/2023/03/29/how-to-play-wordle-with-g...

What's the intelligent agent at that point though - the solver or the prompter?

The LLM isn't solving the actual problem, it's solving a subset problem.

Re: What can LLMs never do?

#327

Post by svachalek »

layer8 wrote: post #305
furyofantares wrote: post #300

Earlier quoted context omitted.

That was my original interpretation, and while all it sees are tokens, roughly none of its training data is metadata about tokenizing. It knows far less about the positions of tokens in words than it does about the positions of letters in words.

I’m not sure that training data about that would be required. Shouldn’t the model be able to recognize that `["re", "cogn", "ize"]` represents the same sequence of tokens as `recognize`, assuming those are tokens in the model? More generally, would you say that LLMs are generally unable to reason about sequences of items (not necessarily tokens) and compare them to some definition of “valid” sequences that would aris…

No. In the model, tokens are random numbers. But if you consider a sentence to be a sequence of words, you can say that LLMs are quite competent about reasoning about those sequences.

Re: What can LLMs never do?

#328

Post by coffeebeqn »

Too wrote: post #242

While the A:B problem technically was solved, look at the solutions, they are several hundreds lines of prompts, rephrasing the problem to the point that a human doesn't understand it any more. Even with a thorough review, nobody can guarantee if the prompts are going to work or not, most of them didn't, 90% pass was considered good enough. The idea of AI is to reduce work, not create more, otherwise what's the point…

I guess the problem is that if you need to teach it tricks for each novel problem still after training then that model can not be a general intelligence. It could still be useful though

Re: What can LLMs never do?

#329

Post by Sleepful »

weitendorf wrote: post #215
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. Wordle and cellular automata are very 2D, and LLMs are fundamentally 1D. You might think "but what about Chess!" -…

Since when can LLM play chess? It can't understand it at all. You would have to filter out all the invalid moves until it spits a valid one.

Re: What can LLMs never do?

#330

Post by baobabKoodaa »

svachalek wrote: post #327
layer8 wrote: post #305

Earlier quoted context omitted.

I’m not sure that training data about that would be required. Shouldn’t the model be able to recognize that `["re", "cogn", "ize"]` represents the same sequence of tokens as `recognize`, assuming those are tokens in the model? More generally, would you say that LLMs are generally unable to reason about sequences of items (not necessarily tokens) and compare them to some definition of “valid” sequences that would aris…

No. In the model, tokens are random numbers. But if you consider a sentence to be a sequence of words, you can say that LLMs are quite competent about reasoning about those sequences.

ChatGPT is able to spell the word "recognize" when asked.

So it is able to take a sequence of tokens ["recogn", "ize"] and transform it into a sequence of tokens [" R", " E", " C", " O", " G", " N", " I", " Z", " E"]

Post reply on HN