Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

341–350 of 385 posts

Re: What can LLMs never do?

#341

Post by naasking »

jncfhnb wrote: post #303
naasking wrote: post #283

Earlier quoted context omitted.

Which is what I described: some successful prompt-replies are fed back into subsequent training runs.

No… that implies the model never has active state and is being replaced with a different, stateless model. This is similar to the difference between Actor.happy = True And Actor = happier(Actor)

Both of your examples are stateful systems from the outside, given a suitable choice of timeframe, the latter one is just how purely functional systems represent state. Theoretically they can simulate each other, and the endpoint you use to access Actor will still reference the latest Actor. The only reason you're calling them different is because you insist on using a specific timeframe to exclude considering one as stateful, and I'm pointing out that that isn't strictly necessary.

Re: What can LLMs never do?

#342

Post by jncfhnb »

naasking wrote: post #341
jncfhnb wrote: post #303

Earlier quoted context omitted.

No… that implies the model never has active state and is being replaced with a different, stateless model. This is similar to the difference between Actor.happy = True And Actor = happier(Actor)

Both of your examples are stateful systems from the outside, given a suitable choice of timeframe, the latter one is just how purely functional systems represent state. Theoretically they can simulate each other, and the endpoint you use to access Actor will still reference the latest Actor. The only reason you're calling them different is because you insist on using a specific timeframe to exclude considering one as…

True but saying “subsequent training” implies very long periods between Updates.

We do not train LLMs to update them to the state of a conversation.

Re: What can LLMs never do?

#343

Post by constantcrying »

Again and again this article claims that surprisingly a LLM fails at a certain problem, when it appears to be easy. Each time it seems pretty obvious why that is the case though.

LLMs rely on the statistical dependencies between words or parts of words. That means any question you ask, which is hard to determine from that statistical dependency is extremely hard for an AI. E.g. ChatGPT fails at determining the length of words made up of random characters. It will fail at even performing the simplest of rules because encoding the rules in the statistical dependencies is extremely hard.

Re: What can LLMs never do?

#344

Post by zer00eyz »

lostmsu wrote: post #321
zer00eyz wrote: post #210

Earlier quoted context omitted.

No simulated water isnt wet. But an LLM isn't even trying to simulate cognition. It's a model that is predicting language. It has all the problems of a predictive model... the "hallucination" problem is just the tyranny of Lorenz.

This is plain wrong due to mixing of concepts. Language is technically something from Chomsky hierarchy. Predicting language is being able to tell if input is valid or invalid. LLMs do that, but they also build a statistical model across all valid inputs, and that is not just the language.

>> Predicting language is being able to tell if input is valid or invalid.

If this were the case then the hallucination problem would be solvable.

That hallucination problem is not only going to be hard to detect in any meaningful way but it's going to be harder to eliminate. The very nature of LLM (mixing in noise aka temperature) means that they always risk going off the rails. This is the same thing Lorenz discovered in modeling weather...

Re: What can LLMs never do?

#345

Post by _akhe »

kromem wrote: post #140

LLMs can't is such an anti-pattern at this point I'm surprised that anyone still dares to stake it. The piece even has an example of a $10k bet around a can't being proven false in under a day, but somehow doesn't think maybe their own can't examples are on similarly thin ice? In particular, the line about "what models can't do tells us what they don't know" is infuriating. No, that's not the case at all. At least in…

An LLM will probably be able to do most of what human minds can do like reason, predict, hypothesize, research, and even get hooked up to other systems to: Visualize, smell, taste, balance, and even direct the movement of limbs, but an LLM can't and won't ever be able to: Feel pain, bliss, anger, sadness, can't feel positive/negative, can't eat/drink, be hungry, feel fatigued, get excited, enjoy things, dislike things, contemplate, meditate, feel warm or cold (though it can detect it), can't feel dizzy (though it can know when it's off balance) - any action where having an experience is a necessary part of what it's doing and the output of it, an LLM is not sufficient to deliver on and never will be.

To compare to a brain, the LLM is like the prefrontal cortex or language and decision network in the outermost layer, but we would still need the amygdala in that metaphor - emotional drives, urges, episodic first-person memories, and experiential components that accompany the language and complete it with personhood.

For raw sensations and tactiles we might need that innermost brain stem - which is probably more chemistry than computation - for the "lights to be on". For example, some jobs will require not just language intelligence, and not just personhood, but for the light behind the images and feelings in the sensations, so that it feels (and would be) alive.

Re: What can LLMs never do?

#346

Post by weebull »

scoot wrote: post #291
Al-Khwarizmi wrote: post #264

Earlier quoted context omitted.

An LLM doesn't even see individual letters at all, because they get encoded into tokens before they are passed as input to the model. It doesn't make much sense to require reasoning with things that aren't even in the input as a requisite for intelligence. That would be like an alien race that could see in an extra dimension, or see the non-visible light spectrum, presenting us with problems that we cannot even see a…

And yet ChatGPT 3.5 can tell me the nth letter of an arbitrary word…

...because it knows that the next token in the sequence "the 5th letter in the word _illusion_ is" happens to be "s". Not because it decomposed the word into letters.

Re: What can LLMs never do?

#347

Post by xanderlewis »

danenania wrote: post #332
xanderlewis wrote: post #331

Earlier quoted context omitted.

How is it not accurate? I haven’t said anything about the internal workings of an LLM — just what it able to produce (which is based on observation). I have more than a basic understanding of the subject matter (neural networks; specifically transformers, etc.). It’s actually not a hugely technical field. By the way, it appears that you are in category (a).

You don’t know what they’re able to produce because you clearly don’t know how they actually work. So your “observations” are not worth much.

Yes I do, right down to the technical details. What makes you think I don’t? Is it because I used the word ‘remixing’?

Re: What can LLMs never do?

#348

Post by fragmede »

kromem wrote: post #309
fragmede wrote: post #293

Earlier quoted context omitted.

what's your puzzle? mines the river crossing puzzle

Mine is also the river crossing puzzle. What's your variation?

the boat is big enough to hold all the items

I asked it

> lion, goat, wolf riddle, but the boat is big enough to carry all of them

and it said it could do it in one step.

https://chat.openai.com/share/7b7a5462-7649-403d-a4f0-01c387...

ChatGPT-4 today (April 28th, 2024) still fails at it sometimes:

https://chat.openai.com/share/1bec923d-d727-42fe-ba9b-9f92b9...

This is ChatGPT-4 getting it wrong, months ago: https://chat.openai.com/share/caa37ad6-b7a8-451d-8f39-8a2c04...

This is ChatGPT-4 getting it right, today (April 28th, 2024): https://chat.openai.com/share/d2d9e63e-819e-4681-9f9f-8f77ea...

Re: What can LLMs never do?

#349

Post by furyofantares »

int_19h wrote: post #187
furyofantares wrote: post #170

Earlier quoted context omitted.

It's actually that it has no idea how words are spelled because they're blind to letters. This is because they operate instead on tokens instead. They do seem to know the first letter of each word pretty well (they love to alliterate and can also produce a list of things starting with A then B then C etc) but they are all very terrible at spelling beyond that. I presume they could be trained to spell even while retai…

With tasks like these it helps if you break your words up with some kind of separator so that it all tokenizes to one character per token. They do know what individual letters are conceptually.

It helps a little. ChatGPT4 still fails the following prompt fairly often (maybe 40% of the time):

which of these 5-letter sequences a has P in position 4?

P U P I L

S T A P H

C R E E P

P O O P Y

L I P I D

It usually gets it if it reiterates each sequence before deciding (but not always) and it almost always fails when it just answers right away.

And this doesn't actually help at all with the actual related Wordle problem - "think of a 5-letter word with P in the fourth position".

You could try filling up its context with a list of a thousand of the most common 5 letter words all spelled out (and I've done so, even with additional hints like the positions of the letters in parens). But it really didn't help (as might be expected since it can't even reliably do it with the above list of 5 words.)

Re: What can LLMs never do?

#350

Post by marquisdepolis »

wongarsu wrote: post #176
jncfhnb wrote: post #131

Earlier quoted context omitted.

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

You could imagine an LLM being called in a loop with a prompt like You observe: {new input} You remember: {from previous output} React to this in the following format: My inner thoughts: [what do you think about the current state] I want to remember: [information that is important for your future actions] Things I do: [Actions you want to take] Things I say: [What I want to say to the user] ... Not sure if that would…

(Author here). I tried creating something similar in order to solve wordle etc, and the interesting part is that it is insufficient still. That's part of the mystery.
Post reply on HN