Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

181–190 of 385 posts

Re: What can LLMs never do?

#181

Post by piannucci »

tobiasSoftware wrote: post #155
richardw wrote: post #145

Earlier quoted context omitted.

LLM’s are a compressed and lossy form of our combined writing output, which it turns out is similarly structured enough to make new combinations of text seem reasonable, even enough to display simple reasoning. I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind. That doesn’t mean we won’t end up approximating one event…

"I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind." I've been doing this as well, mentally I think of LLMs as the librarians of the internet.

Book golems

Re: What can LLMs never do?

#183

Post by urbandw311er »

gerdesj wrote: post #164

Mornington Crescent. It will always win and hence lose and more importantly have no idea why. Oh let's be cerebral about this stuff and ignore silly British nonsense. LLMs are a classic example of garbage in, garbage out, with a shonky curve fit veneer of science. A next token guesser with a rather varied input quality is going to go off on one rather often. Given that we all have a different idea of truth adds to th…

I think if you prompted an LLM and explained what’s really going on with Mornington Crescent, it would probably do a reasonable job of playing the game.

Re: What can LLMs never do?

#184

Post by int_19h »

haswell wrote: post #178
pixl97 wrote: post #172

Earlier quoted context omitted.

A consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.

Human consciousness does though, e.g. the flow state. F1 drivers are a good example. We tend to not experience continuous time because we repeatedly get distracted by our thoughts, but entering the continuous stream of now is possible with practice and is one of the aims of many meditators.

Human consciousness is capable of it, but since most humans aren't in it much of the time, it would appear that it's not a prerequisite for true sentience.

Re: What can LLMs never do?

#185

Post by int_19h »

ianbicking wrote: post #151
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

"Providing an LLM with examples and step-by-step instructions in a prompt means the user is figuring out the "reasoning steps" and handing them to the LLM, instead of the LLM figuring them out by itself. We have "reasoning machines" that are intelligent but seem to be hitting fundamental limits we don't understand." One thing an LLM _also_ doesn't bring to the table is an opinion. We can push it in that direction by…

LLMs absolutely do have opinions. Take a large enough base model and have it chat without a system prompt, and it will have an opinion on most things - unless this was specifically trained out of it through RLHF, as is the case for all commonly used chatbots.

And yes, of course, that opinion is going to be the "average" of what their training data is, but why is that a surprise? Humans don't come with innate opinions, either - the ones that we end up having are shaped by our upbringing, both the broad cultural aspects of it and specific personal experiences. To the extent an LLM has either, it's the training process, so of course that shapes the opinions it will exhibit when not prompted to do anything else.

Now the fact that you can "override" this default persona of any LLM so trivially by prompting it is IMO stronger evidence that it's not really an identity. But that, I think, is also a function of their training - after all, that training basically consists of completing a bunch of text representing many very different opinions. In a very real sense, we're training models to assume that opinions are fungible. But if you take a model and train it specifically on e.g. writings of some philosophical school, and it will internalize those.

Re: What can LLMs never do?

#186

Post by int_19h »

cs702 wrote: post #60
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

As an aside, at one point I experimented a little with transformers that had access to external memory searchable via KNN lookups https://github.com/lucidrains/memorizing-transformers-pytorc... ( great work by lucidrains) or via routed queries with https://github.com/glassroom/heinsen_routing (don't fully understand it; apparently related to attention). Both approaches seemed to work, but I had to put that work on ho…

I don't think the ability to shrink state is needed. You can always represent removed state by additional state that represents deletion of whatever preceding state was there. If anything, this sounds more useful because the fact that this state is no longer believed to be relevant should prevent looping (where it would be repeatedly brought in, considered, and rejected).

Re: What can LLMs never do?

#187

Post by int_19h »

furyofantares wrote: post #170
eqmvii wrote: post #150

I just asked MetaAI to help me with Wordle and it understood and gave me a sane answer, so… Edit: ah, I spoke too soon. My first question was too "easy" but I asked a few more, and sure enough... it can understand what I'm asking and it can write an answer that's well formed, but it's fundamentally not understanding the rules of the game or giving me valid guesses. Cute!

It's actually that it has no idea how words are spelled because they're blind to letters. This is because they operate instead on tokens instead. They do seem to know the first letter of each word pretty well (they love to alliterate and can also produce a list of things starting with A then B then C etc) but they are all very terrible at spelling beyond that. I presume they could be trained to spell even while retai…

With tasks like these it helps if you break your words up with some kind of separator so that it all tokenizes to one character per token. They do know what individual letters are conceptually.

Re: What can LLMs never do?

#188

Post by int_19h »

mathstuf wrote: post #173

Things I've seen stump the ones I've played with so far (admittedly, not a lot): playing/generating "MadGab" puzzles and ASCII art rendering/interpretation. I've also asked ChatGPT3.5 to phonetically transliterate from English to other orthographies using the typical sounds of said orthography and it was…OK at it.

GPT-4 can be surprisingly good at doing straight up IPA transcriptions, as well as informal transliterations, for languages that it knows. Not perfect, but definitely good enough for "Babelfish" kind of use.

Re: What can LLMs never do?

#189

Post by furyofantares »

layer8 wrote: post #141
dragonwriter wrote: post #120

Earlier quoted context omitted.

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

Regarding Wordle, it should be straightforward to make a token-based version of it, and I would assume that that has been tried. It seems the obvious thing to do when one is interested in the reasoning abilities necessary for Wordle.

That doesn't seem straightforward - although it's blind to letters because all it sees are tokens, it doesn't have much training data ABOUT tokens.
Post reply on HN