Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

171–180 of 385 posts

Re: What can LLMs never do?

#171

Post by pbhjpbhj »

tobiasSoftware wrote: post #155
richardw wrote: post #145

Earlier quoted context omitted.

LLM’s are a compressed and lossy form of our combined writing output, which it turns out is similarly structured enough to make new combinations of text seem reasonable, even enough to display simple reasoning. I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind. That doesn’t mean we won’t end up approximating one event…

"I find it useful to think “what can I expect from speaking with the dataset of combined writing of people”, rather than treating a basic LLM as a mind." I've been doing this as well, mentally I think of LLMs as the librarians of the internet.

They're bad librarians. They're not bad, they do a bad job of being librarians, which is a good thing! They can't quite tell you the exact quote, but they do recall the gist, they're not sure it was Gandhi who said that thing but they think he did, it might be in this post or perhaps one of these. They'll point you to the right section of the library to find what you're after, but make sure you verify it!

Re: What can LLMs never do?

#172

Post by pixl97 »

jncfhnb wrote: post #131
dragonwriter wrote: post #120

Earlier quoted context omitted.

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

A consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.

Re: What can LLMs never do?

#173

Post by mathstuf »

Things I've seen stump the ones I've played with so far (admittedly, not a lot): playing/generating "MadGab" puzzles and ASCII art rendering/interpretation. I've also asked ChatGPT3.5 to phonetically transliterate from English to other orthographies using the typical sounds of said orthography and it was…OK at it.

Re: What can LLMs never do?

#174

Post by jncfhnb »

pixl97 wrote: post #172
jncfhnb wrote: post #131

Earlier quoted context omitted.

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

A consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.

I would argue it needs to be at least somewhat continuous. Perhaps discrete on some granularity but if something is just a function waiting to be called it’s not an intelligent entity. The entity is the calling itself.

Re: What can LLMs never do?

#175

Post by pixl97 »

martindbp wrote: post #103
jillesvangurp wrote: post #87

Earlier quoted context omitted.

We have to be a bit more honest about the things we can actually do ourselves. Most people I know would flunk most of the benchmarks we use to evaluate LLMs. Not just a little bit but more like completely and utterly and embarrassingly so. It's not even close; or fair. People are surprisingly alright at a narrow set of problems. Particularly when it doesn't involve knowledge. Most people also suck at reasoning (unles…

It's clear that what's missing is flexibility and agency. For anything that can be put into text or a short conversation, and I'd have to chose between access to ChatGPT or a random human, I know what I'd chose.

Agency is one of those things we probably want to think about quite a bit. Especially with the the willingness for people to hook up it up to things that interact with the real world.

Re: What can LLMs never do?

#176

Post by wongarsu »

jncfhnb wrote: post #131
dragonwriter wrote: post #120

Earlier quoted context omitted.

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

You could imagine an LLM being called in a loop with a prompt like

You observe: {new input}

You remember: {from previous output}

React to this in the following format:

My inner thoughts: [what do you think about the current state]

I want to remember: [information that is important for your future actions]

Things I do: [Actions you want to take]

Things I say: [What I want to say to the user]

...

Not sure if that would qualify as an AGI as we currently define it. Given a sufficiently good LLM with good reasoning capabilities such a setup might be able to It would be able to do many of the things we currently expect AGIs to be able to do (given a sufficiently good LLM with good reasoning capabilities), including planning and learning new knowledge and new skills (by collecting and storing positive and negative examples in its "memory"). But its learning would be limited, and I'm sure as soon as it exists we would agree that it's not AGI

Re: What can LLMs never do?

#177

Post by rainsford »

dragonwriter wrote: post #120
cs702 wrote: post #14

Fantastic essay. Highly recommended! I agree with all key points: * There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don't fully understand why current LLMs are bad at these tasks. * Providing an LLM with examples and step-by-step…

> There are problems that are easy for human beings but hard for current LLMs (and maybe impossible for them; no one knows). Examples include playing Wordle and predicting cellular automata (including Turing-complete ones like Rule 110). We don’t fully understand why current LLMs are bad at these tasks. I thought we did know for things like playing Wordle, that its because they deal with words as sequence of tokens t…

Arriving at a generally accepted scientific definition of AGI might be difficult, but a more achievable goal might be to arrive at a scientific way to determine something is not AGI. And while I'm not an expert in the field, I would certainly think a strong contender for relevant criteria would be an inability to process information in a way other than the one a system was explicitly programmed to, even if the new way of processing information was very related to the pre-existing method. Most humans playing Wordle for the first time probably weren't used to thinking about words that way either, but they were able to adapt because they actually understand how letters and words work.

I'm sure one could train an LLM to be awesome at Wordle, but from an AGI perspective the fact that you'd have to do so proves it's not a path to AGI. The Wordle dominating LLM would presumably be perplexed by the next clever word game until trained on thinking about information that way, while a human doesn't need to absorb billions of examples to figure it out.

I was originally pretty bullish on LLMs, but now I'm equally convinced that while they probably have some interesting applications, they're a dead-end from a legitimate AGI perspective.

Re: What can LLMs never do?

#178

Post by haswell »

pixl97 wrote: post #172
jncfhnb wrote: post #131

Earlier quoted context omitted.

I think one should feel comfortable arguing that AGI must be stateful and experience continuous time at least. Such that a plain old LLM is definitively not ever going to be AGI; but an LLM called in a do while true for loop might.

A consistent stateful experience may be needed, but not sure about continuous time. I mean human consciousness doesn't do that.

Human consciousness does though, e.g. the flow state. F1 drivers are a good example.

We tend to not experience continuous time because we repeatedly get distracted by our thoughts, but entering the continuous stream of now is possible with practice and is one of the aims of many meditators.

Re: What can LLMs never do?

#180

Post by mkl »

bxparks wrote: post #47
Xenoamorphous wrote: post #21

Earlier quoted context omitted.

“Ten” is a word, “10” are digits. I’m not a native English speaker, how would you write it? FWIW the LLMs get it right many times, but fail other times.

I couldn't understand the original wording either, but after reading one of the sibling comments that explains it, it suddenly made sense. I think you left out a few words that most English writers would include. So instead of: > "ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text", something like the following might be better for most people: > "ensure that th…

Much better, but still missing "than" after "greater", which seems kind of critical.

"Using" is important as a number greater than ten can't be written as a digit, but can be written using digits ("with" would be just as good). Repeating "written" makes it clearer that there are two instructions.

Post reply on HN