Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What can LLMs never do?

strangeloopcanon.com

Post reply on HN
Print view View original

91–100 of 385 posts

Re: What can LLMs never do?

#91

Post by mewpmewp2 »

Mordisquitos wrote: post #45
3abiton wrote: post #3

The article should be titled " What can LLM never do, yet". By definition, Large Language Models would keep growing larger and larger, to be trained on faster and more advanced hardware, and certain points like "completing complex chains of logical reasoning" tasks, would be just a time hurdle. Only time will tell.

I think that the article is correct. There are indeed things that L L Ms will never be able to do, at least not consistently, however much the hardware improves or on how much more material they are trained. How come? Note my emphasis on the 2nd 'L'. I'm not saying that there are things that AI models will never be able to do, I'm saying that there are things that Large Language Models will be unable to do. Training…

I don't think values, opinions or things like that are needed at all. These are just aspects we have in order to perform in and together with the society.

Also doubt is just uncertainty, and can be represented as a probability. Actually all values and everything can be presented as a numerical probability, which I personally prefer to do as well.

Re: What can LLMs never do?

#92

Post by shawntan »

jillesvangurp wrote: post #87
shawntan wrote: post #71

If we're trying to quantify what they can NEVER do, I think we'd have to resort to some theoretical results rather than a list empirical evidence of what they can't do now. The terminology you'd look for in the literature would be "expressibility". For a review of this topic, I'd suggest: https://nessie.ilab.sztaki.hu/~kornai/2023/Hopf/Resources/st... The authors of this review have themselves written several article…

We have to be a bit more honest about the things we can actually do ourselves. Most people I know would flunk most of the benchmarks we use to evaluate LLMs. Not just a little bit but more like completely and utterly and embarrassingly so. It's not even close; or fair. People are surprisingly alright at a narrow set of problems. Particularly when it doesn't involve knowledge. Most people also suck at reasoning (unles…

Not sure what you got out of the paper, but for me it was more spurring ideas about how to fix this in future architectures.

Don't think anyone worth their salt would look at this and think : oh well that's that then.

Re: What can LLMs never do?

#93

Post by unparagoned »

shawntan wrote: post #71

If we're trying to quantify what they can NEVER do, I think we'd have to resort to some theoretical results rather than a list empirical evidence of what they can't do now. The terminology you'd look for in the literature would be "expressibility". For a review of this topic, I'd suggest: https://nessie.ilab.sztaki.hu/~kornai/2023/Hopf/Resources/st... The authors of this review have themselves written several article…

Neural nets can approximate any function.

A large enough llm with memory is turning complete.

So theoretically I don’t think there is anything they can never do.

Re: What can LLMs never do?

#95

Post by shawntan »

unparagoned wrote: post #93
shawntan wrote: post #71

If we're trying to quantify what they can NEVER do, I think we'd have to resort to some theoretical results rather than a list empirical evidence of what they can't do now. The terminology you'd look for in the literature would be "expressibility". For a review of this topic, I'd suggest: https://nessie.ilab.sztaki.hu/~kornai/2023/Hopf/Resources/st... The authors of this review have themselves written several article…

Neural nets can approximate any function. A large enough llm with memory is turning complete. So theoretically I don’t think there is anything they can never do.

> Neural nets can approximate any function.

Common misunderstanding of the universal approximation theorem.

Consider this: can an mlp approximate a sine wave?

> A large enough llm with memory is turning complete.

With (a lot of) chain of thought it could be.

Read the paper, and its references.

Re: What can LLMs never do?

#96

Post by shawntan »

cs702 wrote: post #84
shawntan wrote: post #82

Earlier quoted context omitted.

Yes, but unfortunately that doesn't answer the question the title poses.

The OP is not trying to answer the question. Rather, the OP is asking the question and sharing some thoughts on the motivations for asking it.

I agree it's a good question to be asking.

There are good answers to be found if you look.

It feels like no proper looking was attempted.

Re: What can LLMs never do?

#97

Post by shawntan »

hansonw wrote: post #89
shawntan wrote: post #71

If we're trying to quantify what they can NEVER do, I think we'd have to resort to some theoretical results rather than a list empirical evidence of what they can't do now. The terminology you'd look for in the literature would be "expressibility". For a review of this topic, I'd suggest: https://nessie.ilab.sztaki.hu/~kornai/2023/Hopf/Resources/st... The authors of this review have themselves written several article…

This is also a good paper on the subject: What Algorithms can Transformers Learn? A Study in Length Generalization https://arxiv.org/abs/2310.16028

Yes this is a good empirical study on the types of tasks that's been shown to be impossible for transformers to generalise on.

With both empirical and theoretical support I find it's pretty clear this is an obvious limitation.

Re: What can LLMs never do?

#98

Post by exe34 »

vouaobrasil wrote: post #68
exe34 wrote: post #63

Earlier quoted context omitted.

You should ban knives too! We have an epidemic of knife crime here in the UK.

Knives are significantly more useful than LLMs, which are long past the point of diminishing returns.

Are you sure about that? The first knife was a sharp rock - what if they got banned immediately?

Re: What can LLMs never do?

#99

Post by imtringued »

I was onboard with the article up until the middle. After the conclusion where the author simply gives up I felt like it dragged on way too much.

His attempts at training on Conway's game of life are kind of pathetic. The problem isn't a lack of training data and neither is it's "distribution". The fallacy lies in the fact that the dataset itself doesn't contain reasoning in the first place. For example, GitHub CoPilot has fill in the middle capability, while ChatGPT by default does not.

Now here is the shocker about the fill in the middle capability. How does the LLM learn to do it? It does it in an incredibly primitive way. Instead of building a model that can edit its own context, it receives a marker in the context that tells it about the cursor position and then it is finetuned on the expected response.

This means that an LLM could be trained to insert its token at any position in the context or even replace existing tokens, but here is the problem: Once the model has modified its own context, it has exited the training dataset. How do you evaluate the intermediate steps, which can consist of genuinely novel thoughts which are required, but not present in the data? Adding two numbers requires intermediate states which the model may even know how to produce, but it can never be rewarded to utilize them, if they aren't in the training data, because for the LLM, the only goal is to conform to the dataset.

If you wanted to avoid this, you would need to define a metric which allows the model to be rewarded for a success even if that success took a detour. Currently, training is inherently built around the idea of zero shot responses.

Re: What can LLMs never do?

#100

Post by wbogusz »

rambambram wrote: post #37
stavros wrote: post #24

Earlier quoted context omitted.

I really hate this reductive, facile, "um akshually" take. If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. That's like saying "humans aren't supposed to reason, they're supposed to make sounds with their mouths".

> If the text that the text-generating tool generates contains reasoning, then the text generation tool can be said to be reasoning, can't it. I don't know... you're still describing a talking parrot here, if you'd ask me.

I’m not a fan of the talking parrot argument, especially when you’re pointing it at models of scale.

The only thing separating a talking parrot and humans is our accuracy in shaping our words to the context in which they’re spoken.

Sure it’s easy to liken a low resource model to a talking parrot, the output seems no better than selective repetition of training data. But is that really so different from a baby whose first words are mimics from the environment around them?

I would argue that as we learn language we implicitly develop the neural circuitry to continue to improve our lexical outputs, this circuitry being concepts like foresight, reasoning, emotion, logic, etc and that while we can take explicit action to teach these ideas, they naturally develop in isolation as well.

I don’t think language models, especially at scale, are much different. They would seem to similarly acquire implicit circuitry like we do as they are exposed to more data. As I see it, the main difference in what exactly that circuitry accomplishes and looks like in final output has more to do with the limited styles of data we can provide and the limitations of fine tuning we can apply on top.

Humans would seem to share a lot in common with talking parrots, we just have a lot more capable hardware to select what we repeat.

Post reply on HN