Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Knockout Blow for LLMs?

garymarcus.substack.com

Post reply on HN
Print view View original

41–49 of 49 posts

Re: A Knockout Blow for LLMs?

#41

Post by ramblerman »

IshKebab wrote: post #14

Argh please stop. Everyone knows LLMs aren't AGI currently and they have annoying limitations like hallucinations. Even the "giving up" thing was known before Apple's paper. You aren't winning anything by saying "aha! I told you they are useless!" because they demonstrably aren't . Yes everybody is hoping that someone will come up with a better algorithm that solves these problems but until they do it's a little like…

You don’t need to feel offended on behalf of LLMs, they are tremendously useful. But not above criticism.

There is a belief being peddled that AGI is right around the corner and we can get there by just scaling up LLMs.

Papers like this are a good takedown of that thinking

Re: A Knockout Blow for LLMs?

#42

Post by lostmsu »

Den_VR wrote: post #17
1dom wrote: post #5

"They're super expensive pattern matchers that break as soon as we step outside their training distribution" - I find it really weird that things like these are seen as some groundbreaking endgame discovery about LLMs LLMs have a real issues with polarisation. It's probably smart people saying all this stuff about knockout blows, and LLM uselessness, but I find them really useful. Is there some emperor's new clothes…

Marcus’s writing is from a scientific perspective, it’s not general artificial intelligence and probably not a meaningful path it GAI. But the ivory tower misses the point of how LLM improved the ability of regular people to interact with information and technology. While it might not be the grail they were seeking, it’s still a useful thing what will improve life and in turn be improved.

I don't see the relevance of this paper to AGI assuming one considers humans GI. Humans exhibit the same behavior where for each there's a complexity limit beyond which they are unable to solve any tasks in reasonable amount of time. For very complex tasks even training becomes unfeasible.

Re: A Knockout Blow for LLMs?

#43

Post by jhbadger »

kubb wrote: post #3

It would be awesome to develop some theory around what kind of problems LLMs can and cannot solve. That should deter some leads pushing for solving the unsolvable with the technology. That being said, this isn’t a knockout blow by any stretch. The strength of LLMs lies in the people who are excited about them. And there’s a perfect reinforcing mechanism for the excitement - the chatbots that use the models. Admit for…

>It would be awesome to develop some theory around what kind of problems LLMs can and cannot solve. That should deter some leads pushing for solving the unsolvable with the technology.

That could have unfortunate consequences. Most people stopped looking at neural nets for years because they thought that Minsky's and Papert's 1969 proof that perceptrons (linear neural nets) couldn't solve basic problems incorrectly applied to neural nets in general. So the field basically abandoned neural nets for a couple of decades which were more or less wasted on "symbolic" approaches to AI that accomplished little.

Re: A Knockout Blow for LLMs?

#44

Post by tim333 »

1dom wrote: post #5

"They're super expensive pattern matchers that break as soon as we step outside their training distribution" - I find it really weird that things like these are seen as some groundbreaking endgame discovery about LLMs LLMs have a real issues with polarisation. It's probably smart people saying all this stuff about knockout blows, and LLM uselessness, but I find them really useful. Is there some emperor's new clothes…

Marcus’s schtick is going on about things like "knockout blows, and LLM uselessness". He's kind of the go to AI/LLM naysayer.

Re: A Knockout Blow for LLMs?

#45

Post by camgunz »

1dom wrote: post #5

"They're super expensive pattern matchers that break as soon as we step outside their training distribution" - I find it really weird that things like these are seen as some groundbreaking endgame discovery about LLMs LLMs have a real issues with polarisation. It's probably smart people saying all this stuff about knockout blows, and LLM uselessness, but I find them really useful. Is there some emperor's new clothes…

Finding it useful is different than "I can replace my whole customer service department with an LLM", which the hype is convincing people is possible. You're not a dumbass; I hate LLMs and even I admit they're pretty good at coding.

Re: A Knockout Blow for LLMs?

#46

Post by somethingsome »

no_op wrote: post #35

The material finding of this paper is that reasoning models are better than non-reasoning models at solving puzzles of intermediate complexity (where that's defined, essentially, by how many steps are required), but that performance collapses past a certain threshold. This threshold differs for different puzzle types. It occurs even if a model is explicitly supplied with an algorithm it can use to solve the puzzle, a…

While I agree overall, LLMs are pattern matching in a complicated way.

You transform your training data in a very strange and high dimensional space. Then when you write an input, you calculate the distance between that input and the closest point in that space.

So, in some sense.. You pattern match your input with the training data. Of course, in a very non intuitive way for humans.

Now, it doesn't necessarily imply things as 'models cannot solve new problems not seen before' we don't know if our problem could get matched to something completely unrelated for us, but in that space it makes sense.

So with your experiments, if the model is able to solve a new puzzle never seen before, you'll never know why, but it doesn't imply either that the new puzzle was not matched in some sense to some previous data in the dataset.

Re: A Knockout Blow for LLMs?

#47

Post by stevenhuang »

feketegy wrote: post #2

In other news, water is wet. I don't think anybody who uses LLMs professionally day-to-day thinks that it can reason like human beings... If some people thought this, they fundamentally do not understand how LLMs work under the hood.

The knowledge of how LLMs work under the hood says nothing about how humans reason.

They are completely independent. For all we know human reasoning may absolutely share similar underlying principles as LLMs. Or not. We just don't know.

Re: A Knockout Blow for LLMs?

#49

Post by ankit219 »

The paper is sloppy. The original point may have credence[1], but the way they went about showing it is borderline irresponsible. The first aspect is how they conflated the number of steps involved with difficulty level. Not even considering the solution space. Then, the solutions are long, models are trained to keep the answers concise, and they are measuring consistency across tries. (Eg: Tower of hanoi for 13 steps needs 80k tokens to just blurt out the answer. The model already knows there is literally one way to solve it - ergo search space is not that big - but the paper shows that it is not reasoning. (ofc it isnt, since the sonnet64k would run out of tokens even without reasoning). Then, you have the scenario when even a 0.999 accurate llm would mess up one token and goes wrong on one run. They cited that as an example of how LLMs get it wrong and conclude its memorization and pattern matching and not reasoning. Real world data and usage does not correspond to that.

[1]: Anthropic found that reasoning is not 100% accurate. Thats the premise of the paper, just the headline is super clickbaity.

Post reply on HN