Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

351–360 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#351

Post by danielmarkbruce »

twiceaday wrote: post #255
hifromwork wrote: post #205

Earlier quoted context omitted.

>I bet you can reason 1+3x=22 pretty easily without any words whatsoever I've tried to do it, but I can't. I had to do something like "ok, so we subtract one from both sides and then it's easy, 3*7=21". Maybe I could do 2+8 but I still think the word ten "aloud".

I was able to do it with no words. I 'saw' the steps as if on a piece of paper. I saw 3x=22-1=21, then x=21/3=7. But I have a degree in applied math. Perhaps not internally vocalizing is just being extremely familiar. It also happened very quickly, perhaps there was no time to vocalize anyways.

A decent number of folks can jump straight to the answer on something so straightforward, no steps.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#352

Post by bloppe »

The point of this paper is not to claim that LLMs are dumb. It's applying rigor to the notion that they "reason" in a way that's fundamentally non human. They're more like search engines than abstract reasoning machines. There will always be problem structures that are too different from anything in the training set for them to solve

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#353

Post by danielmarkbruce »

PheonixPharts wrote: post #102
tsunamifury wrote: post #90

Earlier quoted context omitted.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful. We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this ling…

Beam search.

Sophisticated folks aren't doing simplistic/stupid decoding.

Gotta go beyond LLMs 101 to see what's actually happening. Even in training folks are building models which predict several tokens ahead.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#354

Post by brcmthrowaway »

bryan0 wrote: post #272

Like others I ran this through gpt-4o and it fails at reasoning because it does not realize Alice is a sister. In fact in one explanation it defines the siblings as: M brothers, N sisters, 1 Alice (!!) I asked gpt “Is Alice a sister?” It answered yes and then answered the problem correctly

> M brothers, N sisters, 1 Alice

Dude, this is shockingly bad if true

Q* algorithm can't come fast enough

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#355

Post by kalkin »

rep_lodsb wrote: post #279
kalkin wrote: post #133

> Paper finds a failure case for current LLMs and draws broad conclusions about the fundamental limits of LLM technology > Evaluates the largest LLMs and finds evidence that actually scale overcomes the problem: "Notable exceptions are Claude 3 Opus and GPT-4 that occasionally manage to provide correct responses backed up with correct reasoning as evident in structured step by step explanations those models deliver t…

The key word is "occasionally". For anything even close to AGI, it would be reasonable to expect a correct answer to such a simple question 100% of the time . Or at least not confidently wrong answers.

The question the paper claims to address isn't whether current LLMs are AGI, the question is whether LLMs can reason. And solving a problem that seems to require reasoning some of the time is still evidence of reasoning - especially if the question is what we can expect this technology to do in the future.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#356

Post by Terr_ »

viking123 wrote: post #274
yoyohello13 wrote: post #106

Earlier quoted context omitted.

Many people, especially on this site, really want LLMs to be everything the hype train says and more. Some have literally staked their future on it so they get defensive when people bring up that maybe LLMs aren’t a replacement for human cognition. The number of times I’ve heard “but did you try model X” or “humans hallucinate too” or “but LLMs don’t get sleep or get sick” is hilarious.

The comment that humans are sometimes wrong too is absolutely ridiculous and I always think about the self-driving car incident in china where the car mistook an advertisement of a car above the road as a real car and hit the breaks causing a chain crash. Even the shittiest driver would never do that.

I like to bring up a certain thought-experiment, where some hypothetical self-driving cars (A) are safe enough to halve the number of the accidents/injuries/deaths compared to human drivers, but (B) almost all of the bad events are from their AI unpredictably deciding to to chase-and-murder pedestrians.

... Would we consider that a good improvement to roll out? Is the only factor short-term utilitarianism?

Sometimes our ability to predict and characterize errors is more important than the total error rate.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#358

Post by fnordpiglet »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

Question is how often do humans get the question wrong unprompted ? I expect given the median IQ of 100 and assessing people of all types including those untrained on logic and generally uneducated humans might not be nearly as close to 1.0 as the authors seem to imply.

But it’s also fairly obvious LLMs don’t reason at all so it’s not shocking that LLMs don’t reason at all. What’s remarkable is that they’re able to perform as well at reasoning tasks as they do given that factual truth they have no reasoning facility. Coupled with their natural language ability and emergent abductive reasoning ability that’s generally hard in computing models it’s hard to draw much conclusion here other than “wow gpt-4o is amazing.”

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#359

Post by chefandy »

floren wrote: post #210
chefandy wrote: post #150

Earlier quoted context omitted.

While LLMs have incredible potential, and are even downright useful in their current format, they have the rather nasty tendency to confidently present bullshit that passes the smell test for most people. When it's wrong, it's not just wrong , by design, it's wrong but sounds plausible. Considering most people with subject matter expertise aren't going to consult such a bot for their own areas of expertise, that mean…

> As soon as I asked "are you giving me instructions for Gemini Advanced or Bard?" it was like "OH LOL WOOPS!! YOU GOT ME BRO! XD I CAN'T DO ANY OF THAT! LOL." Which, honestly, is great. Being able to evaluate its answers to realize it's wrong is really neat. Did it evaluate its answers, or did your expression of doubt cause the eager-to-please language model to switch from "generate (wrong) instructions because that…

I suppose whether or not these things evaluate anything at all depends on your perspective, but as soon as I asked if it was giving me the correct information, it did actually give me useful information after that without further prompting.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#360

Post by aeternum »

danielmarkbruce wrote: post #349
IggleSniggle wrote: post #152

Earlier quoted context omitted.

Maybe, but symbolic thought can get pretty far away from what we generally call "language." I bet you can reason 1+3x=22 pretty easily without any words whatsoever, or the sound of one ascending octave after another, or the approximate G-force induced on your body if you take the next turn without applying the brakes. All of these forms of reasoning are true and useful calculations: when we talk about "intuition" wha…

Most people can't do 1 + 3x = 22 without any words or symbols. People who can don't realize that most people can't. I'd argue one isn't using logic when they do that, it's just very good pattern matching.

It's also possible to do mentally by visualizing it rather than internal monologue. You can imagine the 1 on the left arcing over to the right, cancelling the 22 to 21, then the 3 moving under the 21 and the 21 descending through the 3 to become 7.
Post reply on HN