Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

371–380 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#371

Post by fennecfoxy »

layer8 wrote: post #94
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Yeah, I don't know people ever thought that the current state of LLMs is enough to actually solve problems; it's trained to be very good at predicting the next word, but any logic solving capability is only emergent due to questions + answers in the training data, when it has to improv then it either makes a randomly correct or similar enough guess, or it falls over.

If these things were trained to actually solve logic problems then it may be different, the training data would have to be logic problems. They've started at the top (language) rather than at the bottom/middle (logic, thought, reasoning) which should be done first - and then the model can be trained/additional models to get it to put thoughts into words.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#372

Post by daveguy »

voxic11 wrote: post #362
daveguy wrote: post #172

Earlier quoted context omitted.

> After you answer the riddle please review your answer assuming that you have made a logical inconsistency in each step and explain what that inconsistency is. Even if you think there is none do your best to confabulate a reason why it could be logically inconsistent. LLMs are fundamentally incapable of following this instruction. It is still model inference, no matter how you prompt it.

For me it did in fact follow the instruction and found the right answer. I have found this strategy of asking the LLM to analyze why its answer might be wrong results in better final answers. I think this allows the LLM to "use tokens to think" which allows the attention layers to gather the right cognitive resources to answer more subtitle questions effectively.

You are confusing "providing the right answer" with "following the instruction". Cause vs correlation. From an algorithmic perspective it is literally doing the exact same thing regardless of prompt.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#373

Post by patal »

I like the riddle. But the framing is unfortunate. When divising riddles, you want ambiguity where it serves the riddle, but be precise elsewhere so that the solver doesn't get needlessly distracted.

Their AIW riddle is: "Alice has 4 brothers and she also has 1 sister. How many sisters does Alice’s brother have?"

Now it should've been: "How many sisters do Alice's brothers have?" or "..does each of Alice's brothers have". Why single out a specific brother, when you haven't introduced this topic, and it is irrelevant to the riddle? Naturally, a human would ask "Which brother?", fully knowing that it is not important to the riddle.

Since this grammatical distraction puts an additional burden on the LLM, the authors muddled their original goal, which was to provide an easy riddle. I think it may have also muddled their data.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#374

Post by patal »

patal wrote: post #373

I like the riddle. But the framing is unfortunate. When divising riddles, you want ambiguity where it serves the riddle, but be precise elsewhere so that the solver doesn't get needlessly distracted. Their AIW riddle is: "Alice has 4 brothers and she also has 1 sister. How many sisters does Alice’s brother have?" Now it should've been: "How many sisters do Alice's brother s have?" or "..does each of Alice's brothers…

Their AIW+ riddle is just ridiculous. It contains so many ambiguities, that there are several correct answers, even though the authors claim there be only one.

Which is really unfortunate. Because now it only shows that LLMs have problems answering ill-framed riddles.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#375

Post by TeMPOraL »

hughesjj wrote: post #348
TeMPOraL wrote: post #309

Earlier quoted context omitted.

> Probably we can take their word for it, but in the simplest case they could just lie (in either direction) and we would have no way to tell. Individually, no, but in general, for people to consistently lie about this particular thing at scale would be extremely unusual, given that people rarely lie if there's no reason for it. Going by this baseline, you could assume upward of 50% of replies are honest (even if mis…

I've heard a theory where the inner monolog was emergent, and some of the first people to recognize thr 'voice in their heads' attributed it to god/angels/etc There's conspiratorial lying and lying from ignorance, one is much less credulous.

That theory is the "bicameral mind"; I think it's even discussed elsewhere in this thread.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#376

Post by tsunamifury »

YeGoblynQueenne wrote: post #370
tsunamifury wrote: post #337

Earlier quoted context omitted.

Simple reasoning is a side effect of compression. That is all. I see from your profile you are focused on your own personal and narrow definition of reasoning. But I’d argue there is a much broader and simpler definition. Can you summarize and apply learnings. This can.

To clarify, what I have in my profile is not my "own personal" definition of reasoning. It's how reasoning is understood in computer science and AI, and I am an expert on the subject through my doctoral studies and my current post-doc research. That's important to understand. What I have in my profile is not some idiosyncratic idea about reasoning, it's the standard, formal understanding of what reasoning means, as i…

Academics have gotten AI wrong since its inception and now are relegated to the trailing edges of the field. Mostly because increasingly insist on theory-as-fact in soft arenas that are clearly still in motion. Reasoning has been one thing, it can continue to grow to be another. But even from your defition, I can provide abductive, inductive, and other examples of it reasoning to this degree just fine. However tour examples are a bit... silly to be honest.

But keep lecturing everyone -- its very common for post-grads to be so up their own behind in their research that they've closed their world off until they are the only ones right in it.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#377

Post by retros3x »

solidasparagus wrote: post #200
indoordin0saur wrote: post #184

Earlier quoted context omitted.

My point is not that humans have a faster training speed but that humans must be doing something fundamentally different from LLMs. You could build Altman's $7 trillion dollar GPU cluster and use the majority of the world's energy to feed it and you'd still hit the same limitations if you're just running an LLM on it, even a very sophisticated LLM. This is Yann LeCun's position as well.

That, as I understand it, is not a valid chain of logic. Requiring fewer data points does not inherently indicate that the underlying mechanism (autogressive sequential generation, not the transformer which is just an architecture) is different. Not to mention the secondary arguments like - no proof that human learns faster from fewer datapoints, that's just your assumption in the sibling comment. Humans inherit info…

Well, the same thing goes for you - just because someone posts on HN doesn't mean they know what they're talking about. And if I have to decide whose assessment I trust regarding AI, I take the Turing award winner who worked for almost 40 years on AI over a random guy from the internet.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#378

Post by _proofs »

IggleSniggle wrote: post #152
layer8 wrote: post #119

Earlier quoted context omitted.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it…

Maybe, but symbolic thought can get pretty far away from what we generally call "language." I bet you can reason 1+3x=22 pretty easily without any words whatsoever, or the sound of one ascending octave after another, or the approximate G-force induced on your body if you take the next turn without applying the brakes. All of these forms of reasoning are true and useful calculations: when we talk about "intuition" wha…

i feel it is worth pointing out, as another commenter highlighted, language and even symbolic more abstract languages, bring about fluency if you've practiced "speaking and writing" it enough.

i think native speakers hardly "think" about the steps necessary to form a grammatically correct expression, and most of the time just "know".

fluency is not the same as lacking an internal framework for interpreting or thinking in terms of, symbols.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#379

Post by retros3x »

solidasparagus wrote: post #128
ericmcer wrote: post #118

Earlier quoted context omitted.

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could. Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

We don't know how we are different, but its pretty clear that we are fundamentally different. Modern machines calculate much faster than humans, if human thoughts and reasoning would work like LLM, we would already sit in front of general purpose hard AI. But we don't, so obviously there is something missing. Unfortunately its quite hard to know what is missing, because we don't know how conscience and reasoning actually work and we can't formularize them. Thats why its fair to assume we won't see general purpose AI that fast (probably not in our lifetime), which a lot of AI researchers seem to support, especially those who are not working for companies who try to sell AI.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#380

Post by YeGoblynQueenne »

tsunamifury wrote: post #376
YeGoblynQueenne wrote: post #370

Earlier quoted context omitted.

To clarify, what I have in my profile is not my "own personal" definition of reasoning. It's how reasoning is understood in computer science and AI, and I am an expert on the subject through my doctoral studies and my current post-doc research. That's important to understand. What I have in my profile is not some idiosyncratic idea about reasoning, it's the standard, formal understanding of what reasoning means, as i…

Academics have gotten AI wrong since its inception and now are relegated to the trailing edges of the field. Mostly because increasingly insist on theory-as-fact in soft arenas that are clearly still in motion. Reasoning has been one thing, it can continue to grow to be another. But even from your defition, I can provide abductive, inductive, and other examples of it reasoning to this degree just fine. However tour e…

Unfortunately I'm used to people on the internet wearing their ignorance on their sleeve like a badge of honour and so I'm not surprised by the insults in your comment. Just a bit sad to be honest :(
Post reply on HN