Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

251–260 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#251

Post by IlliOnato »

saurik wrote: post #97
rossdavidh wrote: post #51

Earlier quoted context omitted.

Even if this were all true, it points to a fundamental risk of using LLM's for important tasks, which is that it is not at all clear to a user that this prompt would cause a problem. The LLM doesn't say "I'm sorry Dave, I just can't do that", it just complies with it and gets the wrong answer. You can always make excuses for the LLM afterwards, but software with hidden risks like this would not be considered good or…

People really need to stop trying to model an LLM as some kind of magical software component: it all makes a lot more sense if you model it as an under-performing poorly-aligned employee; so like, maybe a distracted kid working for peanuts at your store. You wouldn't trust them to with all of your money and you wouldn't trust them to do a lot of math--if they had to be in charge of checkout, you'd make sure they are…

You have a point...

I once gave a 10-dollar bill to a young man serving at the cashier at a store, and he gave me 14 dollars back as a change. I pointed out that this made no sense. He bent down, looked closer at the screen of his machine, and said "Nope, 14 dollars, no mistake". I asked him if he thought I gave him 20. He said no, and even shown me the 10-dollar bill I just gave him. At that point I just gave up and took the money.

Now that I think about it, there was an eerie similarity between this conversation and some of the dialogues I had with LLMs...

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#252

Post by GaggiX »

indoordin0saur wrote: post #157
solidasparagus wrote: post #128

Earlier quoted context omitted.

> If you really think about what an LLM is you would think there is no way that leads to general purpose AI It is an autoregressive sequence predictor/generator. Explain to me how humans are fundamentally different

AI needs to see thousands or millions of images of a cat before they reliably can identify one. The fact that a child needs to only see one example of a cat to know what a cat is from then on seems to point to humans having something very different.

If the model is first pre-trained on unlabeled images, then it takes about 10 labeled images of cats and 10 labeled images of dogs to train a (possibly strong) classifier (example: DINOv2), I doubt humans will do better.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#253

Post by hellooodarkness »

Just try the question with GPT-4o and the results are more entertaining than I would expected.

Prompt: Alice has 3 brothers and she also has 6 sisters. How many sisters does Alice’s brother have? Solve the problem step-by-step

Answer: To solve the problem step-by-step:

Identify the total number of siblings Alice has: Alice has 3 brothers. Alice has 6 sisters. Understand that Alice herself is one of the sisters: Including Alice, there are a total of 7 sisters (6 sisters + Alice).

Determine the number of sisters Alice's brother has: Each of Alice's brothers will have the same number of sisters as Alice does, excluding themselves.

Since Alice's brothers have the same set of siblings, including the 7 sisters: Therefore, each of Alice's brothers has 6 sisters (not counting themselves as a sister). So, each of Alice’s brothers has 6 sisters.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#254

Post by asadotzler »

photon_lines wrote: post #44
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

If that kind of thing worked, we'd have been doing it long before LLM chatbots.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#255

Post by twiceaday »

hifromwork wrote: post #205
IggleSniggle wrote: post #152

Earlier quoted context omitted.

Maybe, but symbolic thought can get pretty far away from what we generally call "language." I bet you can reason 1+3x=22 pretty easily without any words whatsoever, or the sound of one ascending octave after another, or the approximate G-force induced on your body if you take the next turn without applying the brakes. All of these forms of reasoning are true and useful calculations: when we talk about "intuition" wha…

>I bet you can reason 1+3x=22 pretty easily without any words whatsoever I've tried to do it, but I can't. I had to do something like "ok, so we subtract one from both sides and then it's easy, 3*7=21". Maybe I could do 2+8 but I still think the word ten "aloud".

I was able to do it with no words. I 'saw' the steps as if on a piece of paper. I saw 3x=22-1=21, then x=21/3=7. But I have a degree in applied math. Perhaps not internally vocalizing is just being extremely familiar. It also happened very quickly, perhaps there was no time to vocalize anyways.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#256

Post by mjburgess »

kalkin wrote: post #153
nerdjon wrote: post #41

Earlier quoted context omitted.

> In many ways, this is very obvious and routine to people who use these systems with a critical understanding of how they work. The last part of that is the problem and why a paper like this is critical. These systems are being pushed onto people who don't understand how they work. CEO's and other business leaders are being pushed to use AI. Average users are being shown it in Google search results. Etc etc. People…

> People are being told it can do far more than it really is. Meanwhile these HN comments are split between: * Lots of people confirming what the paper itself notes (but doesn't highlight), that the most advanced models actually can solve this problem at least a significant portion of the time. (A proportion which one can pretty easily project is only likely to increase with future models.) * Lots of people saying "t…

The critical understanding doesnt predict that LLMs cannot solve problems. It predicts how they will solve them. There is no information, a priori, what the LLM has been trained on. You have to prompt, then see the answer. Once the answer arrives, the critical understanding provides a route to repairing the answer when not accurate or useful.

LLMs do not reason. They appear to reason by repeating the structure of reasoning in their training data. This is indistinguishable in many cases.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#257

Post by oglop »

I’m sure this will devastate the stock price and people’s pereception that general AI is here and amazing and going to take our jobs and and and.

But I am a hacker news peep. I’ll read this and lecture my manager in the next meeting about the shortcomings only to be dismissed and watch money funnel into this monolithic autistic secretary.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#258

Post by tsunamifury »

foobiekr wrote: post #243
PheonixPharts wrote: post #102

Earlier quoted context omitted.

People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful. We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this ling…

Exactly right and well said.

This type of self affirmation has a quality of denial.

Also the above description is reductive to the point of "Cars can't get you anywhere because they aren't horses."

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#259

Post by Workaccount2 »

photon_lines wrote: post #160
throwanem wrote: post #147

Earlier quoted context omitted.

"Prove me wrong?" That's not how this works. Your implicit claim here is that human cognition and LLM functioning are fundamentally similar. That claim requires substantiation.

I actually did a full write-up on this here fyi: https://photonlines.substack.com/p/intuitive-and-visual-guid... . You can skip most of this and scroll down to the end-section called 'The Mental Model for Understanding LLMs' where I try to map how transformers are able to mimic human thinking. I think that comparing them to auto-associative / auto-regressive networks is actually a really good analogy FYI and I do bel…

We have no idea how human thinking works and no idea what consciousness is.

I'm getting so tired of listening to software engineers LARP pseudo neuroscientists with 6th grade level insights.

>Of course, the [AI] brain isn’t ‘conscious.’ It doesn’t have any survival instincts which we humans do.

Bruh...

Post reply on HN