Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

361–370 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#361

Post by aeternum »

ElevenLathe wrote: post #129
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

The existence of an "inner monologue" isn't really a falsifiable claim. Some people claim to have one while other people claim not to, but we can't test the truth of these claims.

Feynman came up with a potential test by tracking the things he could and couldn't do while counting seconds (via internal monologue). He found he generally count not talk while counting.

He then had others try and found that one of his mathematician friends was able to talk just fine while counting because it turned out he was counting visually.

https://www.youtube.com/watch?v=Cj4y0EUlU-Y

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#362

Post by voxic11 »

daveguy wrote: post #172
voxic11 wrote: post #130

Earlier quoted context omitted.

Yeah, I think these chatbots are just too sure of themselves. They only really do "system 1 thinking" and only do "system 2 thinking" if you prompt them to. If I ask gpt-4o the riddle in this paper and tell it to assume its reasoning contains possible logical inconsistencies and to come up with reasons why that might be then it does correctly identify the problems with its initial answer and arrives at the correct on…

> After you answer the riddle please review your answer assuming that you have made a logical inconsistency in each step and explain what that inconsistency is. Even if you think there is none do your best to confabulate a reason why it could be logically inconsistent. LLMs are fundamentally incapable of following this instruction. It is still model inference, no matter how you prompt it.

For me it did in fact follow the instruction and found the right answer. I have found this strategy of asking the LLM to analyze why its answer might be wrong results in better final answers. I think this allows the LLM to "use tokens to think" which allows the attention layers to gather the right cognitive resources to answer more subtitle questions effectively.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#363

Post by zurfer »

atlex2 wrote: post #68

I have an even simpler prompt for LLMs that run on mobile: "how many feet are in four feet"-- confabulations on this one sometimes revert to counting toes.

"Four feet is equal to four feet. The measurement does not change as it is already in the desired unit." gpt4o

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#364

Post by Mmskynettio »

nurple wrote: post #209
Mmskynettio wrote: post #136

Earlier quoted context omitted.

Its not an AI hype. A hype is defined as something which gets oversold: "promote or publicize (a product or idea) intensively, often exaggerating its benefits." Just yesterday I visited a google cloud summit and one person from bosch told the audiance how they are now able to work with less external agencies like texting, graphicsdesigner and photographers for their materials. It already saves money, has real impacts…

Your post is complete hype, all about people saying things instead of showing things that've actually been done. For me, 2024 was the LLM exposed as basically pure hype year. There is no expert of any field I follow online where they're posting up results from AI tooling for any other reason than to show how awful it is. I consider myself an expert in software, and LLMs specifically have only caused me great pain. Ev…

I'm getting jun devs who are brainstorming and troubleshooting with chatgpt before they start talking to me.

I have a good intern who is much faster with chatgpt than before and learning well.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#365

Post by skyde »

Asking gpt to first output prolog program seem to 100% fix it!

Given that Alice has 13 brothers and 31 sisters, we can update the Prolog program with this information. We need to adjust the fact about Alice's siblings and then use the rule to calculate the number of sisters her brothers have.

Here is the updated program:

```prolog % Facts siblings(alice, 13, 31). % Alice has 13 brothers and 31 sisters

% Rules % Alice's brothers have M + 1 sisters, since Alice is also their sister. sisters_of_brother(TotalSisters) :- siblings(alice, _, M), TotalSisters is M + 1.

% Initialization goal :- initialization(main).

% Predicate to find and print the total number of sisters Alice's brothers have main :- sisters_of_brother(TotalSisters), writeln(TotalSisters). ```

In this program, the fact `siblings(alice, 13, 31)` represents that Alice has 13 brothers and 31 sisters. The rule `sisters_of_brother(TotalSisters)` calculates the total number of sisters that Alice's brothers have by adding 1 to the number of sisters Alice has (since Alice is also their sister).

When we run this program, it will print the total number of sisters Alice's brothers have:

1. Alice has 31 sisters. 2. Each of Alice's brothers will also consider Alice as a sister. 3. Therefore, each brother has 31 + 1 = 32 sisters.

The output of the program will be:

``` 32 ```

Thus, Alice's brothers have 32 sisters.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#366

Post by skyde »

photon_lines wrote: post #44
nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

Given that Alice has 13 brothers and 31 sisters, we can update the Prolog program with this information. We need to adjust the fact about Alice's siblings and then use the rule to calculate the number of sisters her brothers have.

Here is the updated program:

```prolog % Facts siblings(alice, 13, 31). % Alice has 13 brothers and 31 sisters

% Rules % Alice's brothers have M + 1 sisters, since Alice is also their sister. sisters_of_brother(TotalSisters) :- siblings(alice, _, M), TotalSisters is M + 1.

% Initialization goal :- initialization(main).

% Predicate to find and print the total number of sisters Alice's brothers have main :- sisters_of_brother(TotalSisters), writeln(TotalSisters). ```

In this program, the fact `siblings(alice, 13, 31)` represents that Alice has 13 brothers and 31 sisters. The rule `sisters_of_brother(TotalSisters)` calculates the total number of sisters that Alice's brothers have by adding 1 to the number of sisters Alice has (since Alice is also their sister).

When we run this program, it will print the total number of sisters Alice's brothers have:

1. Alice has 31 sisters. 2. Each of Alice's brothers will also consider Alice as a sister. 3. Therefore, each brother has 31 + 1 = 32 sisters.

The output of the program will be:

``` 32 ```

Thus, Alice's brothers have 32 sisters.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#367

Post by defrost »

skyde wrote: post #366
photon_lines wrote: post #44

Earlier quoted context omitted.

There's actually a pretty simple solution to this that I thought about testing out and it involves asking the model to re-construct the problem using a logic language (like Prolog) and asking it to execute this type of program in order to come up with a solution rather than attempting simple chain-of-reason training / other methodologies of getting the model to 'reason' through some of these examples. People forget t…

Given that Alice has 13 brothers and 31 sisters, we can update the Prolog program with this information. We need to adjust the fact about Alice's siblings and then use the rule to calculate the number of sisters her brothers have. Here is the updated program: ```prolog % Facts siblings(alice, 13, 31). % Alice has 13 brothers and 31 sisters % Rules % Alice's brothers have M + 1 sisters, since Alice is also their siste…

Problematic in the real world meat space to say the least.

Assuming Alice indeed has that many brothers and sisters (possible) it's highly unlikely they all have the same pair of unique parents.

The Set {} of siblings that share at least one parent with Alice (the siblings of Alice) can easily include one brother of Alice (same father, different mother) who doesn't share a common parent with any other sibling of Alice.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#368

Post by Closi »

PheonixPharts wrote: post #102
tsunamifury wrote: post #90

Earlier quoted context omitted.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful. We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this ling…

I don't think you can evaluate if an LLM is reasoning by looking purely at the mechanics, because if we looked inside a human brain we wouldn't be able to conclude that it can reason either (our test is 'i think therefore i am', not all these neurons look like they are plugged together in such a way that it enables reason).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#369

Post by Mmskynettio »

hellojesus wrote: post #241
bongodongobob wrote: post #218

Earlier quoted context omitted.

Eat something and take a nap, you sound unhinged. ChatGPT has nearly doubled my work output, most of my job is system admin infra type stuff and it's ridiculously good at troubleshooting odd issues. Hopefully you can find a use case for it someday, until then, the rest of us will continue to be more productive.

Do you have any concern about the data you're feeding to the vendor serving your prompts? I've had junior devs tell me they use chatgippity to combine excel workbooks, and when I confirm they're not self hosting a llm to do it, I ask if they think it's a good idea to hand over company data to openai. They don't care. In a world of tight security, I find it astonishing that so many people willingly give away trade sec…

So you are not using Office 365?

Because our company does.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#370

Post by YeGoblynQueenne »

tsunamifury wrote: post #337
YeGoblynQueenne wrote: post #331

Earlier quoted context omitted.

Fine, but we were talking about compression, not about imitation, or inspiration, and not about creating "a work that evokes similar feelings and emotions". If I compress an image, what I get when I decompress it is that image, not "feelings and emotions", yes? In fact, that's kind of the whole point: I can send an image over the web and the receiver can form their own feelings and emotions, without having to rely on…

Simple reasoning is a side effect of compression. That is all. I see from your profile you are focused on your own personal and narrow definition of reasoning. But I’d argue there is a much broader and simpler definition. Can you summarize and apply learnings. This can.

To clarify, what I have in my profile is not my "own personal" definition of reasoning. It's how reasoning is understood in computer science and AI, and I am an expert on the subject through my doctoral studies and my current post-doc research.

That's important to understand. What I have in my profile is not some idiosyncratic idea about reasoning, it's the standard, formal understanding of what reasoning means, as it has developed in practice, in AI research in the last many decades.

I appreciate that there are many people who opine about reasoning who are not aware of that prior work and come up with their own ideas about what "reasoning" means, and some are even AI researches which is very concerning but I can't do anything about that except push back against such uninformed opinions.

>> This can.

I'm sorry, what can?

Post reply on HN