Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

241–250 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#241

Post by hellojesus »

bongodongobob wrote: post #218
nurple wrote: post #209

Earlier quoted context omitted.

Your post is complete hype, all about people saying things instead of showing things that've actually been done. For me, 2024 was the LLM exposed as basically pure hype year. There is no expert of any field I follow online where they're posting up results from AI tooling for any other reason than to show how awful it is. I consider myself an expert in software, and LLMs specifically have only caused me great pain. Ev…

Eat something and take a nap, you sound unhinged. ChatGPT has nearly doubled my work output, most of my job is system admin infra type stuff and it's ridiculously good at troubleshooting odd issues. Hopefully you can find a use case for it someday, until then, the rest of us will continue to be more productive.

Do you have any concern about the data you're feeding to the vendor serving your prompts?

I've had junior devs tell me they use chatgippity to combine excel workbooks, and when I confirm they're not self hosting a llm to do it, I ask if they think it's a good idea to hand over company data to openai. They don't care.

In a world of tight security, I find it astonishing that so many people willingly give away trade secrets to these companies, whom can sell it to any bidder if they choose.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#242

Post by Terr_ »

ElevenLathe wrote: post #238
Terr_ wrote: post #230

Earlier quoted context omitted.

> In this particular case, is there any reason why we simply can't take their word for it? My concern is that if we take their word for it, we're actually buying into two assumptions which (AFAIK) are both unproven: 1. That "Internal Monologues" (not consciously forced by attention) exist in the first place, as opposed to being false-memories generated after-the-fact by our brain to explain/document a non-language pr…

Not only are they unproven, but are ultimately not provable at all. Some people will say yes, some people will say no. Probably we can take their word for it, but in the simplest case they could just lie (in either direction) and we would have no way to tell. In short, maybe these inner monologues exist and maybe they don't, but science can't comment on that. That said, it is clearly something we are interested in, b…

> but are ultimately not provable at all

No, they are potentially falsifiable as we get better at scanning, identifying, intervening in brain activity.

Just off the top of my head here, suppose we create a table puzzle problem that (in itself) doesn't require language to understand, like ones we make for certain animals. Have human subjects (silently) solve it. Afterwards, quiz the solvers about their internal monologue--or lack thereof--dividing them into two groups and noting the words used.

Now change to a second puzzle of similar style and same overall difficult. Stun/anesthetize the language-centers of subjects, to deny access to any of the monologue-words (validating this intervention will involve other research), and then test them on the second problem.

* If performance is constant for both groups, that suggests the monologue is illusory or at least not needed for this kind/scope of problem.

* If performance drops for both groups, that suggests the no-monologue people might just not be as aware of a linguistic process that's actually happening.

* If performance drops for monologue-subjects, that suggests it's a real and important difference in modes of logical thought.

* If some other combination happens, you have an mysterious and exciting new line of research.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#243

Post by foobiekr »

PheonixPharts wrote: post #102
tsunamifury wrote: post #90

Earlier quoted context omitted.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful. We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this ling…

Exactly right and well said.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#244

Post by lupire »

michael-ax wrote: post #86
toxik wrote: post #7

Earlier quoted context omitted.

Seems like an odd thing not to state prominently. ChatGPT 4o solved it every time for me. Seems to work OK even with the "hands tied behind back" prompt. Wonder if this paper is just inaccurate or if OpenAI adjusted the model -- seems unlikely. Actually, 4o is reported to solve this riddle 60% of the time. I guess I have to test more, but with my about a dozen tests so far, all were correct.

Alice has N Brothers, and she has M sisters. How many sisters do Alice’s brothers have? I have not gotten the correct answer to the question as phrased above in one go from Gpt4o yet! (and today was not the first day i tried.) Phrase it as shown above and you'll likely need 5 or more interactions to get it to generate the correct output. With Gemini i could not get it below 8 without feeling like i was cheating. fwiw…

Chat GPT 4o. I was being a bit generous with background information, but still tests ability to interpret:

------

Me:

Background facts:

Alice is a female human.

All sisters are female, and all brothers are male. No one is their own brother or sister.

Alice has N brothers, and Alice has M sisters.

Now, a few questions based on these facts: How many sisters do Alice’s brothers have?

Do Alice's brothers have more sisters, fewer sisters, or same number of sisters than Alice has?

Don't forget to consider Alice when counting.

------

ChatGPT:

  Let's break down the questions step by step:

    How many sisters do Alice’s brothers have?

    Alice has M sisters. Since all sisters are female and Alice is included in the total count of sisters for her brothers, Alice's brothers have M+1 sisters. This includes Alice herself and her M sisters.

    Do Alice's brothers have more sisters, fewer sisters, or the same number of sisters as Alice has?

    Alice has M sisters. Alice's brothers have M+1 sisters (since it includes Alice herself and her M sisters).

    Therefore, Alice's brothers have more sisters than Alice has.

  In conclusion:

    Alice’s brothers have M+1 sisters.

    Alice's brothers have more sisters than Alice has.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#245

Post by TeMPOraL »

indoordin0saur wrote: post #186
TeMPOraL wrote: post #180

Earlier quoted context omitted.

Humans train on continuous video . Even our most expensive models are, in terms of training set size, far behind what an infant processes in the first year of their life. EDIT: and it takes human children a couple years to reliably identify a cat. My 2.5 y.o. daughter still confuses cats with small dogs, despite living under one roof with a cat.

I contend that you could show any child old enough to communicate in basic English a photograph (so not live continuous video) of some obscure animal they've never seen before (say an Okapi) and they'd be able to easily identify another Okapi when seeing one at a zoo.

My daughter is 5 y.o., which means because of kindergarten, I spend plenty of time about kids this age. A random kid this age would absolutely fail your test. They may remember the word after one exposure, but I doubt they'll remember any of the distinctive features.

Hell, many adults would fail it. I'm not sure if I could pass such test - in my experience, you remember the important details only after first experiencing a test and realizing what exactly it is that would be useful in distinguishing the two animals.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#246

Post by lupire »

imtringued wrote: post #148
mjburgess wrote: post #21

Many of the datasets for the "benchmarks" on which the major public LLMs are assessed are clearly present in their training data, making them basically useless for establishing reliability of the models. Its fairly obvious that at least some of the improved scores from later generations of models are that this benchmark data is increasingly represented in the training data. A better way of assessing LLMs is waiting a…

Honestly every time I see someone use MMLU as a benchmark I am wondering what they are trying to prove. MMLU is a simple multiple choice test with the answers being available. Simply picking answers at random should give you 25 points. Knowing 50% of the answers and picking the rest randomly gives you 62.5%, which is very close to the scores of SOTA LLMs. The benchmarks that supposedly show reasoning are pretty bad a…

I remember when ChatGPT 4 posted a set of standardized trst benchmarks, including AMC 12 math test, where ChatGPT scored higher than guessing randomly, but lower than leaving the test blank (due to the penalty for wrong answers).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#247

Post by GaggiX »

slashdave wrote: post #229
solidasparagus wrote: post #176

Earlier quoted context omitted.

Tell me more?

Language is only sequential in the form it is transmitted (verbally). There is no reason that sequential statements are generated sequentially in the brain. Quite the opposite, really, if you consider rules of grammar. I really wish most of the LLM folks just took a few courses in linguistics. It would avoid a lot of noise.

Since LLMs seem to follow the rules of grammar quite well, and this is quite the opposite of "sequential", I guess LLMs do not process informations very sequentially.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#248

Post by zdp7 »

The problem here isn't that you can get the right answer with additional prompting. The problem is how do we verify an answer when we don't know the answer? If it tells me 5 sisters and it should be 6, it failed. If I have to provide additional information, it really isn't reasoning. I would argue to show the ability to reason, it needs to be able to work out the answer completely on it's own. If I were testing a human the only input I would provide is that they are wrong and to try again. AI is very useful, but at this point I'm not comfortable asking questions I don't already know the answer for.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#249

Post by hellojesus »

bithive123 wrote: post #219
glitchc wrote: post #131

Earlier quoted context omitted.

The problem is a good chunk of the global population is also not reasoning and thinking in any sense of the word. Logical reasoning is a higher order skill that often requires formal training. It's not a natural ability for human beings.

In "any sense" of the word? Surely anyone who adjusts their behavior when they get undesired or unexpected results is reasoning and thinking. And since most activities are mediated by thought of some kind, most people are reasoning and thinking otherwise they would never recover from even simple mistakes, like walking east when they need to go north. Saying they're "not thinking in any sense of the word" because they…

This seems to me to be where these systems need to go in the future, akin to reinforcement learning.

You feed an llm a prompt. It then abstracts and approximates what the result should be. It then devises a hypothesis and solves it and compares it to the approximated output. Then it can then formulate a new hypothesis and evaluate it, based off the outcome of hypothesis 1. From there it can either keep iterating or dump that path for a new one (e.g., the next best hypothesis in the original formation).

At some point the answer is "good enough." But along the way it keeps playing against its thoughts to see if it can do better.

A key issue may be the original approximation, so it may need to consider its adjustment when iterating.

Maybe this is how cutting edge llms work now. I have no idea.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#250

Post by psadri »

layer8 wrote: post #119
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it…

Language is a serialization of our brain's "world model" structures.
Post reply on HN