Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

311–320 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#311

Post by nuancebydefault »

nerdjon wrote: post #18

For anyone considering reading the paper and like me don't normally read papers like this, open the PDF and think you don't have time to read it due to its length. The main part of the paper is the first 10 pages and a fairly quick read. On to the topic here. This is an interesting example that they are using. It is fairly simplistic to understand as a human (even if we may be inclined to quickly jump to the wrong co…

> They are not reasoning or thinking in any sense of the word

There is no definition of reasoning or thinking. No single human knows what it is.

The only thing we know is: we as humans are capable of recognizing steps and results of reasoning and thinking.

In a lot of cases, when using LLM's, those results appear to be correct and usable. This is often easy to determine with generated code.

I want to argue that, lacking a definition of reasoning, I am happy to have found that the machine helps me to get results that might as well have been produced by a lot of human knowledge, wisdom and deep reasoning.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#312

Post by GeoAtreides »

glitchc wrote: post #292
GeoAtreides wrote: post #207

Earlier quoted context omitted.

Animals can not manipulate abstract concepts nor can they do long-term plans. No crow can plan an international trip spanning a couple of weeks and two change-overs. And some people definitely can't do it start to end, but they can at least plan the first 5-7 steps. Also, maybe inner monologue is not a binary have/have not, but maybe it is on a continuum.

Not sure. Migratory birds seem to manage this just fine. Not only do they make multiple stops to eat and rest, they also navigate around bad weather and still make it to their intended destination (at least most of the time).

> Migratory birds seem to manage this just fine

Instincts.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#313

Post by TeMPOraL »

slashdave wrote: post #229
solidasparagus wrote: post #176

Earlier quoted context omitted.

Tell me more?

Language is only sequential in the form it is transmitted (verbally). There is no reason that sequential statements are generated sequentially in the brain. Quite the opposite, really, if you consider rules of grammar. I really wish most of the LLM folks just took a few courses in linguistics. It would avoid a lot of noise.

Whoever cares about "rules of grammar"?

Grammar is descriptive, it formalizes the language so it doesn't break down into regional dialects too fast, and otherwise is just a crutch for people learning the language, especially if it's not their first language. The way you acquired your first language is the same way LLMs learned to utter grammatically correct sentences: by being exposed to lots and lots of examples, and eventually getting a feel for it. Similarly, if you're fluent in a language, you don't even think of grammar when using it - the right phrases in correct forms just come to you.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#314

Post by TeMPOraL »

GaggiX wrote: post #252
indoordin0saur wrote: post #157

Earlier quoted context omitted.

AI needs to see thousands or millions of images of a cat before they reliably can identify one. The fact that a child needs to only see one example of a cat to know what a cat is from then on seems to point to humans having something very different.

If the model is first pre-trained on unlabeled images, then it takes about 10 labeled images of cats and 10 labeled images of dogs to train a (possibly strong) classifier (example: DINOv2), I doubt humans will do better.

That's a good point; when comparing performance with humans, one has to remember that a human spends years of training on unlabeled images in form of continuous video stream, on top of audio streams and other senses, before they're able to communicate with you so you could test them.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#315

Post by bradfox2 »

layer8 wrote: post #119
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it…

What defines the boundaries of internal vs external? Certainly nothing about llm weights or ops should.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#316

Post by magicalhippo »

photon_lines wrote: post #265
lupire wrote: post #223

Earlier quoted context omitted.

Can you draw a picture of an example of what you see when you think about something?

Sure - with the Alice example, when I saw the problem I came up with a really simple visual example of Alice having 3 sisters and 4 brothers. When I visualized it I saw Alice standing next to 3 other women (her sisters) and her 4 brothers standing close by. When I imagined asking her brother how many sisters he has, I could see that Alice was standing there next to 3 other women and thus came up with the answer of 4.…

This could account for why some people are much better at say geometry than algebra.

I'm the opposite. I rarely visualize things I read, be it geometry or poetry. I can read a detailed description of a person or an item in a novel, and I don't really "see" anything.

But I have an active inner monologue. I "hear" myself saying the words when reading or writing, I "talk" to myself when solving problems or just thinking about stuff.

Especially when programming I'm reasoning by discussing the problem with myself, the only difference being that I usually don't open my mouth and vocalize the discussion. Though sometimes when I'm alone I do just that.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#317

Post by JohnMakin »

GeoAtreides wrote: post #207
JohnMakin wrote: post #203

Earlier quoted context omitted.

> It's impossible to plan without an internal monologue. Of course it isn't impossible, and this is backed by what we know about paleoanthropology and other instances of cognition in animals - humans were making stone tools millions of years ago, which takes planning in the form of imagining what you want the tool to look like and how you will do it and what it will be used for. It's exceedingly likely we had this ab…

Animals can not manipulate abstract concepts nor can they do long-term plans. No crow can plan an international trip spanning a couple of weeks and two change-overs. And some people definitely can't do it start to end, but they can at least plan the first 5-7 steps. Also, maybe inner monologue is not a binary have/have not, but maybe it is on a continuum.

Yes, no one is disputing that animals are not as intelligent and lack the same capacity for planning that humans do, but the post you're replying to is disputing the fact that planning is done solely through internal narrative/monologue, which is easily disprovable by pointing to the examples I did. There are many more in nature.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#319

Post by TeMPOraL »

rep_lodsb wrote: post #262
TeMPOraL wrote: post #189

Earlier quoted context omitted.

Do you think they are? We haven't had a case of individuals without training in like 10 000 years.

And judging by some comments here, humans must have gotten dumber since then. Or we would never have invented writing, agriculture or even basic clothing. "This problem is actually not that easy, the average person couldn't solve it either, especially if the numbers were bigger", "Yet another cherrypicked clickbait study to make LLMs look bad, those people are just scared of being made obsolete", etc.

> Or we would never have invented writing, agriculture or even basic clothing.

That's the thing. We had to invent those things. Along with counting, numbers, logic, arithmetic, and those stupid-ass annoying family tree riddles. We didn't get them in one step, it took a long time to build one on top of the previous. Generation by generation, each cohort of children growing up in a slightly more complex world than their parents, each generation taught how to navigate this complexity by their families and tribes. Learning a growing collection of facts and beliefs and life hacks.

There were no untrained people for as long as humanity existed. The minimum reproductive unit of homo sapiens is a village.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#320

Post by tsunamifury »

naasking wrote: post #310
PheonixPharts wrote: post #102

Earlier quoted context omitted.

People keep saying it because that's literally how LLMs work. They run Montecarlo sampling over a very impressive latent linguistic space. These models are not fundamentally different than the Markov chains of yore except that these latent representations are incredibly powerful. We haven't even started to approach the largest problem which is moving beyond what is essentially a greedy token level search of this ling…

> LLMs are not reasoning machines. They are basically semantic compression machines with a build in search feature. This is just a god of the gaps argument. Understanding is a form of semantic compression. So you're saying we have a system that can learn and construct a database of semantic information, then search it and compose novel, structured and coherent semantic content to respond to an a priori unknown prompt…

It’s basically an argument that boils down to “it’s not because I don’t like it”
Post reply on HN