Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Simple tasks showing reasoning breakdown in state-of-the-art LLMs

arxiv.org

Post reply on HN
Print view View original

231–240 of 393 posts

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#231

Post by lupire »

more_corn wrote: post #125
ericmcer wrote: post #118

Earlier quoted context omitted.

If you really think about what an LLM is you would think there is no way that leads to general purpose AI. At the same time though they are already doing way more than we thought they could. Maybe people were surprised by what OpenAI achieved so now they are all just praying that with enough compute and the right model AGI will emerge.

LLMs are the language center of an AI. They in no way purport to be: long term memory, critical thinking engines, calculators. If we want those things we can build them. Building them into the language center would be absurd and weird.

And yet this is what all the big companies we re trying to do, because the LLM is incredibly expensive fixed cost, and they marginal cost for each new task.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#232

Post by semi-extrinsic »

photon_lines wrote: post #160
throwanem wrote: post #147

Earlier quoted context omitted.

"Prove me wrong?" That's not how this works. Your implicit claim here is that human cognition and LLM functioning are fundamentally similar. That claim requires substantiation.

I actually did a full write-up on this here fyi: https://photonlines.substack.com/p/intuitive-and-visual-guid... . You can skip most of this and scroll down to the end-section called 'The Mental Model for Understanding LLMs' where I try to map how transformers are able to mimic human thinking. I think that comparing them to auto-associative / auto-regressive networks is actually a really good analogy FYI and I do bel…

We don't have sufficient understanding of human thinking and mental processes to make any claims like the ones you are trying to make. If we did, psychology would be a quantitative hard science; it most definitely is not.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#234

Post by lupire »

dr_dshiv wrote: post #115
colechristensen wrote: post #40

Earlier quoted context omitted.

I don't think anybody who actually uses LLMs thinks they are general purpose AI. Like you ask it to do one thing it's amazing, but then you try to modify or do something with extra steps, or just anything with any complexity to it and it falls over.

Well, I believe LLMs are general purpose AI. And, for the record, so does Peter Norvig. He coauthored the most popular textbook on Artificial Intelligence, so this belief can be considered a somewhat mainstream perspective. His article, “Artificial General Intelligence Is Already Here“ [1] is really worth reading. https://www.noemamag.com/artificial-general-intelligence-is-...

I can't tell whether Norvig is making a subtle's point that artificial general stupidity is still an important form of artificial general intelligence, or he whether he somehow vastly overestimates AI model performance on non-memorization and summarization tasks.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#235

Post by Terr_ »

ElevenLathe wrote: post #129
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

The existence of an "inner monologue" isn't really a falsifiable claim. Some people claim to have one while other people claim not to, but we can't test the truth of these claims.

> The existence of an "inner monologue" isn't really a falsifiable claim.

Another possibility is that inner-monologues (ones not forced by conscious effort) do exist, but are just a kind of false-memory, something one part of our brain generates after-the-fact to explain/document the outcome of another non-language part.

Kind of like how certain reflex-actions can occur before certain decision-making area of the brain light up, yet humans will believe that they sensed the event and made a thinking choice.

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#236

Post by riku_iki »

tsunamifury wrote: post #90
pawelmurias wrote: post #59

Earlier quoted context omitted.

LLMs are idiots. They can't reason properly and only parrot stuff https://chatgpt.com/share/dcb4ff4e-e8a2-463b-86ec-9caf10b6e6... Sometimes they get the answer right to something really complex because it fits a pattern, but sometimes they answer with something really really stupid.

Why are so many people so insistent on saying this? I’m guessing you are in denial that we can make a simulated reasoning machine?

> I’m guessing you are in denial that we can make a simulated reasoning machine?

some people actually try, and see that LLMs are not there yet

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#237

Post by mbesto »

sollewitt wrote: post #105
layer8 wrote: post #94

Earlier quoted context omitted.

For reasoning of any complexity, some sort of internal monologue and iteration is needed. For type-2 thinking, we conceptualize possible solutions, arguments, and reasoning paths in our mind, and judge and evaluate them before deciding which one we think is adequate or correct. This can be emulated to some extent by guiding prompts that make that internal thought process external, but we really want it to be internal…

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

The fact that we don't actually have an understanding and framework for reasoning (e.g. whether inner monologue is a cause or an effect) means we are VERY off from general AI.

https://youtu.be/QGYbbLWn-IE?t=72

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#238

Post by ElevenLathe »

Terr_ wrote: post #230
glitchc wrote: post #132

Earlier quoted context omitted.

In this particular case, is there any reason why we simply can't take their word for it? This is not a case of where if I say "weak" or "strong", most people pick strong because no one wants to be weak, even if the context is unknown (nuclear force for example).

> In this particular case, is there any reason why we simply can't take their word for it? My concern is that if we take their word for it, we're actually buying into two assumptions which (AFAIK) are both unproven: 1. That "Internal Monologues" (not consciously forced by attention) exist in the first place, as opposed to being false-memories generated after-the-fact by our brain to explain/document a non-language pr…

Not only are they unproven, but are ultimately not provable at all. Some people will say yes, some people will say no. Probably we can take their word for it, but in the simplest case they could just lie (in either direction) and we would have no way to tell.

In short, maybe these inner monologues exist and maybe they don't, but science can't comment on that. That said, it is clearly something we are interested in, but it will need to be addressed in some other way (i.e. religion, ideology, etc.).

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#239

Post by nathan_compton »

layer8 wrote: post #119
sollewitt wrote: post #105

Earlier quoted context omitted.

Given many people don’t have an inner monologue and function just fine, it’s more likely inner monologue is a product of the reasoning process and not it’s mechanism.

It’s commonly conjectured that the emergence of human-level reasoning wouldn’t have been possible without the development of language. Personally, I’m able to suppress “word thoughts” in my head (for a short time), but then I lose almost all of my reasoning ability. I could imagine that reasoning is language-based even when it’s not conscious for some people. An internal process being there, and being conscious of it…

Brains are weird. I reason almost entirely non-verbally and I would absolutely struggle if I had to laboriously express every thought in words. Its part of the reason I don't work well in teams. So slow!

Re: Simple tasks showing reasoning breakdown in state-of-the-art LLMs

#240

Post by marzell »

dematz wrote: post #53
llm_trw wrote: post #23

Earlier quoted context omitted.

>Worth noting that the prompts from the experiment include "To answer the question, DO NOT OUTPUT ANY TEXT EXCEPT following format that contains final answer: ### Answer:" so it appears that they are stopping the models from 'thinking out loud'. If I add that to the prompt, GPT4o gets it consistently wrong... Yes this is a common thing I see people who think LLMs are idiots do. The more an LLM talks the smarter it ge…

I often want chatgpt to answer concisely and tell it that. If it really needs to do this 'thinking out loud', could it do that under the hood and not in the final output on my screen? Its first pass could use as many words as it wants to compute the answer, but once the answer is computed please go back and make it short. Not to take away from your point that maybe the prompt is the problem in these reasoning questio…

I believe the "thinking out loud" is fundamentally part of the process of "text completion" which is what it is doing. Certainly we can (and do) break things apart and add layers that could be used to effectively do this by adding more steps and processing time. But ultimately in a single turn, the entire conversation up to that point (including instructions you may have added telling it to not think out loud) is the input, and the output will reflect that.
Post reply on HN