Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case for zero-error horizons in trustworthy LLMs

arxiv.org

Post reply on HN
Print view View original

41–50 of 121 posts

Re: The case for zero-error horizons in trustworthy LLMs

#41

Post by 8note »

staticshock wrote: post #6

LLMs seem to me closer to Kahneman's System 1 than to System 2. When understood in this way, it is obvious why LLMs are bad at counting r's in "strawberries". But it also makes ZEH feel like it couldn't possibly be a useful metric, because it's a System 2 evaluation applied to a System 1 system.

> When understood in this way, it is obvious why LLMs are bad at counting r's in "strawberries".

no it doesnt. it makes sense that they cant count the rs because they dont have access to the actual word, only tokens that might represent parts or the whole of the word

Re: The case for zero-error horizons in trustworthy LLMs

#42

Post by Topfi »

emp17344 wrote: post #28

[flagged]

No disrespect to them, but unless there is a financial incentive at stake for them (beyond SnP500 exposure), I've gotten to viewing this through the lens of sports teams, gaming consoles and religions. You pick your side, early and guided by hype and there is no way that choice can have been wrong (just like the Wii U, Dreamcast, etc. was the best).

Their viewpoint on this technology has become part of the identity for some unfortunately and any position that isn't either "AGI imminent" or "This is useless" can cause some major emotions.

Thing is, this finding being the case (along with all other LLM limits) does not mean that these models aren't impactful and shouldn't be scrutinised, nor does it mean they are useless. The truth is likely just a bit more nuanced than a narrow extreme.

Also, mental health impact, job losses for white collar workers, privacy issues, concerns of rights holders on training data collection, all the current day impacts of LLMs are easily brushed aside by someone believing that LLMs are near the "everyone dies" stage, which just so happens to be helpful if one were to run a lab. Same if you believe these are useless and will never get better, any discussion about real-life impacts is seen as trying to slowly get them to accept LLMs as a reality, when to them, they never were and never will be.

Re: The case for zero-error horizons in trustworthy LLMs

#43

Post by toraway »

raincole wrote: post #24
coldtea wrote: post #16

Earlier quoted context omitted.

Even more interesting to track how many of those are just ad-hoc patched.

Probably zero. At the end of the day people pay for LLMs that write better code or summarize PDFs of hundreds of pages faster, not the ones that can count the letter r's better. When LLMs can't count r's: see? LLMs can't think. Hoax! When LLMs count r's: see? They patched and benchmark-maxxed. Hoax! You just can't reason with the anti-LLM group.

Whenever an "LLM fail" goes viral like the car wash question, you can observe the exact same wording of the question get "fixed" within a week or so. With slight variations in phrasing still able to replicate the problem.

Followed by lots of "works perfectly for me, why are people even talking about this?"

I can't say what exactly they're doing behind the scenes but it's a consistent pattern among the big SOTA model providers. With obvious incentive to "fix" the problem so users will then organically "debunk" the meme as they try it themselves and share their experiences.

Re: The case for zero-error horizons in trustworthy LLMs

#45

Post by azakai »

kenjackson wrote: post #11

Whenveer I see these papers and try them, they always work. This paper is two months old, which in LLM years is like 10 years of progress. It would be interesting to actively track how far long each progressive model gets...

You are trying it on a production model. The paper is using models with tool calls disabled.

Re: The case for zero-error horizons in trustworthy LLMs

#46

Post by im3w1l »

staticshock wrote: post #6

LLMs seem to me closer to Kahneman's System 1 than to System 2. When understood in this way, it is obvious why LLMs are bad at counting r's in "strawberries". But it also makes ZEH feel like it couldn't possibly be a useful metric, because it's a System 2 evaluation applied to a System 1 system.

A big part of skill aquisition in humans is moving tasks from system 2 to system 1, to free up the very scarce thinking resources for ever more complex tasks, that can then in turn be internalized and handled by system 1.

Re: The case for zero-error horizons in trustworthy LLMs

#47

Post by kenjackson »

revachol wrote: post #34
kenjackson wrote: post #11

Whenveer I see these papers and try them, they always work. This paper is two months old, which in LLM years is like 10 years of progress. It would be interesting to actively track how far long each progressive model gets...

I just tried it in ChatGPT "Auto" and it didn't work > Yes — ((((()))))) is balanced. > It has 6 opening ( and 6 closing ), and they’re properly nested. Though it did work when using "Extensive Thinking". The model wrote a Python program to solve this. > Almost balanced — ((((()))))) has 5 opening parentheses and 6 closing parentheses, so it has one extra ). > A balanced version would be: ((((())))) Testing a couple…

Weird. I tried in chatGPT auto and it worked perfectly. I tried like 10 variations. I also did the letters in words. Got all of them right.

The one thing I did trip it up on was "Is there the sh sound in the word transportation". It said no. And then realized I asked for "sound" not letters. It then subsequently got the rest of the "sounds-like" tests I did.

Clearly, my ChatGPT is just better than yours.

Re: The case for zero-error horizons in trustworthy LLMs

#48

Post by Lerc »

wahnfrieden wrote: post #21
pants2 wrote: post #20

Doesn't this just look like another case of "count the r's in strawberry" ie not understanding how tokenization works? This is well known and not that interesting to me - ask the model to use python to solve any of these questions and it will get it right every time.

It's not dismissible as a misunderstanding of tokens. LLMs also embed knowledge of spelling - that's how they fixed the strawberry issue. It's a valid criticism and evaluation.

The r's in strawberry presents a different level of task to what people imagine. It seems trivial to a naive observer because the answer is easily derivable from the question without extra knowledge.

A more accurate analogy for humans would be to imagine if every word had a colour. You are told that there are also a sequence of different colours that correspond to the same colour as that word. You are even given a book showing every combination to memorise.

You learn the colours well enough that you can read and write coherently using them.

Then comes the question of how many chocolate-browns are in teal-with-a-hint-of-red. You know that teal-with-a-hint-of-red is a fruit and you know that the colour can also be constructed by crimson followed by Disney-blond. Now, do both of those contain chocolate-brown or just one of them, how many?

It requires excersizing memory to do a task that is underrepresented in the training data because humans simply do not have to do the task at all when the answer can be derived from the question representation. Humans also don't have the ability that the LLMs need but the letter representation doesn't need that ability.

Re: The case for zero-error horizons in trustworthy LLMs

#49

Post by dwa3592 »

dwa3592 wrote: post #44

Nice! Although I tried the parenthesis balanced question with gemini and it gave the right answer in first attempt.

but it's a tricky question for LLMs; it shows that if it's not in the training set; LLMs could trip which kinda shows that the intelligence is not generalized yet.

I tried this with gemini - (i am trying(something(re(a(l(ly)c)r)a)z)((y)he)re)

and it tripped.

Re: The case for zero-error horizons in trustworthy LLMs

#50

Post by emp17344 »

azakai wrote: post #40
emp17344 wrote: post #37

Earlier quoted context omitted.

I think this is still useful research that calls into question how “smart” these models are. If the model needs a separate tool to solve a problem, has the model really solved the problem, or just outsourced it to a harness that it’s been trained - via reinforcement learning - to call upon?

It has "outsourced" it to another component, sure, but does that matter? What the user sees is the total behavior of the entire system, not whether the system has internal divisions and separations.

It matters if you’re curious about whether AGI is possible. Have we really built “thinking machines”, or are these systems just elaborate harnesses that leverage the non-deterministic nature of LLMs?
Post reply on HN