Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case for zero-error horizons in trustworthy LLMs

arxiv.org

Post reply on HN
Print view View original

111–120 of 121 posts

Re: The case for zero-error horizons in trustworthy LLMs

#111

Post by flextheruler »

orbital-decay wrote: post #54
grey-area wrote: post #22

To those saying this is not surprising, yes it will be surprising to the general public who are being served ads from huge companies like MS or OpenAI saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc. This is important information for anyone to understand who thinks these systems are thinking, reasoning, and learning from them o…

Quick sanity check: you're susceptible to pretty irresistible optical illusions which would never fool a VLM, does it mean you're not thinking? In fact, with a non-monospaced font I also have trouble determining whether these parens are balanced, and have to select them with the mouse, i.e. use a "dumb" tool, to make sure. Reminder that "thinking" is an ill-defined term like others, and the question whether they "thi…

Who is hiring anyone to look at a screen to count characters? Don't be disingenuous in your argument. The apt comparison would be the current technique used to accomplish this task i.e. a pattern matching algorithm.

Re: The case for zero-error horizons in trustworthy LLMs

#112

Post by azakai »

irishcoffee wrote: post #108
azakai wrote: post #39

Earlier quoted context omitted.

I do think this is a tool issue. Here is what the article says: > For the multiplication task, note that agents that make external calls to a calculator tool may have ZEH = ∞. While ZEH = ∞ does have meaning, in this paper we primarily evaluate the LLM itself without external tool calls The models can count to infinity if you give them access to tools. The production models do this. Not that the paper is wrong, it is…

So, the tools can count then? Humans can fly, they just need wings!

It is academically interesting what pure neural networks can do, of course. But when someone goes to Claude and tries to do something, they don't care if it solves the problem using a neural network or a call out to Python. So long as the result is right.

More generally, the ability to use tools is a form of intelligence, just like when humans and crows do it. Being able to craft the right Python script and use the result is non-trivial.

Re: The case for zero-error horizons in trustworthy LLMs

#113

Post by whateveracct »

hu3 wrote: post #38

> we found that GPT-5.2 cannot even compute the parity of a short string like 11000, and GPT-5.2 cannot determine whether the parentheses in ((((()))))) are balanced. I think there is a valid insight here which many already know: LLMs are much more reliable at creating scripts and automation to do certain tasks than doing these tasks themselves. For example if I provide an LLM my database schema and tell it to scan f…

That's because abstraction is compression of information.

Re: The case for zero-error horizons in trustworthy LLMs

#114

Post by Chobilet »

simianwords wrote: post #90
Chobilet wrote: post #87

Earlier quoted context omitted.

I'm sure this comment was made in good faith, but most researchers would rightfully understand these intricacies, and this is likely intentional(as noted in the paper). At a quick glance, I cannot say whether or not the paper has been peer reviewed(though unlikely/in process given how recent it was published). In general, you'd find published papers also listed in a specific journal/conference(i.e. not just the archi…

I'm asking, why use a thinking model without allowing it to reason? No one uses it in that way.. >While LLMs appear extremely intelligent and capable of reasoning, they sometimes make mistakes that seem inconceivably foolish from a human perspective. For example, GPT-5.2 can implement complex fluid dynamics simulation code, yet it cannot even compute the parity of the short string 11000, cannot determine whether the…

"We propose Zero-Error Horizon (ZEH) for trustworthy LLMs, which represents the maximum range that a model can solve without any errors."

I'm again taking your responses in good faith, but the abstract answers your question about what they are trying to achieve. For any statistical significance, you'd want to point to a baseline comparison(e.g. what I'm guessing is what you mean by "no reasoning" here). You'll also note within the paper, the author argues and cites that failing at the baseline step(e.g. multiplication) has shown "that error often adversely affects subsequent reasoning [38, 44]".

Which indicates to me, we don't need to use further "reasoning" given previous results/studies show a decline once our base has an error. To me, this seems like a fair assumption. Given though this is an active field of research, and we are largely testing a black box application, we can't say for certain. Further studies(like this one) will give researchers a better understand at what is and isn't possible.

Re: The case for zero-error horizons in trustworthy LLMs

#117

Post by graemefawcett »

BoredomIsFun wrote: post #105
graemefawcett wrote: post #60

Earlier quoted context omitted.

It's not just an issue of tokenization, it's almost a category error. Lisp, accounting and the number of r's in strawberry are all operations that require state. Balancing ((your)((lisp)(parens))) requires a stack, count r's in strawberry requires a register, counting to 5 requires an accumulator to hold 4. An LLM is a router and completely stateless aside from the context you feed into it. Attention is just routing…

> An LLM is a router and completely stateless aside from the context you feed into it. Not the latest SSM and hybrid attention ones.

Stateless router to router with lossy scratchpad is a step up, still not going to ask it to check my Lisp. That's what linters are for

Re: The case for zero-error horizons in trustworthy LLMs

#118

Post by orbital-decay »

dwa3592 wrote: post #110
orbital-decay wrote: post #67

Earlier quoted context omitted.

Intuitively this looks like an architectural artifact (like optical illusions in humans) or a natural property of learning rather than a lack of generalization. I have issues with your example too and have to count slowly to make sure.

Right, I am sure you were able to solve it albeit slowly- you knew you had to do it slow. LLMs which are mathematicians don't know that and can't seem to understand that they need to do it slowly.

They do if they are trained to use a reasoning chain or another form of loopback, and you don't overwhelm it, or if they are optimized to search for the solution forever. There's nothing fundamental about it, only the fact that the raw transformer expressivity is limited by the single pass through the layers, which is circumvented by the loopback.

And I'm still pretty likely to make the off-by-one error even if I slow down, and there are certain optical illusions are nearly guaranteed to confuse me no matter how hard I try, particularly if I don't use any visual guides (i.e. tools). VLMs will not make my mistakes but will make their own ones, because their quirks are different from the quirks of my visual cortex.

Re: The case for zero-error horizons in trustworthy LLMs

#120

Post by grey-area »

stratos123 wrote: post #107
grey-area wrote: post #102

Earlier quoted context omitted.

A machine which confabulates and cannot count is not a good fit for accounting tasks. They’ll make all sorts of subtle errors which are difficult for humans to notice.

That wouldn't even necessarily be true if models really "couldn't count", since software exists - if an LLM is making an Excel spreadsheet rather than doing everything manually, it's both much harder for it to mess up and easier to notice and recover. It's even less true given that what this paper actually tests is "LLMs don't have a literally perfect accuracy when you make them do increasingly big problems with zero…

They really can’t count, that’s not how they work at all. They don’t reason about maths they predict the most likely output for a given context. That’s sometimes useful but not at all the same thing.
Post reply on HN