Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case for zero-error horizons in trustworthy LLMs

arxiv.org

Post reply on HN
Print view View original

101–110 of 121 posts

Re: The case for zero-error horizons in trustworthy LLMs

#101

Post by dang »

emp17344 wrote: post #84
dang wrote: post #80

Earlier quoted context omitted.

Please don't start generic flamewars on HN or impugn people who take an opposing view to yours. Both these vectors lead to tedious, unenlightening threads. There's plenty of rage to go around on literally every divisive topic, and it's not the place we want discussions to come from here. " Eschew flamebait. Avoid generic tangents. " " Comments should get more thoughtful and substantive, not less, as a topic gets more…

There are other users in this very thread using inflammatory language to attack this paper and those who find the paper compelling. One user says, quote: “You just can't reason with the anti-LLM group.” In light of this, why was my comment - which was in large part a reaction to the behavior of the users described above - the only one called out here?

Purely because I didn't see the others.

Re: The case for zero-error horizons in trustworthy LLMs

#102

Post by grey-area »

stratos123 wrote: post #63
grey-area wrote: post #22

To those saying this is not surprising, yes it will be surprising to the general public who are being served ads from huge companies like MS or OpenAI saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc. This is important information for anyone to understand who thinks these systems are thinking, reasoning, and learning from them o…

> saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc. Why do you think the results of this paper contradict these claims at all?

A machine which confabulates and cannot count is not a good fit for accounting tasks. They’ll make all sorts of subtle errors which are difficult for humans to notice.

Re: The case for zero-error horizons in trustworthy LLMs

#103

Post by grey-area »

dghlsakjg wrote: post #92
emp17344 wrote: post #37

Earlier quoted context omitted.

I think this is still useful research that calls into question how “smart” these models are. If the model needs a separate tool to solve a problem, has the model really solved the problem, or just outsourced it to a harness that it’s been trained - via reinforcement learning - to call upon?

Does it matter if the LLM can solve the problem or if it knows to use a resource? There’s plenty of math that I couldn’t even begin to solve without a calculator or other tool. Doesn’t mean I’m not solving math problems. In woodworking, the advice is to let the tool do the work. Does someone using a power saw have less claim to having built something than a handsaw user? Does a CNC user not count as a woodworker beca…

It does matter because the LLM doesn’t always know when to use tools (e.g. ask it for sales projections which are similar to something in its weights) and is unable to reason about the boundaries of its knowledge.

Re: The case for zero-error horizons in trustworthy LLMs

#104

Post by emp17344 »

dang wrote: post #101
emp17344 wrote: post #84

Earlier quoted context omitted.

There are other users in this very thread using inflammatory language to attack this paper and those who find the paper compelling. One user says, quote: “You just can't reason with the anti-LLM group.” In light of this, why was my comment - which was in large part a reaction to the behavior of the users described above - the only one called out here?

Purely because I didn't see the others.

Fair enough

Re: The case for zero-error horizons in trustworthy LLMs

#105

Post by BoredomIsFun »

graemefawcett wrote: post #60
pants2 wrote: post #20

Doesn't this just look like another case of "count the r's in strawberry" ie not understanding how tokenization works? This is well known and not that interesting to me - ask the model to use python to solve any of these questions and it will get it right every time.

It's not just an issue of tokenization, it's almost a category error. Lisp, accounting and the number of r's in strawberry are all operations that require state. Balancing ((your)((lisp)(parens))) requires a stack, count r's in strawberry requires a register, counting to 5 requires an accumulator to hold 4. An LLM is a router and completely stateless aside from the context you feed into it. Attention is just routing…

> An LLM is a router and completely stateless aside from the context you feed into it.

Not the latest SSM and hybrid attention ones.

Re: The case for zero-error horizons in trustworthy LLMs

#106

Post by simianwords »

dontlikeyoueith wrote: post #100
simianwords wrote: post #81

Earlier quoted context omitted.

It was surprising to me and when I reviewed the paper, I found serious flaws that calls the fundamental claims into question - they didn't use any reasoning tokens. Any LLM or human will fail at a task like this if not allowed to think.

Calling "reasoning tokens" "thinking" is a complete confusion of concepts on your part.

why?

Re: The case for zero-error horizons in trustworthy LLMs

#107

Post by stratos123 »

grey-area wrote: post #102
stratos123 wrote: post #63

Earlier quoted context omitted.

> saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc. Why do you think the results of this paper contradict these claims at all?

A machine which confabulates and cannot count is not a good fit for accounting tasks. They’ll make all sorts of subtle errors which are difficult for humans to notice.

That wouldn't even necessarily be true if models really "couldn't count", since software exists - if an LLM is making an Excel spreadsheet rather than doing everything manually, it's both much harder for it to mess up and easier to notice and recover. It's even less true given that what this paper actually tests is "LLMs don't have a literally perfect accuracy when you make them do increasingly big problems with zero thinking".

(Confabulation is IMO a much bigger problem, but it's unrelated to architecture - it's an artifact of how models are currently trained.)

Re: The case for zero-error horizons in trustworthy LLMs

#108

Post by irishcoffee »

azakai wrote: post #39
wahnfrieden wrote: post #21

Earlier quoted context omitted.

It's not dismissible as a misunderstanding of tokens. LLMs also embed knowledge of spelling - that's how they fixed the strawberry issue. It's a valid criticism and evaluation.

I do think this is a tool issue. Here is what the article says: > For the multiplication task, note that agents that make external calls to a calculator tool may have ZEH = ∞. While ZEH = ∞ does have meaning, in this paper we primarily evaluate the LLM itself without external tool calls The models can count to infinity if you give them access to tools. The production models do this. Not that the paper is wrong, it is…

So, the tools can count then?

Humans can fly, they just need wings!

Re: The case for zero-error horizons in trustworthy LLMs

#109

Post by ubutler »

plagiarist wrote: post #89
hu3 wrote: post #38

> we found that GPT-5.2 cannot even compute the parity of a short string like 11000, and GPT-5.2 cannot determine whether the parentheses in ((((()))))) are balanced. I think there is a valid insight here which many already know: LLMs are much more reliable at creating scripts and automation to do certain tasks than doing these tasks themselves. For example if I provide an LLM my database schema and tell it to scan f…

The dream is probably that the inference software then writes and executes that script without using text generation alone. Analog to how a human might cross off pairs of parentheses to check that example.

ChatGPT already does this, albeit in limited circumstances, through the use of its sandbox environment. Asking GPT in thinking mode to, for example, count the number of “l”s in a long text may see it run a Python script to do so.

There’s a massive issue with extrapolating to more complex tasks, however, where either you run the risk of prompt injection via granting your agent access to the internet or, more commonly, an exponential degradation in coherence over long contexts.

Re: The case for zero-error horizons in trustworthy LLMs

#110

Post by dwa3592 »

orbital-decay wrote: post #67
dwa3592 wrote: post #49

Earlier quoted context omitted.

but it's a tricky question for LLMs; it shows that if it's not in the training set; LLMs could trip which kinda shows that the intelligence is not generalized yet. I tried this with gemini - (i am trying(something(re(a(l(ly)c)r)a)z)((y)he)re) and it tripped.

Intuitively this looks like an architectural artifact (like optical illusions in humans) or a natural property of learning rather than a lack of generalization. I have issues with your example too and have to count slowly to make sure.

Right, I am sure you were able to solve it albeit slowly- you knew you had to do it slow. LLMs which are mathematicians don't know that and can't seem to understand that they need to do it slowly.
Post reply on HN