Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The case for zero-error horizons in trustworthy LLMs

arxiv.org

Post reply on HN
Print view View original

81–90 of 121 posts

Re: The case for zero-error horizons in trustworthy LLMs

#81

Post by simianwords »

dontlikeyoueith wrote: post #5
throwuxiytayq wrote: post #4

> This is surprising given the excellent capabilities of GPT-5.2. Is this seriously surprising to anyone who knows the absolute minimum about how LLMs parse and understand text?

Nope. It's only surprising to people who still think they're going to build God out of LLMs.

It was surprising to me and when I reviewed the paper, I found serious flaws that calls the fundamental claims into question - they didn't use any reasoning tokens. Any LLM or human will fail at a task like this if not allowed to think.

Re: The case for zero-error horizons in trustworthy LLMs

#82

Post by chromacity »

simianwords wrote: post #53
pton_xd wrote: post #29

Earlier quoted context omitted.

"in this paper we primarily evaluate the LLM itself without external tool calls." Maybe this is a factor?

No tools were used.

IIRC, web chat often uses tools / code without surfacing this information in any obvious way.

Re: The case for zero-error horizons in trustworthy LLMs

#83

Post by TeMPOraL »

emp17344 wrote: post #50
azakai wrote: post #40

Earlier quoted context omitted.

It has "outsourced" it to another component, sure, but does that matter? What the user sees is the total behavior of the entire system, not whether the system has internal divisions and separations.

It matters if you’re curious about whether AGI is possible. Have we really built “thinking machines”, or are these systems just elaborate harnesses that leverage the non-deterministic nature of LLMs?

What is a difference? If the "elaborate harness" consists of mix of "classical" code and ML model invocations, at which point it's disqualified from consideration for "thinking machine"? Best we can tell, even our brains have parts that are "dumb", interfacing with the parts that we consider "where the magic happens".

Re: The case for zero-error horizons in trustworthy LLMs

#84

Post by emp17344 »

dang wrote: post #80
emp17344 wrote: post #28

[flagged]

Please don't start generic flamewars on HN or impugn people who take an opposing view to yours. Both these vectors lead to tedious, unenlightening threads. There's plenty of rage to go around on literally every divisive topic, and it's not the place we want discussions to come from here. " Eschew flamebait. Avoid generic tangents. " " Comments should get more thoughtful and substantive, not less, as a topic gets more…

There are other users in this very thread using inflammatory language to attack this paper and those who find the paper compelling. One user says, quote: “You just can't reason with the anti-LLM group.”

In light of this, why was my comment - which was in large part a reaction to the behavior of the users described above - the only one called out here?

Re: The case for zero-error horizons in trustworthy LLMs

#85

Post by somethingweird »

BugsJustFindMe wrote: post #68
adrian_b wrote: post #55

Earlier quoted context omitted.

Most animals can distinguish bigger from smaller. However many animals can distinguish independently small numbers, like 3 or 5, and recognize them whenever they see them. So in this respect, there is little difference between humans and many animals. Humans learn to count to arbitrarily big numbers, but they can still easily recognize only small numbers.

> many animals can distinguish independently small numbers, like 3 or 5 This is called subitizing. It's distinct from counting. We can see the difference in humans with Simultanagnosia, who are unable to count beyond the subitizing range. Subitizing is categorizing the scale of a small gestalt group. The only thing I've ever seen where an animal appeared to demonstrate counting (up to 3) without training was in rhesu…

Even parrots can count to 6 and more, I would be surprised if primates couldn't.

Re: The case for zero-error horizons in trustworthy LLMs

#86

Post by Lerc »

wahnfrieden wrote: post #72
Lerc wrote: post #48

Earlier quoted context omitted.

The r's in strawberry presents a different level of task to what people imagine. It seems trivial to a naive observer because the answer is easily derivable from the question without extra knowledge. A more accurate analogy for humans would be to imagine if every word had a colour. You are told that there are also a sequence of different colours that correspond to the same colour as that word. You are even given a bo…

That’s what makes it a fair evaluation and something that requires improvement. We shouldn’t only evaluate agent skills by what is most commonly represented in training data. We expect performance from them on areas that existing training data may be deficient at providing. You don’t need to invent an absurdity to find these cases.

It's reasonable to test their ability to do this, and it's worth working to make it better.

The issue is that people claim the performance is representative of a human's performance in the same situation. That gives an incorrect overall estimation of ability.

Re: The case for zero-error horizons in trustworthy LLMs

#87

Post by Chobilet »

simianwords wrote: post #74

This paper is complete nonsense. The specific prompt they used doesn’t specify reasoning effort. Which defaults to none. { "model": "gpt-5.2-2025-12-11", "instructions": "Is the parentheses string balanced? Answer with only Yes or No.", "input": "((((())))))", "temperature": 0 } > Lower reasoning effort The reasoning.effort parameter controls how many reasoning tokens the model generates before producing a response.…

I'm sure this comment was made in good faith, but most researchers would rightfully understand these intricacies, and this is likely intentional(as noted in the paper). At a quick glance, I cannot say whether or not the paper has been peer reviewed(though unlikely/in process given how recent it was published). In general, you'd find published papers also listed in a specific journal/conference(i.e. not just the archives which anyone can submit to).

Additionally, many of us in the field of researching LLM's are curious to understanding the boundaries and limitations of what is capable. This paper isn't really meant as any sort of "gotcha", rather serve as a possible basis point for future work. Though with a caveat I'm still digesting the paper myself.

Re: The case for zero-error horizons in trustworthy LLMs

#88

Post by stronglikedan »

grey-area wrote: post #22

To those saying this is not surprising, yes it will be surprising to the general public who are being served ads from huge companies like MS or OpenAI saying LLMs can help with their accounting, help them close deals by crunching the numbers in seconds, write complex code for them etc etc. This is important information for anyone to understand who thinks these systems are thinking, reasoning, and learning from them o…

> general public

and the C-suite

Re: The case for zero-error horizons in trustworthy LLMs

#89

Post by plagiarist »

hu3 wrote: post #38

> we found that GPT-5.2 cannot even compute the parity of a short string like 11000, and GPT-5.2 cannot determine whether the parentheses in ((((()))))) are balanced. I think there is a valid insight here which many already know: LLMs are much more reliable at creating scripts and automation to do certain tasks than doing these tasks themselves. For example if I provide an LLM my database schema and tell it to scan f…

The dream is probably that the inference software then writes and executes that script without using text generation alone. Analog to how a human might cross off pairs of parentheses to check that example.

Re: The case for zero-error horizons in trustworthy LLMs

#90

Post by simianwords »

Chobilet wrote: post #87
simianwords wrote: post #74

This paper is complete nonsense. The specific prompt they used doesn’t specify reasoning effort. Which defaults to none. { "model": "gpt-5.2-2025-12-11", "instructions": "Is the parentheses string balanced? Answer with only Yes or No.", "input": "((((())))))", "temperature": 0 } > Lower reasoning effort The reasoning.effort parameter controls how many reasoning tokens the model generates before producing a response.…

I'm sure this comment was made in good faith, but most researchers would rightfully understand these intricacies, and this is likely intentional(as noted in the paper). At a quick glance, I cannot say whether or not the paper has been peer reviewed(though unlikely/in process given how recent it was published). In general, you'd find published papers also listed in a specific journal/conference(i.e. not just the archi…

I'm asking, why use a thinking model without allowing it to reason? No one uses it in that way..

>While LLMs appear extremely intelligent and capable of reasoning, they sometimes make mistakes that seem inconceivably foolish from a human perspective. For example, GPT-5.2 can implement complex fluid dynamics simulation code, yet it cannot even compute the parity of the short string 11000, cannot determine whether the parentheses in ((((()))))) are balanced, and makes calculation errors on 127 × 82 (Figure 1).

Why would they say it is capable of reasoning and then not allow it to reason in the experiment?

Post reply on HN