Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study identifies weaknesses in how AI systems are evaluated

oii.ox.ac.uk

Post reply on HN
Print view View original

71–80 of 204 posts

Re: Study identifies weaknesses in how AI systems are evaluated

#71

Post by Dylan16807 »

zamadatix wrote: post #33
layer8 wrote: post #25

Earlier quoted context omitted.

I don’t think the fact that LLMs can handle small numbers more reliably has anything to do with their reasoning ability. To the contrary, reasoning ability should enable them to handle numbers of arbitrary size, just as it enables humans to do so, given some pencil and paper. However: > Testing only on these problems would not predict performance on larger numbers, where LLMs struggle. Since performance on large numb…

Pencil and paper is just testing with tools enabled.

On some level this makes sense, but on the other hand LLMs already have perfect recall of thousands of symbols built into them, which is what pencil and paper gives to a human test taker.

Re: Study identifies weaknesses in how AI systems are evaluated

#72

Post by novok »

bee_rider wrote: post #23

> "For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle." When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much mo…

IMO I think the calculator problem goes away with tool use or NN architectures that basically add a calculator equivalent as one of the potential 'experts' or similar. It won't be much of a trope for longer.

Re: Study identifies weaknesses in how AI systems are evaluated

#73

Post by novok »

luke0016 wrote: post #39
layer8 wrote: post #25

Earlier quoted context omitted.

I don’t think the fact that LLMs can handle small numbers more reliably has anything to do with their reasoning ability. To the contrary, reasoning ability should enable them to handle numbers of arbitrary size, just as it enables humans to do so, given some pencil and paper. However: > Testing only on these problems would not predict performance on larger numbers, where LLMs struggle. Since performance on large numb…

> reasoning ability should enable them to handle numbers of arbitrary size, just as it enables humans to do so, given some pencil and paper. Or given a calculator. Which it's running on. Which it in some sense is . There's something deeply ironic about the fact that we have an "AI" running on the most technologically advanced calculator in the history of mankind and...it can't do basic math.

This is like saying it's ironic that an alternator in a car cannot combust gasoline when the gasoline engine is right beside it, even though the alternator 'runs' on the gasoline engine.

Re: Study identifies weaknesses in how AI systems are evaluated

#74

Post by doctorpangloss »

The problem with the LLM benchmarks is that if you see one that shows high performance by something that isn’t from Anthropic, Google or OpenAI, you don’t believe it, even if it were “true.” In that sense, benchmarks are a holistic social experience in this domain, less a scientific endeavour.

Re: Study identifies weaknesses in how AI systems are evaluated

#75

Post by TheOtherHobbes »

anamexis wrote: post #68
rrr_oh_man wrote: post #65

Earlier quoted context omitted.

What's meaning of `computer`, remind me quick?

Computer vision algorithms run on computers and they can’t do basic arithmetic. My email client runs on my computer and it doesn’t do basic arithmetic either. Something running on a computer does not imply that it can or should do basic arithmetic

That's confusing basic arithmetic as a user feature and as an implementation requirement.

I guarantee that computer vision and email clients both use basic arithmetic in implementation. And it would be trivially easy to bolt a calculator into an email app, because the languages used to write email apps include math features.

That's not true of LLMs. There's math at the bottom of the stack. But LLMs run as a separate closed and opaque application of a unique and self-contained type, which isn't easily extensible.

They don't include hooks into math features on the GPUs, and there's no easy way to add hooks.

If you want math, you need a separate tool call to conventional code.

IMO testing LLMs as if they "should" be able to do arithmetic is bizarre. They can't. They're not designed to. And even if they did, they'd be ridiculously inefficient at it.

Re: Study identifies weaknesses in how AI systems are evaluated

#76

Post by throwup238 »

benjiro wrote: post #70
luke0016 wrote: post #39

Earlier quoted context omitted.

> reasoning ability should enable them to handle numbers of arbitrary size, just as it enables humans to do so, given some pencil and paper. Or given a calculator. Which it's running on. Which it in some sense is . There's something deeply ironic about the fact that we have an "AI" running on the most technologically advanced calculator in the history of mankind and...it can't do basic math.

Thing is, a LLM is nothing but a prediction algorithm based upon what it trained. So it missing basic calculator functionality is a given. This is why tool usage is more and more a thing for LLMs. So that the LLM can from itself use a calculator for the actual math parts it needs. Thus increasing accuracy ...

Why is it a given? The universal approximation theorem should apply since addition is a continuous function. Now whether the network is sufficiently trained for that is another question but I don’t think it's a given that a trillion parameter model can’t approximate the most basic math operations.

I think the tokenization is a bigger problem than the model itself.

Re: Study identifies weaknesses in how AI systems are evaluated

#77

Post by jvanderbot »

bee_rider wrote: post #23

> "For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle." When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much mo…

Absolutely not.

College exam takers use those tricks because they are on a time limit and are gaming the system. It's clever and wink wink nudge nudge ok everyone does it. But it's one tiny signal in a huge spectrum of things we use to evaluate people.

Instead, these metrics are gamed and presented as the entire multi special signal of competence for LLMs because it is literally impossible to say that success in one domain would translate the way it might with a good hire.

What I want is something I don't have to guard against gaming. Something conscientious and capable like my co workers. Until then it's google version 2 married to intellisense and I'm not letting do anything by itself.

Re: Study identifies weaknesses in how AI systems are evaluated

#78

Post by anamexis »

TheOtherHobbes wrote: post #75
anamexis wrote: post #68

Earlier quoted context omitted.

Computer vision algorithms run on computers and they can’t do basic arithmetic. My email client runs on my computer and it doesn’t do basic arithmetic either. Something running on a computer does not imply that it can or should do basic arithmetic

That's confusing basic arithmetic as a user feature and as an implementation requirement. I guarantee that computer vision and email clients both use basic arithmetic in implementation. And it would be trivially easy to bolt a calculator into an email app, because the languages used to write email apps include math features. That's not true of LLMs. There's math at the bottom of the stack. But LLMs run as a separate…

Yes, you are agreeing with me.

Re: Study identifies weaknesses in how AI systems are evaluated

#79

Post by gishh »

anamexis wrote: post #68
rrr_oh_man wrote: post #65

Earlier quoted context omitted.

What's meaning of `computer`, remind me quick?

Computer vision algorithms run on computers and they can’t do basic arithmetic. My email client runs on my computer and it doesn’t do basic arithmetic either. Something running on a computer does not imply that it can or should do basic arithmetic

Pretty sure the only thing computer vision does is math.

I’ve also observed email clients tallying the number of unread emails I have. It’s quite obnoxious actually, but I qualify adding as math.

Re: Study identifies weaknesses in how AI systems are evaluated

#80

Post by anamexis »

gishh wrote: post #79
anamexis wrote: post #68

Earlier quoted context omitted.

Computer vision algorithms run on computers and they can’t do basic arithmetic. My email client runs on my computer and it doesn’t do basic arithmetic either. Something running on a computer does not imply that it can or should do basic arithmetic

Pretty sure the only thing computer vision does is math. I’ve also observed email clients tallying the number of unread emails I have. It’s quite obnoxious actually, but I qualify adding as math.

Yes, everything that a computer does, it does using math. This does not imply that things running on the computer can do basic arithmetic tasks for the user.
Post reply on HN