Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study identifies weaknesses in how AI systems are evaluated

oii.ox.ac.uk

Post reply on HN
Print view View original

81–90 of 204 posts

Re: Study identifies weaknesses in how AI systems are evaluated

#81

Post by cavisne »

pahae wrote: post #48

I wish the big providers would offer some sort of trial period where you can evaluate models in a _realistic_ setting yourself (i.e cli tools or IDE integrations). I wouldn't even mind strict limits -- just give me two hours or so of usage and I'd already be happy. Seriously. My use-case is probably pretty far from the usual tasks: I'm currently implementing a full observability platform based on VictoriaMetrics / Vi…

https://wuu73.org/blog/aiguide1.html

You can get a lot of free usage out of the models.

Re: Study identifies weaknesses in how AI systems are evaluated

#83

Post by vntok »

daveguy wrote: post #60
LadyCailin wrote: post #35

Earlier quoted context omitted.

I’d say it’s fair for LLMs to be able to use any tool in benchmarks, so long as they are the ones to decide to use them.

I don't think it should be completely open ended. I mean, you could have an "ask_hooman" tool that solves a ton of problems with current LLMs. But that doesn't mean the LLM is capable with respect to the benchmark.

Why not? One of the most intelligent things to do when stuck on a problem is to get outside help.

If allowing this behaviour raises a problem, you can always add constraints to the benchmark such as "final answer must come out under 15s" or something. The LLM can then make the decision to ask around in accordance to the time risk.

Re: Study identifies weaknesses in how AI systems are evaluated

#84

Post by tim333 »

pahae wrote: post #48

I wish the big providers would offer some sort of trial period where you can evaluate models in a _realistic_ setting yourself (i.e cli tools or IDE integrations). I wouldn't even mind strict limits -- just give me two hours or so of usage and I'd already be happy. Seriously. My use-case is probably pretty far from the usual tasks: I'm currently implementing a full observability platform based on VictoriaMetrics / Vi…

There's the github copilot 30 day trial? "Access to Anthropic Claude Sonnet 4, GPT-5, Gemini 2.5 Pro, and more 300 premium requests to use the latest models and code review"

Re: Study identifies weaknesses in how AI systems are evaluated

#85

Post by riskable »

We should make a collective git repo full of every kind of annoying bug we (expert developers) can think of. Then use that to benchmark LLMs.

Someone want to start? I've got a Yjs/CRDT collaborative editing bug that took like a week and a half of attempts with Claude Code (Sonnet 4.5), GPT5-codex (medium), and GLM-4.6 many, many attempts to figure out. Even then they didn't really get it... Just came up with a successful workaround (which is good enough for me but still...).

Aside: You know what really moved the progress bar on finding and fixing the bug? When I had a moment of inspiration and made the frontend send all it's logs to the backend so the AIs could see what was actually happening on the frontend (near real-time). Really, I was just getting sick of manual testing and pasting the console output into the chat (LOL). Laziness FTW!

I have the Google Chrome Dev Tools MCP but for some reason it doesn't work as well :shrug:

Re: Study identifies weaknesses in how AI systems are evaluated

#86

Post by ambicapter »

singron wrote: post #53
ambicapter wrote: post #42

Earlier quoted context omitted.

> Since performance on large numbers is not what these exams are intended to test for, How so? Isn't the point of these exams to test arithmetic skills? I would hope we'd like arithmetic skills to be at a constant level regardless of the size of the number?

No. AIME is a test for advanced high schoolers that mostly tests higher level math concepts like algebra and combinatorics. The arithmetic required is basic. All the answers are 3-digit numbers so that judging is objective and automated while making guessing infeasible. You have 12 minutes on average for each question, so even if you are terribly slow at arithmetic, you should still be able to calculate the correct a…

That's probably a great test for high schoolers but it doesn't really test what we want from AI, no? I would expect AI to be limited by the far greater constraints of its computing ability, and not the working memory of a human high schooler.

Re: Study identifies weaknesses in how AI systems are evaluated

#87

Post by riskable »

yunyu wrote: post #66
6510 wrote: post #64

Earlier quoted context omitted.

I don't claim to know anything but I thought tool usage was a major sign of intelligence. For example floats are a wonderful technology but people use them as if chainsaws are great for cutting bread and butter. We now have entire languages that cant do basic arithmetic. I thought it was alarming: People it cant compute like this! Now we have language models, those are still computers, why cant we just give them.. yo…

We do? Tool use started coming in vogue around 2023

Actually, tool use started coming into vogue around 3.3 million years ago.

Re: Study identifies weaknesses in how AI systems are evaluated

#88

Post by idontpost »

bee_rider wrote: post #23

> "For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle." When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much mo…

[dead]

Re: Study identifies weaknesses in how AI systems are evaluated

#89

Post by idontpost »

layer8 wrote: post #25
bee_rider wrote: post #23

> "For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle." When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much mo…

I don’t think the fact that LLMs can handle small numbers more reliably has anything to do with their reasoning ability. To the contrary, reasoning ability should enable them to handle numbers of arbitrary size, just as it enables humans to do so, given some pencil and paper. However: > Testing only on these problems would not predict performance on larger numbers, where LLMs struggle. Since performance on large numb…

[dead]

Re: Study identifies weaknesses in how AI systems are evaluated

#90

Post by riskable »

Forgeties79 wrote: post #29
bee_rider wrote: post #23

> "For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle." When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much mo…

>the point of these LLMs is to do things that computers were bad at. The way they’re being deployed it feels like the point of LLMs is largely to replace basic online search or to run your online customer support cheaply. I’m a bit out on a limb here because this is not really my technical expertise by any stretch of the imagination, but it seems to me these benchmark tests don’t really tell us much about how LLM’s p…

Nobody really knows "the point" of LLMs yet. They weren't even "invented" as much as they emerged as a trick to get computers to better understand human language.

They're still brand spanking new and everyone's trying to figure out how to best use them. We don't even really know if they're ever going to be "really good at" any given task!

Are they "really good at" these things or are they merely "OK-ish"?

    * Answering factual questions.
    * Programming.
    * Understanding what the user wants from natural language. 
    * Searching/recommending stuff.
Real world testing suggests that with billions and billions of dollars spent, you really can get an LLM to be "OK-ish" at all those things :D
Post reply on HN