Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study identifies weaknesses in how AI systems are evaluated

oii.ox.ac.uk

Post reply on HN
Print view View original

111–120 of 204 posts

Re: Study identifies weaknesses in how AI systems are evaluated

#111

Post by hamdingers »

shanev wrote: post #17

This is solvable at the level of an individual developer. Write your own benchmark for code problems that you've solved. Verify tests pass and that it satisfies your metrics like tok/s and TTFT. Create a harness that works with API keys or local models (if you're going that route).

At the developer level all my LLM use is in the context of agentic wrappers, so my benchmark is fairly trivial:

Configure aider or claude code to use the new model, try to do some work. The benchmark is pass/fail, if after a little while I feel the performance is better than the last model I was using it's a pass, otherwise it's a fail and I go back.

Building your own evaluations makes sense if you're serving an LLM up to customers and want to know how it performs, but if you are the user... use it and see how it goes. It's all subjective anyway.

Re: Study identifies weaknesses in how AI systems are evaluated

#112

Post by daveguy »

vntok wrote: post #83
daveguy wrote: post #60

Earlier quoted context omitted.

I don't think it should be completely open ended. I mean, you could have an "ask_hooman" tool that solves a ton of problems with current LLMs. But that doesn't mean the LLM is capable with respect to the benchmark.

Why not? One of the most intelligent things to do when stuck on a problem is to get outside help. If allowing this behaviour raises a problem, you can always add constraints to the benchmark such as "final answer must come out under 15s" or something. The LLM can then make the decision to ask around in accordance to the time risk.

Because AI are good at devolving to the highest score, regardless of test intent. For most problems "ask_hooman", or especially the plural, would be much more effective. So, the degenerate case would dominate and tell you precisely zero about the intelligence of the AI. If a specific "tool" is more adept than the "AI" then "choose tool" will always be the correct answer. But I agree, a tight time constraint would help.

Re: Study identifies weaknesses in how AI systems are evaluated

#113

Post by embedding-shape »

atn34 wrote: post #104
riskable wrote: post #85

We should make a collective git repo full of every kind of annoying bug we (expert developers) can think of. Then use that to benchmark LLMs. Someone want to start? I've got a Yjs/CRDT collaborative editing bug that took like a week and a half of attempts with Claude Code (Sonnet 4.5), GPT5-codex (medium), and GLM-4.6 many, many attempts to figure out. Even then they didn't really get it... Just came up with a succes…

I actually started a collection of annoying bugs I’ve seen in the wild. I give the llm the buggy implementation and ask it to write a test that catches it. So far not even a frontier model (Claude Sonnet) can do it, even though they can find and fix the bug itself.

> even a frontier model (Claude Sonnet) can do it

Probably because Sonnet is no longer a frontier model, it isn't even the best model Anthropic offers, according to themselves.

Re: Study identifies weaknesses in how AI systems are evaluated

#114

Post by simonw »

kvirani wrote: post #96
simonw wrote: post #91

Earlier quoted context omitted.

Have you tried the Playwright libraries? Not the MCP, instead telling Claude Code to use the Node.js or Python Playwright libraries directly. I have had some really good results for this for gnarly frontend challenges.

Curious why not the MCP? I use that

I don't really like MCPs, at least when I'm working with coding agents like Claude Code or Codex CLI. I'd rather let the agents write code that can do anything the underlying library is capable of, rather than restricting them to just the functionality that the MCP exposes.

It's more token efficient too since I don't need to load the full MCP description into my context.

Re: Study identifies weaknesses in how AI systems are evaluated

#115

Post by embedding-shape »

hamdingers wrote: post #111
shanev wrote: post #17

This is solvable at the level of an individual developer. Write your own benchmark for code problems that you've solved. Verify tests pass and that it satisfies your metrics like tok/s and TTFT. Create a harness that works with API keys or local models (if you're going that route).

At the developer level all my LLM use is in the context of agentic wrappers, so my benchmark is fairly trivial: Configure aider or claude code to use the new model, try to do some work. The benchmark is pass/fail, if after a little while I feel the performance is better than the last model I was using it's a pass, otherwise it's a fail and I go back. Building your own evaluations makes sense if you're serving an LLM…

> Building your own evaluations makes sense if you're serving an LLM up to customers and want to know how it performs, but if you are the user... use it and see how it goes. It's all subjective anyway.

I'd really caution against this approach, mainly because humans suck at removing emotions and other "human" factors when judging how well something works, but also because comparing across models gets a lot easier when you can see 77/100 vs 91/100 as a percentage score, over your own tasks that you actually use the LLMs for. Just don't share this benchmark publicly once you're using it for measurements.

Re: Study identifies weaknesses in how AI systems are evaluated

#116

Post by ghurtado »

gishh wrote: post #79
anamexis wrote: post #68

Earlier quoted context omitted.

Computer vision algorithms run on computers and they can’t do basic arithmetic. My email client runs on my computer and it doesn’t do basic arithmetic either. Something running on a computer does not imply that it can or should do basic arithmetic

Pretty sure the only thing computer vision does is math. I’ve also observed email clients tallying the number of unread emails I have. It’s quite obnoxious actually, but I qualify adding as math.

> Pretty sure the only thing computer vision does is math.

That is only marginally less pedantic than saying that the only thing computer vision does is run discrete electrical signals through billions of transistors.

Re: Study identifies weaknesses in how AI systems are evaluated

#117

Post by embedding-shape »

SkyPuncher wrote: post #46

Benchmarks are nothing more than highly contextual specs (in traditional code). They demonstrate your code works in a certain way in certain use cases, but they do not prove your code works as expected in all use cases.

> Program testing can be used to show the presence of bugs, but never to show their absence. Edsger W. Dijkstra

Maybe we need something similar for benchmarks, and updated for today's LLMs, like:

> LLM benchmarks can be used to show what tasks they can do, but never to show what tasks they cannot.

Re: Study identifies weaknesses in how AI systems are evaluated

#118

Post by purple_turtle »

jennyholzer wrote: post #5

I've been getting flagged by high-on-their-own-supply AI boosters for identifying that LLM benchmarks have been obvious bullshit for at least the last year and a half. What changed to make "the inevitable AI bubble" the dominant narrative in last week or so?

It is possible to be right on the main theme but only by accident (with arguments and claims being wrong), communicating in a highly faulty way, with pointless insults, doing it in offtopic derails, being correct on minor point while being mostly wrong etc.

Can you link some of these comments you consider useful but got flagged?

Re: Study identifies weaknesses in how AI systems are evaluated

#119

Post by j45 »

shanev wrote: post #17

This is solvable at the level of an individual developer. Write your own benchmark for code problems that you've solved. Verify tests pass and that it satisfies your metrics like tok/s and TTFT. Create a harness that works with API keys or local models (if you're going that route).

We have to keep in mind that "solving" might mean having the LLM recognize the pattern of solving something.

Re: Study identifies weaknesses in how AI systems are evaluated

#120

Post by j45 »

bubblelicious wrote: post #4

I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…

What gets measured, gets managed and improved, though.
Post reply on HN