Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent benchmarks are broken

ddkang.substack.com

Post reply on HN
Print view View original

71–80 of 88 posts

Re: AI agent benchmarks are broken

#71

Post by alextheparrot »

diggan wrote: post #54
alextheparrot wrote: post #16

Earlier quoted context omitted.

LLMs evaluating LLM outputs really isn’t that dire… Discriminating good answers is easier than generating them. Good evaluations write test sets for the discriminators to show when this is or isn’t true. Evaluating the outputs as the user might see them are more representative than having your generator do multiple tasks (e.g. solve a math query and format the output as a multiple choice answer). Also, human labels a…

> Discriminating good answers is easier than generating them. Lots of other good replies to this specific part, but also, lots of developers are struggling with the feeling that reviewing code is harder than writing code (something I personally not sure I agree with), seen that sentiment being shared here on HN a lot, and would directly go against that particular idea.

I wish the other replies and this would engage with the sentence right after it indicating that you should test this premise empirically.

Re: AI agent benchmarks are broken

#72

Post by alextheparrot »

majormajor wrote: post #44
alextheparrot wrote: post #16

Earlier quoted context omitted.

LLMs evaluating LLM outputs really isn’t that dire… Discriminating good answers is easier than generating them. Good evaluations write test sets for the discriminators to show when this is or isn’t true. Evaluating the outputs as the user might see them are more representative than having your generator do multiple tasks (e.g. solve a math query and format the output as a multiple choice answer). Also, human labels a…

> Discriminating good answers is easier than generating them. I don't think this is true for many fields - especially outside of math/programming. Let's say the task is "find the ten most promising energy startups in Europe." (This is essentially the sort of work I see people frequently talk about using research modes of models for here or on LinkedIn.) In ye olden days pre-LLM you'd be able to easily filter out a bu…

> Good evaluations write test sets for the discriminators to show when this is or isn’t true.

If they can’t write an evaluation for the discriminator I agree. All the input data issues you highlight also apply to generators.

Re: AI agent benchmarks are broken

#73

Post by alextheparrot »

tempfile wrote: post #48
alextheparrot wrote: post #16

Earlier quoted context omitted.

LLMs evaluating LLM outputs really isn’t that dire… Discriminating good answers is easier than generating them. Good evaluations write test sets for the discriminators to show when this is or isn’t true. Evaluating the outputs as the user might see them are more representative than having your generator do multiple tasks (e.g. solve a math query and format the output as a multiple choice answer). Also, human labels a…

> Discriminating good answers is easier than generating them. This is actually very wrong. Consider for instance the fact that people who grade your tests in school are typically more talented, capable, trained than the people taking the test. This is true even when an answer key exists. > Also, human labels are good but have problems of their own, Granted, but... > it isn’t like by using a “different intelligence ar…

It isn’t actually very wrong. Your example is tangential as graders in school have multiple roles — teaching the content and grading. That’s an implementation detail, not a counter to the premise.

I don’t think we should assume answering a test would be easy for a Scantron machine just because it is very good at grading them, either.

Re: AI agent benchmarks are broken

#74

Post by rybosworld »

Based on the comments, I think a lot of people are missing what the AI Agent actually got wrong here. Nowhere did the agent claim that 45 + 8 = 63.

You can see the Agent's step by step thought process here (also linked in the article):

https://ibm-cuga.19pc1vtv090u.us-east.codeengine.appdomain.c...

The Agent correctly entered the starting point (MIT) and the ending point (Harvard) and the mode of transport (on foot). OpenStreetMap returns this as taking 45 minutes long.

Then the agent reversed the directions, and changed the mode of transport to car. What it should have also done, is change the destination to Logan Airport. This is the part that the agent missed. OpenStreetMap then returns that the drive from Harvard to MIT takes 8 minutes.

The agent then returned the answer as being 45 minutes walking and 8 minutes driving. The first number is correct. The second is wrong because the agent chose the wrong destination, not because it did math incorrectly.

Seems like lots of readers are chomping at the bit to prove how stupid the models are rather than focus on the real problem the author is highlighting.

Re: AI agent benchmarks are broken

#75

Post by nerevarthelame »

KTibow wrote: post #53

This is more or less a funnel to their Agentic Benchmark Checklist: https://arxiv.org/abs/2507.02825

Finally, a benchmark for benchmarks. And what's great is that they already benchmarked their benchmark benchmark.

(Apologies for the benchmark snark. I'm glad people are doing this research, thanks for sharing it.)

Re: AI agent benchmarks are broken

#76

Post by asadotzler »

rybosworld wrote: post #74

Based on the comments, I think a lot of people are missing what the AI Agent actually got wrong here. Nowhere did the agent claim that 45 + 8 = 63. You can see the Agent's step by step thought process here (also linked in the article): https://ibm-cuga.19pc1vtv090u.us-east.codeengine.appdomain.c... The Agent correctly entered the starting point (MIT) and the ending point (Harvard) and the mode of transport (on foot).…

"champing"

Re: AI agent benchmarks are broken

#77

Post by th0ma5 »

bwfan123 wrote: post #6
deepdarkforest wrote: post #4

It's very funny how many layers of abstraction we are going through. We have limited understanding of how LLM's work exactly and why. We now do post training with RL, which again, we don't have perfect understanding of it either. Then you stack LLMs calls and random tools, and you have agents, and you are attempting to benchmark those . (and this exclude voice, computer use agents etc). It's all just vibes,there is n…

I can hear echos of an earlier era. There was yahoo-pipes and web-services frameworks which rhyme with MCP and agentic.

Pipes and services in general are reliable but the issues were social and economic. Getting everyone to agree was seen as a great way to poach users and give up control, plus the usual problems with open world vs. closed world assumptions. Thanks for mentioning this!

Re: AI agent benchmarks are broken

#78

Post by th0ma5 »

ttoinou wrote: post #23

What makes LLMs amazing (fuzzy input, fuzzy output) is exactly why they are hard to benchmark. If they could be benchmarked easily, they wouldn't be powerful by definition. I have no idea what's going on in the minds of people benchmarking LLMs for fuzzy tasks, and in the minds of people relying on benchmarks to make decisions about LLMs, I never looked at them. People doing benchmarks have to prove what they do is u…

Since when do people like the fuzziness of outputs? I think you make an interesting point but it also seems to imply that benchmarking will never truly be possible, which I think is true unless we can also make them observable which also as you say gives up the mystique.

Re: AI agent benchmarks are broken

#79

Post by suddenlybananas »

rybosworld wrote: post #74

Based on the comments, I think a lot of people are missing what the AI Agent actually got wrong here. Nowhere did the agent claim that 45 + 8 = 63. You can see the Agent's step by step thought process here (also linked in the article): https://ibm-cuga.19pc1vtv090u.us-east.codeengine.appdomain.c... The Agent correctly entered the starting point (MIT) and the ending point (Harvard) and the mode of transport (on foot).…

The model's scoring was done by another model though no? That was the source of the answer being mislabed as correct. So a different model thought that 45+8=63.

Re: AI agent benchmarks are broken

#80

Post by btdmaster »

There is a cool solution for this: https://huggingface.co/spaces/Jellyfish042/UncheatableEval

This doesn't work for instruction-tuned models, but it's an interesting alternative approach that doesn't need a complicated (and thus gameable) evaluation function or human interaction. Instead, predict the next word with data newer than the training set.

Post reply on HN