Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI agent benchmarks are broken

ddkang.substack.com

Post reply on HN
Print view View original

51–60 of 88 posts

Re: AI agent benchmarks are broken

#51

Post by AIPedant »

DonHopkins wrote: post #40
jerf wrote: post #8

When I was being a bad HN reader and just reacting to the title, my initial impulse was to be placating, and observe that they are probably just immature. After all, for all that has happened, this is still only a couple year's worth of development, and it does tend to take a long time to develop good benchmarks. However the article does seem to be pointing out some fundamental issues. I'm particularly annoyed by usi…

It's like using steel to produce steel. What else are you going to use? Bamboo?

It's more like using a faulty and dangerous automated foundry to make steel when you could just hire steelworkers.

That's the real problem here - these companies are swimming in money and have armies of humans working around the clock training LLMs, there is no honest reason to nickel-and-dime the actual evaluation of benchmarks. It's like OpenAI using exact text search to identify benchmark contamination for the GPT-4 technical report. I am quite certain they had more sophisticated tools available.

Re: AI agent benchmarks are broken

#52

Post by hddbbdbfnfdk »

greatpostman wrote: post #7

Benchmarks aren’t broken, the models can learn anything. If we give them true real world data (physics engine), they will learn the real world. We are going to see artificial general intelligence in our lifetime

more like in the next two weeks methinks

Re: AI agent benchmarks are broken

#54

Post by diggan »

alextheparrot wrote: post #16
jerf wrote: post #8

When I was being a bad HN reader and just reacting to the title, my initial impulse was to be placating, and observe that they are probably just immature. After all, for all that has happened, this is still only a couple year's worth of development, and it does tend to take a long time to develop good benchmarks. However the article does seem to be pointing out some fundamental issues. I'm particularly annoyed by usi…

LLMs evaluating LLM outputs really isn’t that dire… Discriminating good answers is easier than generating them. Good evaluations write test sets for the discriminators to show when this is or isn’t true. Evaluating the outputs as the user might see them are more representative than having your generator do multiple tasks (e.g. solve a math query and format the output as a multiple choice answer). Also, human labels a…

> Discriminating good answers is easier than generating them.

Lots of other good replies to this specific part, but also, lots of developers are struggling with the feeling that reviewing code is harder than writing code (something I personally not sure I agree with), seen that sentiment being shared here on HN a lot, and would directly go against that particular idea.

Re: AI agent benchmarks are broken

#55

Post by sdenton4 »

ttoinou wrote: post #24
sdenton4 wrote: post #17

Earlier quoted context omitted.

When I was working in audio compression, evaluation was very painful because we had no programmatic way to measure how good some reconstructed audio sounds to a human. Any metric you could come up with was gameable, and direct optimization would lead to artifacts. As a result, we always had a two-step evaluation process. We would use a suite of metrics to guide development progress (validation), but the final evaluat…

Wouldn't that process avoid you finding a better subjective audio codec that doesn't reduce typical metrics (PSNR etc.) ? another process would rather be to first construct a metric software that tries to be similar to the subjective experience of humans, then use that to create audio codecs optimizing this metric

There's two answers to that....

The first is, how do you know the subjective optimization your making is actually any good? You're just moving the problem back one layer of abstraction.

The second is, we did that, eventually, by training models to predict subjective listening scores from the giant pile of subjective test data we had collected over the years. (ViSQoL) It's great, but we still don't trust it for end-of-the-day, cross codec comparison, because we don't want to reward overfit on the trained model.

https://arxiv.org/abs/2004.09584

Re: AI agent benchmarks are broken

#56

Post by jgraettinger1 »

majormajor wrote: post #44
alextheparrot wrote: post #16

Earlier quoted context omitted.

LLMs evaluating LLM outputs really isn’t that dire… Discriminating good answers is easier than generating them. Good evaluations write test sets for the discriminators to show when this is or isn’t true. Evaluating the outputs as the user might see them are more representative than having your generator do multiple tasks (e.g. solve a math query and format the output as a multiple choice answer). Also, human labels a…

> Discriminating good answers is easier than generating them. I don't think this is true for many fields - especially outside of math/programming. Let's say the task is "find the ten most promising energy startups in Europe." (This is essentially the sort of work I see people frequently talk about using research modes of models for here or on LinkedIn.) In ye olden days pre-LLM you'd be able to easily filter out a bu…

> You can't do that for LLM output.

That's true if you're just evaluating the final answer. However, wouldn't you evaluate the context -- including internal tokens -- built by the LLM under test ?

In essence, the evaluator's job isn't to do separate fact-finding, but to evaluate whether the under-test LLM made good decisions given the facts at hand.

Re: AI agent benchmarks are broken

#57

Post by lcnPylGDnU4H9OF »

yifanl wrote: post #9
xnx wrote: post #5

All benchmarks are flawed. Some benchmarks are useful.

Here's a third sentence fragment: These benchmarks are not.

Just want to nit: none of those are sentence fragments, they are complete thoughts with a subject and a predicate. Yours kinda comes close to being a fragment but it really just omits what "are not" (the predicate) is referring to, which is included in prior context.

For example, a fragment with a missing predicate.

Re: AI agent benchmarks are broken

#58

Post by yeahyeahok »

RansomStark wrote: post #3

I really like the CMU Agents Company approach of simulating a real world environment [0]. Is it perfect, no. Does it show you want to expect in production, not really, but it's much closer than anything else I've seen. [0] https://the-agent-company.com/

Damn. Super bullish on CMU. Somehow, they seem routinely left out of the top CS schools discussion at least in mainstream discourse: MIT, Stanford, Cal, .... Seen a disproportionate amount of stellar research come from there. Also, interestingly, I have met really incompetent people from all the other top 3 schools but am yet to meet an incompetent CMU SCS alum -- wtf are they feeding them in pitsburgh??

Re: AI agent benchmarks are broken

#59

Post by ttoinou »

sdenton4 wrote: post #55
ttoinou wrote: post #24

Earlier quoted context omitted.

Wouldn't that process avoid you finding a better subjective audio codec that doesn't reduce typical metrics (PSNR etc.) ? another process would rather be to first construct a metric software that tries to be similar to the subjective experience of humans, then use that to create audio codecs optimizing this metric

There's two answers to that.... The first is, how do you know the subjective optimization your making is actually any good? You're just moving the problem back one layer of abstraction. The second is, we did that, eventually, by training models to predict subjective listening scores from the giant pile of subjective test data we had collected over the years. (ViSQoL) It's great, but we still don't trust it for end-of…

Nice

Well yeah you would still need human testing

Post reply on HN