Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Leaderboard Illusion

arxiv.org

Post reply on HN
Print view View original

31–40 of 56 posts

Re: The Leaderboard Illusion

#31

Post by n8m8 »

ekidd wrote: post #12

Also, I've been hearing a lot of complaints that Chatbot Arena tends to favor: - Lots of bullet points in every response. - Emoji. ...even at the expense of accurate answers. And I'm beginning to wonder if the sycophantic behavior of recent models ("That's a brilliant and profound idea") is also being driven by Arena scores. Perhaps LLM users actually do want lots of bullets, emoji and fawning praise. But this seems…

Interesting idea, I think I'm on board with this correlation hypothesis. Obviously it's complicated, but it does seems like over-reliance on arbitrary opinions from average people would result in valuing "feeling" over correctness.

Re: The Leaderboard Illusion

#33

Post by VladVladikoff »

StevenWaterman wrote: post #13
amelius wrote: post #4

Earlier quoted context omitted.

Are the benchmark prompts public and isn't that where the problem lies?

No, even if the benchmarks are private, it's still an issue. Because you can overfit to the benchmark by trying X random variations of the model, and picking the one that performs best on the benchmark It's similar to how I can pass any multiple-choice exam if you let me keep attempting it and tell me my overall score at the end of each attempt - even if you don't tell me which answers were right/wrong

Now I’m wondering what the most efficient algorithm to obtain a mark of 100% in the least amount of attempts. Guessing one question per attempt seems inefficient. Perhaps guessing the whole exam as option A. Then submitting the whole exam as option B. And so on, at the start, could give you a count of how many As are correct. Then maybe some sort of binary sort through the rest of the options? You could submit the first 1/2 as A and the second 1/2 as B. Etc. hmmmm

Re: The Leaderboard Illusion

#34

Post by __alexs »

ilrwbwrkhv wrote: post #18
pongogogo wrote: post #2

I think this is a really interesting paper from Cohere, it really feels that at this point in time you can't trust any public benchmark, and you really need your own private evals.

Yup in my private evals I have repeatedly found that DeepSeek has the best models for everything and yet in a lot of these public ones it always seems like someone else is on the top. I don't know why.

Publishing them might help you find out.

Re: The Leaderboard Illusion

#35

Post by refulgentis »

__alexs wrote: post #34
ilrwbwrkhv wrote: post #18

Earlier quoted context omitted.

Yup in my private evals I have repeatedly found that DeepSeek has the best models for everything and yet in a lot of these public ones it always seems like someone else is on the top. I don't know why.

Publishing them might help you find out.

^ This.

If I had to hazard a guess, as a poor soul doomed to maintain several closed and open source models acting agentically, I think you are hyper focused on chat trivia use cases (DeepSeek has a very, very, hard time tool calling and they say as much themselves in their API docs)

Re: The Leaderboard Illusion

#36

Post by amelius »

VladVladikoff wrote: post #33
StevenWaterman wrote: post #13

Earlier quoted context omitted.

No, even if the benchmarks are private, it's still an issue. Because you can overfit to the benchmark by trying X random variations of the model, and picking the one that performs best on the benchmark It's similar to how I can pass any multiple-choice exam if you let me keep attempting it and tell me my overall score at the end of each attempt - even if you don't tell me which answers were right/wrong

Now I’m wondering what the most efficient algorithm to obtain a mark of 100% in the least amount of attempts. Guessing one question per attempt seems inefficient. Perhaps guessing the whole exam as option A. Then submitting the whole exam as option B. And so on, at the start, could give you a count of how many As are correct. Then maybe some sort of binary sort through the rest of the options? You could submit the fi…

Maybe an llm can tell you how to best approach this problem ;)

Re: The Leaderboard Illusion

#37

Post by bob1029 »

> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

In context of genetic programming and other non-traditional ML techniques, I've been having difficulty attempting to locate a simple fitness function that reliably proxies natural language string similarity due to this effect.

For example, say you use something like common prefix length to measure how close a candidate's output string is to an objective string given an input string. The underlying learner will inevitably start doing things like repeating the input verbatim, especially if the input/output training tuples often share a lot of prefixes. So, you might try doing something like reversing the input to force learning to take a less crappy path [0]. The learner may respond degenerately by inventing a string reversing technique and repeating its prior behavior. So, you iterate again and try something like base64 encoding the input. This might take, but eventually you wind up with so many weird hacks that the learner can't make progress and the meaning of the quantities evaporates.

Every metric I've ever looked at gets cheated in some way. The holy grail is probably normalized information distance (approximated by normalized compression distance), but then you have a whole new problem of finding an ideal universal compressor which definitely doesn't exist.

[0]: https://arxiv.org/abs/1409.3215 (Figure 1)

Re: The Leaderboard Illusion

#38

Post by godelski »

Many of these things are ones that people have been screaming about for years (including Sarah Hooker). It's great to see some numbers attached. And in classic Cohere manner, they are not holding punches on some specific people. Expect them to push back.

There's a crux that makes it easy to understand why we should expect it. If you code (I assume you do) you probably (hopefully) know that you can't test your way into proving your code is correct. Test Driven Development (TDD) is a flawed paradigm. You should use tests, but they are hints. That's why Cohere is quoting Goodhart at the top of the intro[0]. There is NO metric where the metric is perfectly aligned with the reason you implemented that metric in the first place (intent). This is fucking alignment 101 here. Which is why it is really ironic how prolific this attitude is in ML[1]. I'm not sure I believe any person or company that claims they can make safe AI if they are trying to shove benchmarks at you.

Pay close attention, evaluation is very hard. It is also getting harder. Remember reward hacking, it is still alive and well (it is Goodhart's Law). You have to think about what criteria meets your objective. This is true for any job! But think about RLHF and similar strategies. What methods also maximize the reward function? If it is human preference, deception maximizes just as well (or better) than accuracy. This is bad design pattern. You want to make errors as loud as possible, but this paradigm makes errors as quiet as possible and you cannot confuse that with lack of errors. It makes evaluation incredibly difficult.

Metrics are guides, not targets

[0] Users that recognize me may remember me for mentioning 'Goodhart's Hell', the adoption of Goodhart's Law as a feature instead of a bug. It is prolific, and problematic.

[1] We used to say that when people say "AI" instead of "ML" to put your guard up. But a very useful one that's been true for years is "if people try to prove by benchmarks alone, they're selling snakeoil." There should always be analysis in addition to metrics.

Re: The Leaderboard Illusion

#39

Post by internet_rand0 »

bob1029 wrote: post #37

> Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. In context of genetic programming and other non-traditional ML techniques, I've been having difficulty attempting to locate a simple fitness function that reliably proxies natural language string similarity due to this effect. For example, say you use something like common prefix length to measure how clo…

> finding an ideal universal compressor which definitely doesn't exist.

if only we could explain this in "politician" language... too many with too much power think the second coming will deliver the "ideal universal" which doesn't exist

Re: The Leaderboard Illusion

#40

Post by mrandish »

I'm not even following AI model performance testing that closely but I'm hearing increasing reports they're inaccurate due to accidental or intentional test data leaking into training data and other ways of training to the test.

Also, ARC AGI reported they've been unable to independently replicate OpenAI's claimed breakthrough score from December. There's just too much money at stake now to not treat all AI model performance testing as an adversarial, no-holds-barred brawl. The default assumption should be all entrants will cheat in any way possible. Commercial entrants with large teams of highly-incentivized people will search and optimize for every possible advantage - if not outright cheat. As a result, smaller academic, student or community teams working part-time will tend to score lower than they would on a level playing field.

Post reply on HN