Study identifies weaknesses in how AI systems are evaluated
1–10 of 204 posts
Re: Study identifies weaknesses in how AI systems are evaluated
#2Don’t get high on your own supply.
Re: Study identifies weaknesses in how AI systems are evaluated
#3Definitely one of the weaker areas in the current LLM boom. Comparing models, or even different versions of the same model, is a pseudo-scientific mess.
I'm still using https://lmarena.ai/leaderboard. Perhaps there is something better and someone will pipe up to tell me about it. But we use LLMs at work and have unexplainable variations between them.
And when we get a prompt working reliably on one model, we often have trouble porting it to another LLM - even straight "version upgrades" such as from GPT-4 to -5. Your prompt and your model become highly coupled quite easily.
I dunno what to do about it and am tending to just pick Gemini as a result.
Re: Study identifies weaknesses in how AI systems are evaluated
#4I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mess.
Product testing (with traditional A/B tests) are kind of the best bet since you can measure what you care about _directly_ and at scale.
I would say there is of course “benchmarketing” but generally people do sincerely want to make good benchmarks it’s just hard or impossible. For many of these problems we’re hitting capabilities where we don’t even have a decent paradigm to use,
Re: Study identifies weaknesses in how AI systems are evaluated
#5I've been getting flagged by high-on-their-own-supply AI boosters for identifying that LLM benchmarks have been obvious bullshit for at least the last year and a half.
What changed to make "the inevitable AI bubble" the dominant narrative in last week or so?
Re: Study identifies weaknesses in how AI systems are evaluated
#6Definitely one of the weaker areas in the current LLM boom. Comparing models, or even different versions of the same model, is a pseudo-scientific mess. I'm still using https://lmarena.ai/leaderboard . Perhaps there is something better and someone will pipe up to tell me about it. But we use LLMs at work and have unexplainable variations between them. And when we get a prompt working reliably on one model, we often h…
Ratings on LMArena are too easily gamed.
Even professional human evaluators are quite vulnerable to sycophancy and overconfident-and-wrong answers. And LMArena evaluators aren't professionals.
A lot of the sycophancy mess that seeps from this generation of LLM stems from reckless tuning based on human feedback. Tuning for good LMArena performance has similar effects - and not at all by a coincidence.
Re: Study identifies weaknesses in how AI systems are evaluated
#7I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…
A/B testing is radioactive too. It's indirectly optimizing for user feedback - less stupid than directly optimizing for user feedback, but still quite dangerous.
Human raters are exploitable, and you never know whether the B has a genuine performance advantage over A, or just found a meat exploit by an accident.
It's what fucked OpenAI over with 4o, and fucked over many other labs in more subtle ways.
Re: Study identifies weaknesses in how AI systems are evaluated
#8[flagged]
Re: Study identifies weaknesses in how AI systems are evaluated
#9[flagged]
[flagged]
Re: Study identifies weaknesses in how AI systems are evaluated
#10[flagged]
Any argument to back up your claim?