A test doesn't need to be objectively meaningful or rigorous in any sense in order to still be useful for comparative ranking.
Study identifies weaknesses in how AI systems are evaluated
121–130 of 204 posts
Re: Study identifies weaknesses in how AI systems are evaluated
#122I've written about Humanity's Last Exam, which crowdsources tough questions for AI models from domain experts around the world.
https://www.happiesthealth.com/articles/future-of-health/hum...
It's a shifting goalpost, but one of the things that struck me was how some questions could still be trivial for a fairly qualified human (a doctor in this case) but difficult for an AI model. Reasoning, visual or logic, is built on a set of assumptions that are better gained through IRL experience than crawling datasets and matching answers.
This leads me to believe that much of the future for training AI models will lie in exposing them to "meatspace" and annotating their inferences, much like how we train a child. This is a long, long process, and one that is already underway at scale. But it's what might give us emergent intelligences rather than just a basket of competing yet somehow-magic thesaurus.
Re: Study identifies weaknesses in how AI systems are evaluated
#123Earlier quoted context omitted.
At the developer level all my LLM use is in the context of agentic wrappers, so my benchmark is fairly trivial: Configure aider or claude code to use the new model, try to do some work. The benchmark is pass/fail, if after a little while I feel the performance is better than the last model I was using it's a pass, otherwise it's a fail and I go back. Building your own evaluations makes sense if you're serving an LLM…
> Building your own evaluations makes sense if you're serving an LLM up to customers and want to know how it performs, but if you are the user... use it and see how it goes. It's all subjective anyway. I'd really caution against this approach, mainly because humans suck at removing emotions and other "human" factors when judging how well something works, but also because comparing across models gets a lot easier when…
So what? I'm the one that's using it, I happen to be a human, my human factor is the only one that matters.
At this point anyone using these LLMs every day have seen those benchmark numbers go up without an appreciable improvement in the day to day experience.
Re: Study identifies weaknesses in how AI systems are evaluated
#124A test doesn't need to be objectively meaningful or rigorous in any sense in order to still be useful for comparative ranking.
yes it does - it has to be meaningful or rigorous for the comparative ranking to be meaningful or rigorous, or else wtf are you doing? Say I have all the information on my side but only these questions that you are showing the user? Who cares about that comparison?
Re: Study identifies weaknesses in how AI systems are evaluated
#125Benchmarks are like SAT scores. Can they guarantee you'll be great at your future job? No, but we are still roughly okay with what they signify. Clearly LLMs are getting better in meaningful ways, and benchmarks correlate with that to some extend.
There’s no a priori reason to expect a test designed to test human academic performance would be a good one to test LLM job performance.
For example a test of “multiply 1765x9392” would have some correlation with human intelligence but it wouldn’t make sense to apply it to computers.
Re: Study identifies weaknesses in how AI systems are evaluated
#126Earlier quoted context omitted.
> Building your own evaluations makes sense if you're serving an LLM up to customers and want to know how it performs, but if you are the user... use it and see how it goes. It's all subjective anyway. I'd really caution against this approach, mainly because humans suck at removing emotions and other "human" factors when judging how well something works, but also because comparing across models gets a lot easier when…
So what? I'm the one that's using it, I happen to be a human, my human factor is the only one that matters. At this point anyone using these LLMs every day have seen those benchmark numbers go up without an appreciable improvement in the day to day experience.
> So what? I'm the one that's using it, I happen to be a human, my human factor is the only one that matters.
Yeah no you're right, if consistency isn't important to you as a human, then it doesn't matter. Personally, I don't trust my "humanness" and correctness is the most important thing for me when working with LLMs, so that's why my benchmarks focus on.
> At this point anyone using these LLMs every day have seen those benchmark numbers go up without an appreciable improvement in the day to day experience.
Yes, this is exactly my point. The benchmarks the makers of these LLMs seems to always provide a better and better score, yet the top scores in my own benchmarks have been more or less the same for the last 1.5 years, and I'm trying every LLM I can come across. These "the best LLM to date!" hardly ever actually is the "best available LLM", and while you could make that judgement by just playing around with LLMs, actually be able to point to specifically why that is, is something at least I find useful, YMMV.
Re: Study identifies weaknesses in how AI systems are evaluated
#127Humans are much better at out of sample prediction than LLMs. And inherently benchmarks cannot be out of sample. So I believe that leads to the disconnect between LLMs getting better and better at in sample prediction (benchmarks) while not improving nearly as much at out of sample (actual work).
Re: Study identifies weaknesses in how AI systems are evaluated
#128We should make a collective git repo full of every kind of annoying bug we (expert developers) can think of. Then use that to benchmark LLMs. Someone want to start? I've got a Yjs/CRDT collaborative editing bug that took like a week and a half of attempts with Claude Code (Sonnet 4.5), GPT5-codex (medium), and GLM-4.6 many, many attempts to figure out. Even then they didn't really get it... Just came up with a succes…
This may be intentional, but I'd like to point out that your basically suggesting that others aggregate high-quality training data for AI companies to use free of charge to replace software engineers.
Re: Study identifies weaknesses in how AI systems are evaluated
#129"Measuring money turns out to be easier than measuring intelligence."
Don't ever change, El Reg.
Re: Study identifies weaknesses in how AI systems are evaluated
#130This might explain the zeitgeist that new models feel same-ish, despite model developers saying they're getting spectacularly better.