AI detractors can say whatever. As a developer Claude Code is almost an unfair cheat code. AI valuations may be absurd but the hype is justified.
Study identifies weaknesses in how AI systems are evaluated
141–150 of 204 posts
Re: Study identifies weaknesses in how AI systems are evaluated
#142Benchmarks are like SAT scores. Can they guarantee you'll be great at your future job? No, but we are still roughly okay with what they signify. Clearly LLMs are getting better in meaningful ways, and benchmarks correlate with that to some extend.
Isn’t this like grading art critics? We took objective computers, and made them generate subjective results. Isn’t this a problem that we already know there’s no solution to? That grading subjectivity is just subjective itself.
[deleted]
Re: Study identifies weaknesses in how AI systems are evaluated
#143I'm already quite put off by the title (it's science -- if you have a better benchmark, publish it!), but the contents aren't great either. It keeps citing numbers about "445 LLM benchmarks" without confirming whether any of the ones they deem insufficiently statistical are used by any of the major players. I've seen a lot of benchmarks, but maybe 20 are used regularly by large labs, max. "For example, if a benchmark…
(We've since changed both title and URL - see https://news.ycombinator.com/item?id=45860056)
Re: Study identifies weaknesses in how AI systems are evaluated
#144They should laugh while they can ;) Still waiting for the crash and to see what lives on and what gets recycled. My bet is that grok is here to stay ;) (Don't hurt me, I just like his chatbot. It's the best I've tried at, "Find the passage in X that reminded me of the passage in Y given this that and the other thing." It has a tendency to blow smoke if you let it, but they all seek to affirm more than I'd like, but a…
Grok is terrible at coding though.
If models get commoditised, distribution (and vertical integration) become key. OpenAI and xAI are the only companies that seem to be well hedged for this risk.
Re: Study identifies weaknesses in how AI systems are evaluated
#145A test doesn't need to be objectively meaningful or rigorous in any sense in order to still be useful for comparative ranking.
yes it does - it has to be meaningful or rigorous for the comparative ranking to be meaningful or rigorous, or else wtf are you doing? Say I have all the information on my side but only these questions that you are showing the user? Who cares about that comparison?
[deleted]
Re: Study identifies weaknesses in how AI systems are evaluated
#146I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…
For what it's worth, I work on platforms infra at a hyperscaler and benchmarks are a complete fucking joke in my field too lol. Ultimately we are measuring extremely measurable things that have an objective ground truth. And yet: - we completely fail at statistics (the MAJORITY of analysis is literally just "here's the delta in the mean of these two samples". If I ever do see people gesturing at actual proper analysi…
“Here’s the throughout at sustained 100% load with the same ten sample queries repeated over and over.”
“The customers want lower latency at 30% load for unique queries.”
“Err… we can scale up for more throughput!”
ಠ_ಠ
Re: Study identifies weaknesses in how AI systems are evaluated
#147Earlier quoted context omitted.
I can't tell how much of this is sarcasm > we (expert developers) ... > took like a week and a half of attempts with Claude Code ... What kind of expert developer wastes that much time prompting a bunch of different LLMs to end up with a workaround, instead of actually debugging and fixing the bug themselves?
Fair question but I think the tone of this is a bit abrasive towards the poster, and unnecessarily so.
I really don't think so. "Expert" developer really needs to mean something other than "prompting and poking at Claude Code".
Re: Study identifies weaknesses in how AI systems are evaluated
#148I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…
For what it's worth, I work on platforms infra at a hyperscaler and benchmarks are a complete fucking joke in my field too lol. Ultimately we are measuring extremely measurable things that have an objective ground truth. And yet: - we completely fail at statistics (the MAJORITY of analysis is literally just "here's the delta in the mean of these two samples". If I ever do see people gesturing at actual proper analysi…
In AI though, you also have the world trying to compete with you, so even if you do totally cheat and put the benchmark answers in your training set and over fit, if it turns out that you model sucks, it doesn't matter how much your marketing department tells everyone you scored 110% on SWE bench, if it doesn't work out that well in production, your announcement's going to flow as users discover it doesn't work that well on their personal/internal secret benchmarks and tell /r/localLLAMA it isn't worth the download.
Whatever happened with Llama 4?
Re: Study identifies weaknesses in how AI systems are evaluated
#149> "For example, if a benchmark reuses questions from a calculator-free exam such as AIME," the study says, "numbers in each problem will have been chosen to facilitate basic arithmetic. Testing only on these problems would not predict performance on larger numbers, where LLMs struggle." When models figure out how to exploit an effect that every clever college student does, that should count as a win. That’s a much mo…
The point of these LLMs is to do things that computers were bad at.
That's a good point imo but we achieved this stuff by at least 2022 when ChatGPT was released. The thing about these giant black boxes is that they also fail to do things that directly human-written software ("computers") does easily. The inability to print text onto generated images or do general arithmetic is important. And sure, some of these limits look like "limits of humans". But it is important to avoid jumping from "they do this human-thing" to "they're like humans".
Re: Study identifies weaknesses in how AI systems are evaluated
#150We should make a collective git repo full of every kind of annoying bug we (expert developers) can think of. Then use that to benchmark LLMs. Someone want to start? I've got a Yjs/CRDT collaborative editing bug that took like a week and a half of attempts with Claude Code (Sonnet 4.5), GPT5-codex (medium), and GLM-4.6 many, many attempts to figure out. Even then they didn't really get it... Just came up with a succes…
What was the CRDT bug?