11–20 of 204 posts
I've been getting flagged by high-on-their-own-supply AI boosters for identifying that LLM benchmarks have been obvious bullshit for at least the last year and a half. What changed to make "the inevitable AI bubble" the dominant narrative in last week or so?
Definitely one of the weaker areas in the current LLM boom. Comparing models, or even different versions of the same model, is a pseudo-scientific mess. I'm still using https://lmarena.ai/leaderboard . Perhaps there is something better and someone will pipe up to tell me about it. But we use LLMs at work and have unexplainable variations between them. And when we get a prompt working reliably on one model, we often h…
I've been getting flagged by high-on-their-own-supply AI boosters for identifying that LLM benchmarks have been obvious bullshit for at least the last year and a half. What changed to make "the inevitable AI bubble" the dominant narrative in last week or so?
[flagged]
I've been getting flagged by high-on-their-own-supply AI boosters for identifying that LLM benchmarks have been obvious bullshit for at least the last year and a half. What changed to make "the inevitable AI bubble" the dominant narrative in last week or so?
I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…
Ultimately we are measuring extremely measurable things that have an objective ground truth. And yet:
- we completely fail at statistics (the MAJORITY of analysis is literally just "here's the delta in the mean of these two samples". If I ever do see people gesturing at actual proper analysis, if prompted they'll always admit "yeah, well, we do come up with a p-value or a confidence interval, but we're pretty sure the way we calculate it is bullshit")
- the benchmarks are almost never predictive of the performance of real world workloads anyway
- we can obviously always just experiment in prod but then the noise levels are so high that you can entirely miss million-dollar losses. And by the time you get prod data you've already invested at best several engineer-weeks of effort.
AND this is a field where the economic incentives for accurate predictions are enormous.
In AI, you are measuring weird and fuzzy stuff, and you kinda have an incentive to just measure some noise that looks good for your stock price anyway. AND then there's contamination.
Looking at it this way, it would be very surprising if the world of LLM benchmarks was anything but a complete and utter shitshow!
[flagged]
Any argument to back up your claim?
I decided to take a leap and use AI as much as possible to complete a ticket at work. Now, 3 weeks later AI is writing 90% of my code.
Granted, I'm not sitting back sipping on a latte while AI does my job. It's a very interactive process, I spend more time reviewing code and going back and forth with the AI to get the result I want. But it's become surprisingly good.
I wouldn't say I'm anywhere close to 10x more productive, but perhaps 50%.
I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…
A/B testing is radioactive too. It's indirectly optimizing for user feedback - less stupid than directly optimizing for user feedback, but still quite dangerous. Human raters are exploitable, and you never know whether the B has a genuine performance advantage over A, or just found a meat exploit by an accident. It's what fucked OpenAI over with 4o, and fucked over many other labs in more subtle ways.