Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Study identifies weaknesses in how AI systems are evaluated

oii.ox.ac.uk

Post reply on HN
Print view View original

191–200 of 204 posts

Re: Study identifies weaknesses in how AI systems are evaluated

#191

Post by mbesto »

scuff3d wrote: post #156
bubblelicious wrote: post #4

I work on LLM benchmarks and human evals for a living in a research lab (as opposed to product). I can say: it’s pretty much the Wild West and a total disaster. No one really has a good solution, and researchers are also in a huge rush and don’t want to end up making their whole job benchmarking. Even if you could, and even if you have the right background you can do benchmarks full time and they still would be a mes…

The big problem is that tech companies and journalist aren't transparent about this. They tout benchmark numbers constantly, like they're an object measure of capabilities.

HN members do too. Look at my comment history.

The general populace doesn't care to question how benchmarks are formulated and what their known (and unknown) limitations are.

That being said, they are likely decent proxies. For example, I think the average user isn't going to observe a noticeable difference between Claude Sonnet and OpenAI Codex.

Re: Study identifies weaknesses in how AI systems are evaluated

#192

Post by naasking »

Clearly we need tests that check for effectiveness at applying general mathematical, logical and relational operations, eg. set theory, relational algebra, first and second order logic, type theory, the lambda calculus, recurrence and induction, etc., and the ability to use these to abstract over specifics and the ability to generalize.

The upside is that these can all be generated and checked synthetically so large data sets are possible, in both formal and natural languages.

Re: Study identifies weaknesses in how AI systems are evaluated

#193

Post by andai »

calpaterson wrote: post #3

Definitely one of the weaker areas in the current LLM boom. Comparing models, or even different versions of the same model, is a pseudo-scientific mess. I'm still using https://lmarena.ai/leaderboard . Perhaps there is something better and someone will pipe up to tell me about it. But we use LLMs at work and have unexplainable variations between them. And when we get a prompt working reliably on one model, we often h…

>when we get a prompt working reliably on one model, we often have trouble porting it to another LLM

I saw a study where a prompt massively boosted one model's performance on a task, but significantly reduced another popular model's performance on the same task.

Re: Study identifies weaknesses in how AI systems are evaluated

#194

Post by ACCount37 »

achierius wrote: post #179
ACCount37 wrote: post #178

Earlier quoted context omitted.

That's because they are as close to "object measure capabilities" as anything we're ever going to get. Without benchmarks, you're down to evaluating model performance based on vibes and vibes only, which plain sucks. With benchmarks, you have numbers that correlate to capabilities somewhat.

That's assuming these benchmarks are the best we're ever going to get, which they clearly aren't. There's a lot to improve even without radical changes to how things are done.

The assumption I make is that "better benchmarks" are going to be 5% better, not 5000% better. LLMs are getting better capabilities faster than the benchmarks get better at measuring them accurately.

So, yes, we just aren't going to get anything that's radically better. Just more of the same, and some benchmarks that are less bad. Which is still good. But don't expect a Benchmark Revolution when everyone suddenly realizes just how Abjectly Terrible the current benchmarks are, and gets New Much Better Benchmarks to replace them with. The advances are going to be incremental, unimpressive, and meaningful only in aggregate.

Re: Study identifies weaknesses in how AI systems are evaluated

#195

Post by ACCount37 »

scuff3d wrote: post #182
ACCount37 wrote: post #178

Earlier quoted context omitted.

That's because they are as close to "object measure capabilities" as anything we're ever going to get. Without benchmarks, you're down to evaluating model performance based on vibes and vibes only, which plain sucks. With benchmarks, you have numbers that correlate to capabilities somewhat.

So because there isn't a better measure it's okay that tech companies effectively lie and treat these benchmarks like they mean more then they actually do?

Sorry, pal, but if benchmarks were to disagree with opinions of a bunch of users saying "tech companies bad"? I'd side with benchmarks at least 9 times out of 10.

Re: Study identifies weaknesses in how AI systems are evaluated

#196

Post by grugagag »

Al-Khwarizmi wrote: post #159
stogot wrote: post #152

Earlier quoted context omitted.

Don’t most computer science programs require this? Mine had a statistics requirement

I don't know how it is in the US and other countries, but in my country I would say statistics is typically not taught well, at least in CS degrees. I was a very good student, always had good understanding at the subjects at university, but in the case of statistics they just taught us formulae and techniques as dogmas without much explanation of where they came from, why, and when to use them. It didn't help either…

I had the same experience in the US

Re: Study identifies weaknesses in how AI systems are evaluated

#197

Post by boccaff »

andai wrote: post #193
calpaterson wrote: post #3

Definitely one of the weaker areas in the current LLM boom. Comparing models, or even different versions of the same model, is a pseudo-scientific mess. I'm still using https://lmarena.ai/leaderboard . Perhaps there is something better and someone will pipe up to tell me about it. But we use LLMs at work and have unexplainable variations between them. And when we get a prompt working reliably on one model, we often h…

>when we get a prompt working reliably on one model, we often have trouble porting it to another LLM I saw a study where a prompt massively boosted one model's performance on a task, but significantly reduced another popular model's performance on the same task.

Do you have any pointer to search for that?

Re: Study identifies weaknesses in how AI systems are evaluated

#198

Post by the8472 »

SV_BubbleTime wrote: post #134
sroussey wrote: post #132

Earlier quoted context omitted.

Actually… ask gpt1 to multiply 1765x9392.

I wish this was more broadly, explained to people… There are LLMs, the engines that make these products run, and then the products themselves. GPT anything should not be asked math problems. LLMs are language models, not math. The line is going to get very blurry because ChatGPT, or Claude or Gemini, are not LLM’s. Their products driven by LLMs. The question or requisite should not be can my LLM do math. It can I bui…

I think the argument was more about that the ability of LLMs to multiply without tool use improves over the generations, so it does happen to be yet another test showing improved abilities.

Re: Study identifies weaknesses in how AI systems are evaluated

#199

Post by scuff3d »

ACCount37 wrote: post #195
scuff3d wrote: post #182

Earlier quoted context omitted.

So because there isn't a better measure it's okay that tech companies effectively lie and treat these benchmarks like they mean more then they actually do?

Sorry, pal, but if benchmarks were to disagree with opinions of a bunch of users saying "tech companies bad"? I'd side with benchmarks at least 9 times out of 10.

How does that have anything to do with what we're talking about?

Re: Study identifies weaknesses in how AI systems are evaluated

#200

Post by gishh »

ghurtado wrote: post #116
gishh wrote: post #79

Earlier quoted context omitted.

Pretty sure the only thing computer vision does is math. I’ve also observed email clients tallying the number of unread emails I have. It’s quite obnoxious actually, but I qualify adding as math.

> Pretty sure the only thing computer vision does is math. That is only marginally less pedantic than saying that the only thing computer vision does is run discrete electrical signals through billions of transistors.

If you’ve ever written code for a computer vision application, you’d realize how incorrect this statement is.
Post reply on HN