Maybe use fixed bugs from closed source to benchmark these "frontier" models.
The Benchmarkpocalypse
21–30 of 74 posts
Re: The Benchmarkpocalypse
#22Fascinating article. I daily catch LLMs in “lies” like: “I found the root cause of the bug” or “this approach is twice as fast”. It’s hard to say what causes this uninformed certainty - is it intrinsic to being trained on human writing, or something that comes from the RLHF process afterwards, but it’s extremely annoying. It’s one thing to have a LLM make poor decisions, but it feels worse to have it “lie” to you in…
This is a "don't make me tap the sign" moment. LLMs are next token prediction models. If there are factual errors, confused ideas, etc. in the preceding tokens, that will affect the generation of subsequent tokens, and the error accumulates. Case in point, I hit an error in a SQL query today because it turned out I was trying to do something that wasn't supported by the query engine. I pasted the error message and a…
Why is this being downvoted?
Re: The Benchmarkpocalypse
#23Fascinating article. I daily catch LLMs in “lies” like: “I found the root cause of the bug” or “this approach is twice as fast”. It’s hard to say what causes this uninformed certainty - is it intrinsic to being trained on human writing, or something that comes from the RLHF process afterwards, but it’s extremely annoying. It’s one thing to have a LLM make poor decisions, but it feels worse to have it “lie” to you in…
Ungrounded LLM outputs are a bit like your dreams. Without anything to test hypotheses against, stuff can pop in and out of existence and physics is just advice. Ground your LLM. Tests, documentation, give it many ways to run the thing its reasoning about. It needs to be able to test its hypotheses on its own. Take yourself out of that loop so you only find out once it's sure.
But this is exactly what the AI labs should be doing ...
Re: The Benchmarkpocalypse
#24[flagged]
Re: The Benchmarkpocalypse
#25> it's become easier than ever to make serious performance gains
Is that true and if yes why? I was under the impression that it would become more difficult over time to make serious performance gains, which would also fit with reaching for benchmark hacking rather than relying on natural gains.
Re: The Benchmarkpocalypse
#26I stopped trusting benchmarks ever since LLMs started speaking alien like English, if I can't understand what they are saying how can I trust them.
This article is about using LLMs to overfit for a specific benchmark (or make a custom software for niche use cases) though. Not about LLMs benchmaxxxing
Re: The Benchmarkpocalypse
#27Cheating and overfitting, as discussed in the article, are the most obvious problems with benchmarking LLMs. But there is also the aspect that, at least for closed models, the tokens still have to be sent to the provider's servers for inference. This makes the holdout set not as held out as it may appear. OpenAI and Antropic probably don't care about your private set of regex benchmarks, but for the headline "closed" benchmarks, I'd be surprised if they haven't collected a nice representative set of "holdout" problems to be examined at leisure.
Re: The Benchmarkpocalypse
#28Not trying to defend anyone, but in my experience the latest models have been performing significantly better than 8 months ago. So, in my book the extent of over-fitting on benchmarks seems to be covering my use cases.
Re: The Benchmarkpocalypse
#29I stopped trusting benchmarks ever since LLMs started speaking alien like English, if I can't understand what they are saying how can I trust them.
That's the load-bearing smoking gun—should I write a better benchmarks to catch the seams?
Re: The Benchmarkpocalypse
#30Fascinating article. I daily catch LLMs in “lies” like: “I found the root cause of the bug” or “this approach is twice as fast”. It’s hard to say what causes this uninformed certainty - is it intrinsic to being trained on human writing, or something that comes from the RLHF process afterwards, but it’s extremely annoying. It’s one thing to have a LLM make poor decisions, but it feels worse to have it “lie” to you in…
This is a "don't make me tap the sign" moment. LLMs are next token prediction models. If there are factual errors, confused ideas, etc. in the preceding tokens, that will affect the generation of subsequent tokens, and the error accumulates. Case in point, I hit an error in a SQL query today because it turned out I was trying to do something that wasn't supported by the query engine. I pasted the error message and a…
> Except it turned out I pointed it to the wrong file, and there wasn't a mistake in that file.
That’s a fairly common error, on my part. I may have a wrong file, selected in the Xcode Navigator, and the LLM accepts that, as the problematic file. In other cases, I’m just plain wrong, in my diagnosis. Maybe the error manifests in that file, but is caused, in another one. I have taken to giving the LLM as many involved files as possible, when asking for help.
I found that, as the LLM progressed through my project, learning more and more about it, it caught these errors, but early on, it ran with my wrong input, and gave bad advice.
It still, occasionally, gives bad advice, but that’s just because it was probably trained on bad code. I have found that it gives good advice, more often than not. It’s just incumbent on me, to treat its input as “advice,” and carefully consider it, before integrating it into my own work.
I have just upgraded my account. In order to do this, I had to create a new one, for billing purposes. This means that it needs to start over, learning the project. I am in the process of exporting from the previous account, but I’m not sure how well this will transfer the accumulated knowledge.