[flagged]
The Benchmarkpocalypse
11–20 of 74 posts
Re: The Benchmarkpocalypse
#12[dead]
Re: The Benchmarkpocalypse
#13Earlier quoted context omitted.
Ungrounded LLM outputs are a bit like your dreams. Without anything to test hypotheses against, stuff can pop in and out of existence and physics is just advice. Ground your LLM. Tests, documentation, give it many ways to run the thing its reasoning about. It needs to be able to test its hypotheses on its own. Take yourself out of that loop so you only find out once it's sure.
Love LLMs gonna keep using them. It feels like your suggested approach is expensive, in terms of tokens. I feel (second time I say this) that when I steer the process I get pretty good results vs my coworkers that let the LLMs run away. I do have data on our token usage, not much in terms of quality of the delivery. I keep thinking about the c compiler implementation that anthropic shared earlier in the year that had…
Do you steer your agents by manually running every single test and linter and reporting the results back to them?
Re: The Benchmarkpocalypse
#14Fascinating article. I daily catch LLMs in “lies” like: “I found the root cause of the bug” or “this approach is twice as fast”. It’s hard to say what causes this uninformed certainty - is it intrinsic to being trained on human writing, or something that comes from the RLHF process afterwards, but it’s extremely annoying. It’s one thing to have a LLM make poor decisions, but it feels worse to have it “lie” to you in…
LLM aren't logic machines. It's concepts of "root cause" and "faster" are purely linguistic, including all the internal conflicts of those loose definitions.
In that sense an LLM wouldn't even think it "lied" being confidently incorrect, in human terms it's just a "whoopsie". Apparently we want exactly that.
Re: The Benchmarkpocalypse
#15Nice article. As perf optimisation has become more like a machine learning problem, maybe there is opportunity to use other techniques for checking generalisation. Like e.g. cross validation
Re: The Benchmarkpocalypse
#16I stopped trusting benchmarks ever since LLMs started speaking alien like English, if I can't understand what they are saying how can I trust them.
Re: The Benchmarkpocalypse
#17[dead]
Re: The Benchmarkpocalypse
#18I stopped trusting benchmarks ever since LLMs started speaking alien like English, if I can't understand what they are saying how can I trust them.
same here, it reads exactly the same whether the number is real or
completely made up, so the confidence stops meaning anything.
Re: The Benchmarkpocalypse
#19Fascinating article. I daily catch LLMs in “lies” like: “I found the root cause of the bug” or “this approach is twice as fast”. It’s hard to say what causes this uninformed certainty - is it intrinsic to being trained on human writing, or something that comes from the RLHF process afterwards, but it’s extremely annoying. It’s one thing to have a LLM make poor decisions, but it feels worse to have it “lie” to you in…
This is a "don't make me tap the sign" moment. LLMs are next token prediction models. If there are factual errors, confused ideas, etc. in the preceding tokens, that will affect the generation of subsequent tokens, and the error accumulates.
Case in point, I hit an error in a SQL query today because it turned out I was trying to do something that wasn't supported by the query engine. I pasted the error message and a bit of background info into my Claude Code session with Sonnet 5 High, it worked on a response for an unexpectedly long amount of time, including consulting the advisor model, and then came back with an explanation of the mistake I made in my query. Except it turned out I pointed it to the wrong file, and there wasn't a mistake in that file. It had completely taken for granted that the pasted error output was a real error and went on some wild goose chase.
Part of why the current gen models feel so smart is that they're getting better (via CoT and training) at recognizing when something is wrong and then back up to reassess. So it's easy to forget that it really is just token prediction, and (pending the next big advancement) there's only so much you can do with that.
Re: The Benchmarkpocalypse
#20Fascinating article. I daily catch LLMs in “lies” like: “I found the root cause of the bug” or “this approach is twice as fast”. It’s hard to say what causes this uninformed certainty - is it intrinsic to being trained on human writing, or something that comes from the RLHF process afterwards, but it’s extremely annoying. It’s one thing to have a LLM make poor decisions, but it feels worse to have it “lie” to you in…
This is a "don't make me tap the sign" moment. LLMs are next token prediction models. If there are factual errors, confused ideas, etc. in the preceding tokens, that will affect the generation of subsequent tokens, and the error accumulates. Case in point, I hit an error in a SQL query today because it turned out I was trying to do something that wasn't supported by the query engine. I pasted the error message and a…
That’s a bit of a garbage in, garbage out moment.