Why Claude's Comment Paper Is a Poor Rebuttal
victoramartinez.com
Why Claude's Comment Paper Is a Poor Rebuttal
1–10 of 77 posts
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#2A fundamental problem that we’re still far away from solving is not necessarily that LLMs/LRMs cannot reason the same way that we do (which I guess should be clear by now); but that they might not have to. They generate slop so fast that, if one can benefit a little bit from each output, i.e. if you can find a little bit of use hidden beneath the mountain of meaningless text they’ll create, then this might still be more valuable than preemptively taking the time to create something more meaningful to begin with. I can’t say for sure what is the reward system behind LLM use in general, but given how much money people are willing to spend with models even in their current deeply flawed state, I’d say it’s clear that the time savings are outweighing the mistakes and shallowness.
Take the comment paper, for example. Since Claude Opus is the first author, I’m assuming that the human author took a backseat and let the AI build the reasoning and most of the writing. Unsurprisingly, it is full of errors and contradictions, to a point where it looks like the human author didn’t bother too much to check what was being published. One might say that the human author, in trying to build some reputation by showing that their model could answer a scientific criticism, actually did the opposite: it provided more evidence that its model cannot reason deeply, and maybe hurt their reputation even more.
But the real question is, did they really? How much backlash will they possibly get from submitting this to arxiv without checking? Would that backlash keep them from submitting 10 more papers next week with Claude as the first author? If one puts in a balance the amount of slop you can put out (with a slight benefit) vs. the bad reputation one gets from it, I cannot say that “human thinking” is actually worth it anymore.
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#3A fundamental problem that we’re still far away from solving is not necessarily that LLMs/LRMs cannot reason the same way that we do (which I guess should be clear by now); but that they might not have to. They generate slop so fast that, if one can benefit a little bit from each output, i.e. if you can find a little bit of use hidden beneath the mountain of meaningless text they’ll create, then this might still be m…
Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling.
If anything the outcome will be good: mediocre people will produce even worse work and will weed themselves out.
Cause in point: the author of the rebuttal made basic and obvious mistakes that make his work even easier to dismiss and no further paper of his will be considered seriously.
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#4A fundamental problem that we’re still far away from solving is not necessarily that LLMs/LRMs cannot reason the same way that we do (which I guess should be clear by now); but that they might not have to. They generate slop so fast that, if one can benefit a little bit from each output, i.e. if you can find a little bit of use hidden beneath the mountain of meaningless text they’ll create, then this might still be m…
Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling. If anything the outcome will be good: mediocre people will produce even worse work and will weed themselves out. Cause in point: the author of the rebuttal made basic and obvious mistakes that make his work eve…
I think the pull will be hard to resist even for competent people.
Like the obesity crisis driven by sugar highs, the overall population will be affected, and overall quality will suffer, at least for a while.
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#5A fundamental problem that we’re still far away from solving is not necessarily that LLMs/LRMs cannot reason the same way that we do (which I guess should be clear by now); but that they might not have to. They generate slop so fast that, if one can benefit a little bit from each output, i.e. if you can find a little bit of use hidden beneath the mountain of meaningless text they’ll create, then this might still be m…
Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling. If anything the outcome will be good: mediocre people will produce even worse work and will weed themselves out. Cause in point: the author of the rebuttal made basic and obvious mistakes that make his work eve…
>mediocre people will produce even worse work and will weed themselves out.
[[Citation needed]]
I don't believe anyone who has experienced working with other people - in the workspace, in school, whatever - believes that people get weeded out for mediocre output.
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#6Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence.
Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#7A fundamental problem that we’re still far away from solving is not necessarily that LLMs/LRMs cannot reason the same way that we do (which I guess should be clear by now); but that they might not have to. They generate slop so fast that, if one can benefit a little bit from each output, i.e. if you can find a little bit of use hidden beneath the mountain of meaningless text they’ll create, then this might still be m…
Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling. If anything the outcome will be good: mediocre people will produce even worse work and will weed themselves out. Cause in point: the author of the rebuttal made basic and obvious mistakes that make his work eve…
> Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling.
this is clearly not the case, given:
- mass layoffs in the tech industry to force more use of such things - extremely strong pressure from management to use it, rarely framed as "please use this tooling as you see fit" - extremely low quality bars in all sorts of things, e.g. getting your dumb "We wrote a 200 word prompt then stuck that and some web scraped data in to an LLM run by Google/OpenAI/Anthropic" site to the top of hacker news, or most of VC funding in the tech world - extremely large swathes of (at least) the western power structures not giving a shit about doing anything well, e.g. the entire US Federal government leadership now, or the UK government's endless idiocy about "AI Policy development", lawyers getting caught in court having just not even read the documents they put their name on, etc - actual strong desire from many people to outsource their toxic plans to "AI", e.g. the US's machine learning probation or sentencing stuff
I don't think any of us are ready for the tsunami of garbage that's going to be thrown in to every facet of our lives, from government policy to sending people to jail to murdering people with robots to spamming open source projects with useless code and bug reports etc etc etc
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#8Earlier quoted context omitted.
Mediocre people produce mediocre work. Using AI might make those mediocre people produce even worse work, but I don't think it'll affect competent people who have standards regardless of the available tooling. If anything the outcome will be good: mediocre people will produce even worse work and will weed themselves out. Cause in point: the author of the rebuttal made basic and obvious mistakes that make his work eve…
>mediocre people will produce even worse work and will weed themselves out. [[Citation needed]] I don't believe anyone who has experienced working with other people - in the workspace, in school, whatever - believes that people get weeded out for mediocre output.
You can also be mediocre in a lot of different ways. Some people are mediocre thinkers, but fantastic hype men. Some people are fantastic at thinking, but suck at playing the political games you have to play in an office. Personally I find that I need some of all of those aspects to have success in a project, the amount varies by the work and external collaborators.
Intelligence isn't just one measure you can have less or more of. I thought we figured this out 20 years ago.
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#9A fundamental problem that we’re still far away from solving is not necessarily that LLMs/LRMs cannot reason the same way that we do (which I guess should be clear by now); but that they might not have to. They generate slop so fast that, if one can benefit a little bit from each output, i.e. if you can find a little bit of use hidden beneath the mountain of meaningless text they’ll create, then this might still be m…
The human is a bad co-author here really.
I deployed lots of high performance, clean, well documented etc code generated by Claude or o3. I reviewed it wrt requirements, added tests and so on. Even with that in mind it allowed me to work 3x faster.
But it required conscious effort on my part to point out issues and inefficiencies on LLMs part.
It is a collaborative type of work where LLMs shine (even in so called agentic flows)
Re: Why Claude's Comment Paper Is a Poor Rebuttal
#10Has anyone come up with a definition of AGI where humans are near-universally capable of GI? These articles seem to be slowly pushing the boundaries past the point where slower humans are disbarred from intelligence. Many years ago I bumped in to Towers of Hanoi in a computer game and failed to solve it algorithmicly, so I suppose I'm lucky I only work a knowledge job rather than an intelligence-based one.
The original Turing Test was one of the more interesting standards... An expert judge talks with two subjects in order to determine which is the human: one is a human who knows the point of the test, and one is machine trying to fool the judge into being no better than a coin flip at correctly choosing who was human. Allow for many judges and experience in each etc.
The brilliance of the test, which was strangely lost on Turing, is that the test is doubtful to be passed with any enduring consistency. Intelligence is actually more of a social description. Solving puzzles, playing tricky games, etc is only intelligent if we agree that the actor involved faces normal human constraints or more. We don't actually think machines fulfill that (they obviously do not, that's why we build them: to overcome our own constraints), and so this is why calculating logarithms or playing chess ultimately do not end up counting as actual intelligence when a machine does them.