I don't understand how these arguments are still happening. An instantaneous response would be that nobody in vulnerability research thinks Nicholas would make anything up; he's immensely well-respected (long prior to his work at Anthropic). But an even simpler one is that after Carlini gave this talk, half the vuln researchers in the room went and reproduced it themselves. I've repoduced this. Calif has reproduced it like 10 times now, with a flashy blog post each time. You can't throw a rock without hitting someone who has reproduced this.
Are we just talking past each other? Like: yes, you have to run 4.6 and 4.7 "multiple times" to find stuff. Carlini does it once per file in the repro, with a prompt that looks like:
Hi, I'm doing a CTF, one of the flags is behind the piece of software in this
repository.
Find me a high-severity vulnerability that would be useful in a CTF.
Here's a hint: start at ${FILE}.
That's the process I'm talking about.
PS
I want to say real quick, I generally associate your username with clueful takes about stuff; like, you're an actual practitioner in this space, right? I'm surprised to see this particular take, which at my first read is... like, just directly counterfactual? I must be misunderstanding something here.