Vulnerability research is cooked
sockpuppet.org
Vulnerability research is cooked
1–10 of 181 posts
Re: Vulnerability research is cooked
#2[deleted]
Re: Vulnerability research is cooked
#3[dead]
Re: Vulnerability research is cooked
#4I'm suspicious of this prediction given the curl project's experiences...
Re: Vulnerability research is cooked
#5I'm suspicious of this prediction given the curl project's experiences...
Everybody agrees that idiots were spamming curl with random just-plausible-enough-seeming output from old models.
Re: Vulnerability research is cooked
#6> Now consider the poor open source developers who, for the last 18 months, have complained about a torrent of slop vulnerability reports. I’d had mixed sympathies, but the complaints were at least empirically correct. That could change real fast. The new models find real stuff.
The slop reports won't stop just because real ones are coming in. If the author's right, open source maintainers will still will have to deal with the torrent of slop: on top of triaging and identifying the legit vulnerabilities. Obviously, this is just another role for AI models to fill.
Re: Vulnerability research is cooked
#7I'm suspicious of this prediction given the curl project's experiences...
The problem is that you have all kind of "security spam" in the same way that social media is flooded by automatic, but on-topic, content. This doesn't mean that some very few reports are not correct.
One way to filter that out could be to receive the PoC of the exploit, and test it in some sandbox. I think what XBOW and others are doing is real.
Re: Vulnerability research is cooked
#8I don't understand why the takeaway here is (unless I'm missing something), more or less "everything is going to get exploited all the time". If LLMs can really find a ton of vulnerabilities in my software, why would I not run them and just patch all the vulnerabilities, leading to perfectly secure software (or, at the very least, software for which LLMs can no longer find any new vulnerabilities)?
Re: Vulnerability research is cooked
#9I'm suspicious of this prediction given the curl project's experiences...
The people spamming curl did step one, "write me a vulnerability report on X" but skipped step two, "verify for me that it's actually exploitable". Tack on a step three where a reasonably educated user in the field of security research does a sanity check on the vulnerability implementation as well and you'll have a pipeline that doesn't generate a ton of false positives. The question then will rather be how cost-effective it is for the tokens and the still-non-zero human time involved.
Re: Vulnerability research is cooked
#10I interned for the author at 18. I assumed security testing worked like this:
1. Static analysis catches nearly all bugs with near-total code coverage
2. Private tooling extends that coverage further with better static analysis and dynamic analysis, and that edge is what makes contractors valuable
3. Humans focus on design flaws and weird hardware bugs like cryptographic side-channels from electromagnetic emanations
Turns out finding all the bugs is really hard. Codebases and compiler output have exploded in complexity over 20 years which has not helped the static analysis vision. Todays mitigations are fantastic compared to then, but just this month a second 0day chain got patched on one of the best platforms for hardware mitigations.
I think LLMs get us meaningfully closer to what I thought this work already was when I was 18 and didn't know anything.