Tangent point, I think more broadly this is a big piece of AI-cynicism in general- “x isn’t special anymore”. It’s tough staying motivated on a craft when an AI is nearly as good as you. Chess players manage to do it at least.
[dead]
Vulnerability reports are not special anymore
31–40 of 249 posts
Re: Vulnerability reports are not special anymore
#32At risk of quoting too much of the article, it opens with this: > A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all. > Except… > For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t app…
> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog.
I think your logic is partly correct but the fact that the same LLMs are allowing an exponential increase in insecure code generated is a counterbalancing point. I do not think this phenomena will slow down.
Re: Vulnerability reports are not special anymore
#33I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…
I believe the term is Beg Bounties and they are constant and annoying.
Re: Vulnerability reports are not special anymore
#34I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special.
Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to minimize creating bugs in the first place. That could be new or better languages, less 3rd party dependencies, more vetted code, better linters, better fuzzers, whatever. The point the new reality of bugs being easy to find will, actually must, lead to less bugs eventually because the world can't function with easy to find bugs.
Re: Vulnerability reports are not special anymore
#35I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…
Have you considered having an agent, or just a model, classify/triage them for you? Modern problems require modern solutions.
Re: Vulnerability reports are not special anymore
#36>> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation.
I don't think the gift analogy works well. In most cultures, turning down or even ignoring a gift is considered anywhere from impolite to hugely offensive. But that's the opposite of open source: there's nothing wrong with requesting changes to a PR or even closing it.
Re: Vulnerability reports are not special anymore
#37Vulnerability reports were never special. The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now. Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstratio…
This x 1000
I’ve been screaming this from the rooftops. Impact is what was always important. No one is going to take down prod to do an emergency patch on an RCE that COULD NEVER ACTUALLY BE EXPLOITED.
I feel like we’re witnessing the result of multiple roles suddenly becoming security aware but not having the background or understanding to make any sense of it.
Re: Vulnerability reports are not special anymore
#38At risk of quoting too much of the article, it opens with this: > A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all. > Except… > For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t app…
It's not (just) more of them, it's the same ones reported by multiple people.
I think the point is those issues are now easily discoverable and are nearly public because of it.
Re: Vulnerability reports are not special anymore
#39I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…
Lol you think LLMs are generating bug free code?
Re: Vulnerability reports are not special anymore
#40Linus Torvalds once went on record saying security vulnerabilities are no more important than regular bugs.
This of course made vulnerability researchers seethe worse than aggrieved Redditors.
It turns out he was right all along.
The author also gets it wrong by assuming that regular bug reporters are not "providing a service". They are.
When I wrote up a bug report, I made sure it's thorough with detailed steps to reproduce. It takes a lot of time and I've done it professionally for projects you've absolutely heard of.
Having said that, getting them ignored repeatedly and — even worse — having my detailed PRs rejected, sometimes within minutes, as if I'm some ignorant luser is why I don't do it anymore. My time is more valuable than your hubris.
A lot of open source developers have their heads so far up their own asses they forgot that it takes a community for projects to be successful.