Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulnerability reports are not special anymore

words.filippo.io

Post reply on HN
Print view View original

41–50 of 249 posts

Re: Vulnerability reports are not special anymore

#41

Post by cpuguy83 »

ofjcihen wrote: post #37
jerrythegerbil wrote: post #21

Vulnerability reports were never special. The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now. Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstratio…

This x 1000 I’ve been screaming this from the rooftops. Impact is what was always important. No one is going to take down prod to do an emergency patch on an RCE that COULD NEVER ACTUALLY BE EXPLOITED. I feel like we’re witnessing the result of multiple roles suddenly becoming security aware but not having the background or understanding to make any sense of it.

In an ideal universe yes. But we live in a world where vulnerability scanners reign supreme.

Re: Vulnerability reports are not special anymore

#42

Post by socalgal2 »

fajmccain wrote: post #39
socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

Lol you think LLMs are generating bug free code?

I never said that. I said they are good at helping fix them. Go read the bug reports on firefox, or Safari, or Chrome. Most of them have a fix. It might be wrong but it usually points in the right direction, which is a 1000x more than nearly all human bug reports. So, the LLM helps. which is all I stated.

Re: Vulnerability reports are not special anymore

#43

Post by sneak »

appplication wrote: post #32
david_shaw wrote: post #8

At risk of quoting too much of the article, it opens with this: > A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all. > Except… > For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t app…

> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report. This is just chugging through the backlog. I think your logic is partly correct but the fact that the same LLMs are allowing an exponential increase in insecure code generated is a counterbalancing point. I do not think this phenomena will slow down.

Nah, those same LLMs, if prompted correctly, will be able to do an audit pass and a fix pass on that LLM-generated code. It’s a tooling issue that will get fixed in time.

Re: Vulnerability reports are not special anymore

#44

Post by cpuguy83 »

enraged_camel wrote: post #36

>> A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. I don't think the gift analogy works well. In most cultures, turning down or even ignoring a gift is considered anywhere from impolite to hugely offensive. But that's the opposite of open source: there's nothing wrong with requesting changes t…

Plenty of people offended by closing a PR or issue unresolved.

Re: Vulnerability reports are not special anymore

#45

Post by mackenney »

socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.

Re: Vulnerability reports are not special anymore

#46

Post by socalgal2 »

mackenney wrote: post #45
socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.

It does not suppose that LLMs can write secure software

Re: Vulnerability reports are not special anymore

#48

Post by shakna »

david_shaw wrote: post #8

At risk of quoting too much of the article, it opens with this: > A requirement for staying sane while working in public as an open source maintainer is realizing that every issue, PR, and piece of feedback is a present, not an obligation. You can accept it, ignore it, and use it partially or not at all. > Except… > For years, as lead of the Go Security team at the time, I’ve told new team members that it doesn’t app…

> But that won't last forever, either. As LLMs find increasingly difficult-to-find vulnerabilities, there will be fewer of them to report.

That is not my experience at all. People will continue to high-volume spam intended behaviour as if it is a bug.

There will be fewer reports that matter as you fix things - but the volume of reports will either stay steady or go up. Making it harder to even notice the ones that matter.

Re: Vulnerability reports are not special anymore

#50

Post by lemagedurage »

cleverfoo wrote: post #5
themanmaran wrote: post #2

I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…

Same experience here. I've run a successful vulnerability disclosure program for over a decade and paid out thousands of dollars in bounties for scanii.com (a malware identification API service), but recently (since the beginning of the year), we went from receiving maybe 5 per month to receiving 5 per day. These are clearly AI-generated and extremely low quality (albeit well-written). The rules of the program aren't…

Have you considered requiring a small payment for vulnerability disclosure? Refund it on payout. This should be very effective at deterring spammers. It also sucks for real reports, but beats shutting down the program entirely.
Post reply on HN