Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vulnerability reports are not special anymore

words.filippo.io

Post reply on HN
Print view View original

111–120 of 249 posts

Re: Vulnerability reports are not special anymore

#111

Post by bostik »

socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

"Temporary" can be an awfully long time. There is ample evidence that discovery rate of bugs (many of which can be bucketed into vulnerabilities) in any non-trivial piece of software is more or less stable.[0] In a recent podcast episode the ex-CISO of Adobe commented that every now and then they'd take a sustained squeeze to find all occurrences of a given type of bug (ie. source of vulnerability) in a codebase. They'd find a good amount of them and fix them.

Then a year or two later they'd repeat the operation and they'd find about the same amount of same types of bugs. In many occasions in code that had been in place in the previous round and had remained essentially untouched.

Paraphrasing what the Gruqg has quipped - a large piece of software has infinity bugs. Infinity minus N is still infinity.

0: Discovery rate with regards to the time spent looking for bugs. LLM-powered bug hunting has amped up the speed with which code bases can be investigated.

Re: Vulnerability reports are not special anymore

#112

Post by Maxion »

tempay wrote: post #80
zmgsabst wrote: post #67

Earlier quoted context omitted.

Dev dependencies is how they compromised SolarWinds and thereby most of the US federal government. > The attackers used a supply chain attack. The attackers accessed the build system belonging to the software company SolarWinds, possibly via SolarWinds's Microsoft Office 365 account, which had also been compromised at some point. SolarWinds was using build management and continuous integration server TeamCity provide…

This is very real, but such CVEs are such a tiny fraction in relation to denial-of-service-due-to-regex that it’s hard to take the system seriously. So far as I’m concerned the solution is to isolate everything as much as possible. I’d love to see something on the CVE classification side to also address the signal to noise problem but I don’t see it happening.

These DoS Regex 10/10 CVEs in some minor helper function in some package that is used once in some random side code pathway are so damn annoying.

If I could filter out DoS CVEs‚ I would.

Re: Vulnerability reports are not special anymore

#113

Post by iririririr »

socalgal2 wrote: post #34

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

thats is definitely NOT what the article says.

Are you making a counterpoint that the reports are so good and must all be addresses, but the problem is "llm finding all the bugs" so fast us poor slow humana cannot keep up?

because if so, i suggest you write a new article.

Re: Vulnerability reports are not special anymore

#114

Post by technion »

zmgsabst wrote: post #67
jamesfinlayson wrote: post #51

Earlier quoted context omitted.

I tried to raise that with my internal security team recently - don't clutter my vulnerability dashboard with issues in dev dependencies. They somewhat rightly pointed out that malware needs to be dealt even if it's a dev dependency. So my suggestion went nowhere because I guess we can't filter by type of vulnerability.

Dev dependencies is how they compromised SolarWinds and thereby most of the US federal government. > The attackers used a supply chain attack. The attackers accessed the build system belonging to the software company SolarWinds, possibly via SolarWinds's Microsoft Office 365 account, which had also been compromised at some point. SolarWinds was using build management and continuous integration server TeamCity provide…

Vulnerable dependencies are very different to compromised or backdoored dependencies though. Noone's taking over Solarwinds because their build tools had a ReDOS involving input from their own config files.

Re: Vulnerability reports are not special anymore

#117

Post by aetherspawn »

LLMs find more vulnerabilities than people because people time is heaps more expensive than LLM time, that’s it.

We’ve always been able to find heaps, we’ve just never had the right structures to put in the effort and renumerate people for looking (even if they don’t find anything).

Re: Vulnerability reports are not special anymore

#119

Post by kirici »

jerrythegerbil wrote: post #21

Vulnerability reports were never special. The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now. Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstratio…

This screams LLM to me and I couldn't bear to read past the second paragraph.

Re: Vulnerability reports are not special anymore

#120

Post by SkiFire13 »

jamesfinlayson wrote: post #51
themanmaran wrote: post #30

Earlier quoted context omitted.

Seriously. We got 116 github dependabot alerts this week. Half of them for dev dependencies.

I tried to raise that with my internal security team recently - don't clutter my vulnerability dashboard with issues in dev dependencies. They somewhat rightly pointed out that malware needs to be dealt even if it's a dev dependency. So my suggestion went nowhere because I guess we can't filter by type of vulnerability.

Everyone talking about malware in dev dependencies as if dependabot only raises issues about that, but it does not. It raises warnings about all sort of "vulnerabilities" irrespective of the threat model.

Even worse, it incentivizes randomly updating dependencies, which is what actually allows supply chain attacks.

Post reply on HN