Live data from Hacker News

Vulnerability reports are not special anymore

words.filippo.io

71–80 of 249 posts

Re: Vulnerability reports are not special anymore

#71

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.

> That supposes that LLMs can write secure software.

I think we're at the point that the best LLMs can indeed write software that's far more secure than your average programmer. Partly because the average is so terrible.

Re: Vulnerability reports are not special anymore

#72

I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…

Its been like that for half a decade across all software. People act like finding a linux kernel bug is a big deal, completely ignoring the fact that in order to exploit that bug, the attacker has to be able to run code on your computer in the first place, which is extremely hard to do these days remotely.

Also people ironically just DGAF that much. The last actual bad exploit was log4shell in java, which given how it was introduced (i.e someone purposefully at Apache made it so a log statement can execute code, and nobody questioned it before pushing it to prod), should have been the signal for everyone to completely remove all Apache libraries from their services, but yet all the software is still being used.

Re: Vulnerability reports are not special anymore

#73

I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…

Yeah, I help review security reports for a small FOSS organization, and someone reported a "critical" vulnerability about a publicly-accessible SVN server. Like yes, that is indeed the purpose of hosting open source software. But at least that report was obviously bogus; much worse are the ones that look legitimate at first, so you have to read through dozens of AI-generated paragraphs to make sure that there's nothing valid hidden in there.

Re: Vulnerability reports are not special anymore

#74
post #6

I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…

I'm getting CVE fatigue with all of these super ultra critical 10/10 vulnerabilities that are some node package that compiles my frontend can get stuck if I give it a malicious regex. It's hard to spot the stuff that actually matters.

The common thread of late really seems to be the node ecosystem

Re: Vulnerability reports are not special anymore

#75

I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…

Its been like that for half a decade across all software. People act like finding a linux kernel bug is a big deal, completely ignoring the fact that in order to exploit that bug, the attacker has to be able to run code on your computer in the first place, which is extremely hard to do these days remotely. Also people ironically just DGAF that much. The last actual bad exploit was log4shell in java, which given how i…

These bugs are indeed important, you need them once you‘ve found a bug in an application.

Re: Vulnerability reports are not special anymore

#76

I feel like the current situation is temporary. LLMs are finding all the bugs. LLMs are also help fixing most of the bugs. Once most of the bugs are fixed, LLMs should be good at finding bugs before shipping them, the stream of bug reports will die down, and we'll be back to vulnerabiltiy reports being special. Further, the fact that bugs are so easy to find by LLMs means there is strong incentives to find ways to mi…

That supposes that LLMs can write secure software. Also, if we assume that finding bugs is easier that not creating them (reasonable I would say), the supply of bugs will never be exhausted.

What's the difference between finding bugs and not making? Just run the bug finding in during CICD.

Re: Vulnerability reports are not special anymore

#77

I feel like it's also been overrun by a lot of spam. As someone running a company, I get 2-5 unsolicited "vulnerability reports" per week. Half of them are an LLM finding some bad CSS on our framer splash page. The other half I assume are an extortion attempt so we just mark as spam. Occasionally I see real security researchers on HN complaining that no one takes the disclosure seriously, or that people reply immedia…

Yeah, I help review security reports for a small FOSS organization, and someone reported a "critical" vulnerability about a publicly-accessible SVN server. Like yes, that is indeed the purpose of hosting open source software. But at least that report was obviously bogus; much worse are the ones that look legitimate at first, so you have to read through dozens of AI-generated paragraphs to make sure that there's nothi…

I use AI to read such emails!

Re: Vulnerability reports are not special anymore

#79

Earlier quoted context omitted.

Seriously. We got 116 github dependabot alerts this week. Half of them for dev dependencies.

I tried to raise that with my internal security team recently - don't clutter my vulnerability dashboard with issues in dev dependencies. They somewhat rightly pointed out that malware needs to be dealt even if it's a dev dependency. So my suggestion went nowhere because I guess we can't filter by type of vulnerability.

Working in the EU energy sector where we have to work with NIS2 compliance, I'd argue that your security team rightly pointed it out. I suspect that's what you mean though, and the rightly is just there because you agree with it but don't like it. We work with even more tight dependencies policies than just having alerts. We have a set of pre-approved and yearly vetted packages, like pandas or pyarrow for Python data work. Aside from that we have some isolated development environments where your pipeline can get access to something like SQLC for Go. Which is essentially where your dev dependency lives in it's own environment where it can produce the code it needs to and then submit it for approval into your regular dev environment.

Ironically we'd probably need to run Dependabot itself in a mirrored environment since it too has external dependencies we'd probably not want to vet.

I do think external dependencies are among our biggest security threats though. It's so hard to vet them, and compliance basically comes down to "We trust the apache software foundation enough, and pyarrow is vital to our business, so we accept the risks", and then you lock versions and aren't the first to update except for vulnerabilities. Shadow AI is obviously the number one security threat right now, especially in enterprise with people who are very tech savvy. This makes dependencies so much worse though, because now everyone can (if their systems aren't locked down tight) do so many crazy things. Both with the "non-sanctioned" AI but also with the code it can generate for them.

Re: Vulnerability reports are not special anymore

#80

Earlier quoted context omitted.

I tried to raise that with my internal security team recently - don't clutter my vulnerability dashboard with issues in dev dependencies. They somewhat rightly pointed out that malware needs to be dealt even if it's a dev dependency. So my suggestion went nowhere because I guess we can't filter by type of vulnerability.

Dev dependencies is how they compromised SolarWinds and thereby most of the US federal government. > The attackers used a supply chain attack. The attackers accessed the build system belonging to the software company SolarWinds, possibly via SolarWinds's Microsoft Office 365 account, which had also been compromised at some point. SolarWinds was using build management and continuous integration server TeamCity provide…

This is very real, but such CVEs are such a tiny fraction in relation to denial-of-service-due-to-regex that it’s hard to take the system seriously.

So far as I’m concerned the solution is to isolate everything as much as possible. I’d love to see something on the CVE classification side to also address the signal to noise problem but I don’t see it happening.

Post reply on HN