Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

71–80 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#71

Post by [deleted] »

michaelt wrote: post #66
lucideer wrote: post #9

Earlier quoted context omitted.

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

SOC2 CC7.1 [1] requires a vulnerability scanner, findings tracked with tickets, assigned severities according to a documented risk-based system, severity-based SLAs for remediation, and that the SLAs mostly be complied with or have tracked exceptions. However it doesn't mandate any particular SLA, or the details of how risks are to be evaluated. Organisations get to write their own policy, and they don't need to comm…

[deleted]

Re: SQLite Critical CVEs or LLM Slop?

#72

Post by bluGill »

fabioborellini wrote: post #48
delfinom wrote: post #37

Earlier quoted context omitted.

It's been a problem for awhile. Daniel Stenberg has talked about it numerous times on his/curl's blog for the last 4 years. They became their own CNA to try and control it, they opened a hackerone with rewards, but now removed the rewards because it got flooded with AI generated slop daily. https://daniel.haxx.se/blog/2023/08/26/cve-2020-19909-is-eve... https://daniel.haxx.se/blog/2024/01/16/curl-is-a-cna/ https://da…

And this year, some of the reports suddenly got good and meaningful. https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/ So the agents started doing something useful after a period of filling mailing lists and bug bounties with slop. Sound good, but that's not entirely a good thing. The volume of good reports is a burden as well, and it's likely that long-lasting open source C/C++ projects have legitimate vuln…

The real question is will [when will] these vulnerabilities get fixed faster than they can be found. LLMs are finding a lot of real bugs that have existed without being noticed for 20+ years. It is generally expected [hoped] that in a short time all those 20 year old bugs will be fixed and LLMs will settle down to just things introduced in the last month (or perhaps this pull request) and thus be manageable.

Re: SQLite Critical CVEs or LLM Slop?

#74

Post by stabbles »

cogman10 wrote: post #56
flerchin wrote: post #39

Earlier quoted context omitted.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

This is what I truly don't like about the CVE system. One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It w…

In the Spack package manager [1] we're trying to work around this with directives:

    deprecated("@:1.3 +minizip", reason="cve", severity="high")
So, you can define conditional deprecation, and the dependency resolver can still pick up `zlib ~minizip` (i.e. with a variant/component disabled).

Deprecation on versions only isn't the right granularity.

[1]: https://github.com/spack/spack/pull/52372

Re: SQLite Critical CVEs or LLM Slop?

#75

Post by tantalor »

gste wrote: post #60

> The cited code didn't even exist in those versions or referenced unrelated logic. > When testing the PoC payloads they didn’t work (not triggering any crash). I think the future is pretty obvious, if this isn't being done on projects already: you need to automate these checks and reject automatically

Great idea! We can use LLM to do this.

LLM: I ran the check and it repro'd

> Did you really?

LLM: You're absolutely right. I didn't actually run the check. Good catch! One sec let me do that now... yep I ran the check and it definitely repro'd this time.

> I hate my job

Re: SQLite Critical CVEs or LLM Slop?

#76

Post by cogman10 »

post-it wrote: post #70
cogman10 wrote: post #56

Earlier quoted context omitted.

This is what I truly don't like about the CVE system. One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It w…

But could an attacker with access to zip tools exploit it to get a root shell? Probably not, but maybe.

Well that's the thing, to build the utilities required extra flags which were pretty rarely enabled. You can, in most distros, ultimately install minizip as a separate package but few do. It's not that useful of a utility. Most people will likely just grab the full blown "zip" application https://infozip.sourceforge.net/Zip.html . Though on linux, even more people are simply using tar with a zlib extension. That's the `tar.gz` files.

Re: SQLite Critical CVEs or LLM Slop?

#77

Post by dncornholio »

cogman10 wrote: post #56
flerchin wrote: post #39

Earlier quoted context omitted.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

This is what I truly don't like about the CVE system. One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It w…

If the solution was to remove an unused dependency, I think this CVE isn't silly at all.

Re: SQLite Critical CVEs or LLM Slop?

#78

Post by regularfry »

lucideer wrote: post #9
inigyou wrote: post #3

This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

Any org large enough to have separated the people responsible for the security exposure of the organisation from the developers with familiarity of what's deployed is likely to have done exactly this.

The thing you have to remember is that CVEs can be a) scanned for without exerting mental effort, and b) counted.

Re: SQLite Critical CVEs or LLM Slop?

#79

Post by inigyou »

masklinn wrote: post #29
Ekaros wrote: post #17

Not validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.

That is exactly why many big projects are migrating to becoming CNA, so that randos can’t get assigned unqualified CVEs which nobody has looked at or validated. Apparently RedHat is a CNA of last resort, so it might be possible to get your project under Redhat’s scope and go through them without having to be a CNA yourself.

What are the requirements to become a CNA?

Re: SQLite Critical CVEs or LLM Slop?

#80

Post by ptx »

vrighter wrote: post #64
lucideer wrote: post #9

Earlier quoted context omitted.

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

I have been given a list by security. "We had an automated tool scan that machine. It reported these. Fix anything medium severity and above. Never mind that some of them involved vulnerabilities in some part of the bluetooth stack (servers in our datacenter don't even have bluetooth). But they just didn't care

This does make some sense if it's considered a valid fix to document that you have verified that Bluetooth is disabled on the servers and therefore not vulnerable. But that assumes that the scanning tool can be told about this kind of fix, so that it stops warning about it, which I guess it might not.
Post reply on HN