Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

91–100 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#91

Post by jmull »

flerchin wrote: post #39
ChrisMarshallNY wrote: post #4

The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

Yes, the CVE system is broken. LLM-generated CVEs is making it even worse.

I suppose all these fake issues and the many more that have absurdly elevated severities could be considered an attack on the system itself, stripping it of credibility.

Re: SQLite Critical CVEs or LLM Slop?

#92

Post by ptx »

mahogany wrote: post #50
awkwardpotato wrote: post #38

Earlier quoted context omitted.

How so? All 6 of the CVEs covered in the article did not actually exist when investigated

The comment is making fun of a Claude-ism where it becomes super “honest” about stuff. It’s a joke but there is an underlying real effect where this type of language is psychologically manipulative and I would guess makes people believe LLMs output more than if it didn’t use “honest” (or “load bearing” or whatever super serious important sounding word).

Or maybe they didn't train it that way to be manipulative (although it's certainly a plausible explanation) but simply as an accidental artifact of trying to make it give honest answers?

LLM-generated images sometimes includes text from the prompt as literal text in the image, so perhaps this is the same sort of artifact? If they've told it to be honest, it responds by talking about being honest instead of actually being honest, because it has no actual understanding of anything.

Re: SQLite Critical CVEs or LLM Slop?

#93

Post by swiftcoder »

lucideer wrote: post #9
inigyou wrote: post #3

This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

> I'm very curious what organisations would have such a policy

Do you provide SOC2, HIPAA, GDPR, or similar certifications to your b2b customers? Then your tech stack undergoes an annual audit, and in your audit you will need to provide a paper trail for every single vulnerability in your stack.

In practice, this means that your audit compliance software (something like Vanta.com) is going to be setup to mandate every CVE in the whole stack is patched within SLA.

Re: SQLite Critical CVEs or LLM Slop?

#94

Post by dspillett »

flerchin wrote: post #39
ChrisMarshallNY wrote: post #4

The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

> The vast majority of CVEs are not exploitable, basically noise.

This is not true if you consider security-in-depth. Many of them are exploitable on their own but maybe not in combination with other issues that are as yet unknown or known but not patched everywhere.

As a simple example a local privilege escalation issue that is not exploitable on a device that only I ever have access to, essentially becomes a remote root access flaw if you have untrusted or unreliable users (clients with accounts for instance) on the system. This works on a finer grain too, seemingly minor issues spread through the kernel and user space can add up to a serious exploit.

Re: SQLite Critical CVEs or LLM Slop?

#95

Post by brabel »

jmull wrote: post #91
flerchin wrote: post #39

Earlier quoted context omitted.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

Yes, the CVE system is broken. LLM-generated CVEs is making it even worse. I suppose all these fake issues and the many more that have absurdly elevated severities could be considered an attack on the system itself, stripping it of credibility.

That's the worst case scenario right there: so many fake reports that the real vulnerabilities can no longer be reported properly and fixed quickly. Assuming some LLMs really are capable of finding serious vulnerabilities (which in my experience is absolutely the case : just let a good model run on your security code base and see for yourself) that means that we can more easily find vulnerabilities, but it's much harder to report them and be taken seriously - so the incentive to exploit such vulnerabilities instead has become much, much higher.

Re: SQLite Critical CVEs or LLM Slop?

#96

Post by insanitybit »

inigyou wrote: post #51
insanitybit wrote: post #24

Earlier quoted context omitted.

This is what the Linux kernel is currently attempting since becoming a CNA. edit: "This" being to sabotage the CVE system via DoS, to be clear.

What do you mean? It isn't a DoS to assign every single bug fix a CVE! Every single bug is making someone vulnerable in some way.

That's nonsense. The goal is and always has been to undermine the CVE system because Greg does not believe in the system. The kernel is the only project that marks CVEs to fixes, it's the only project that considers every bug to be a "Security" bug, and it has a multi-decades long history of telling the security world to fuck off - all consistent with what I've said.

No one else has the process that the kernel has, despite plenty of people having software that's deployed in very similar ways.

There's zero question - this is ideologically motivated, not a genuine good-faith attempt to leverage the system.

Re: SQLite Critical CVEs or LLM Slop?

#97

Post by unknown_user_84 »

cogman10 wrote: post #56
flerchin wrote: post #39

Earlier quoted context omitted.

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

This is what I truly don't like about the CVE system. One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It w…

reminds me of the cvss scoring system. that thing is great at telling a story. for better or worse XD.

Re: SQLite Critical CVEs or LLM Slop?

#98

Post by hoppp »

ChrisMarshallNY wrote: post #4

The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.

Basically nothing is real nowadays unless you verify it for yourself.

Re: SQLite Critical CVEs or LLM Slop?

#99

Post by CodesInChaos »

ymir_e wrote: post #84
bluGill wrote: post #57

Earlier quoted context omitted.

All the big maintainers I know of disagree. Curl for example is getting overwhelmed with real issues that LLMs are finding. A year ago most LLM found issues where false positives (the code is correct), or hallucinations (the code in question doesn't even exist) - but that has changed and now LLMs are finding real issues in code. Many different projects have reported the same rules.

Absolutely. I'm not sure how universal this is, but I saw George Hotz tweeted something about Mythos not being such a big deal since he's able to find security issues without a claimed $20,000 in token spend per vulnerability. The reality is that humans aren't perfect, so we do not make "perfect programs". It is also possible to test it yourself to see how good LLMs are at finding software issues: Find any open sourc…

I'm sure if somebody hired him for a pentest it'd cost more per vulnerability found.

Re: SQLite Critical CVEs or LLM Slop?

#100

Post by jmull »

lucideer wrote: post #9
inigyou wrote: post #3

This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

It's quite common in enterprisey environments.

For one thing, bigcorps in regulated areas like it a lot. They push hard to get it required by the regulations (in practice if not directly). Although it's quite inefficient, it becomes a regulatory moat. A cost they can bear that potential upstart competitors cannot.

Post reply on HN