Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

21–30 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#21
post #9
post #3

This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

If I remember correctly, we had to patch or provide justification for CVEs flagged by tools like AWS Inspector for SOC2 as well.

Re: SQLite Critical CVEs or LLM Slop?

#22
post #9

Earlier quoted context omitted.

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

ITAR

ITAR has no such hard requirements. Might be some orgs that tell themselves they're attempting this under ITAR but they're not doing it in any comprehensive way.

The only thing within ITAR that I'm aware of concerning itself with software supply chain is SP 800-218 requirements & that's just a load of open-to-interpretation weasel words about having CVE detection & automations in place & some defined plans for reducing the number of vulns. Pretty sure that component of it is even eligible for self-assessment.

Re: SQLite Critical CVEs or LLM Slop?

#23
post #9

Earlier quoted context omitted.

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

If I remember correctly, we had to patch or provide justification for CVEs flagged by tools like AWS Inspector for SOC2 as well.

So you didn't have to patch all of them.

Re: SQLite Critical CVEs or LLM Slop?

#24
post #17

Not validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.

This is what the Linux kernel is currently attempting since becoming a CNA. edit: "This" being to sabotage the CVE system via DoS, to be clear.

Re: SQLite Critical CVEs or LLM Slop?

#26
post #20

The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.

Edits for clarity: seems like this was a controversial take, which I find pretty interesting. I believe people took this comment as LLMs being better than security aware engineers who have the time to spend building solid systems. This wasn't the point, the reality outside "established" tech companies is that software security can be lackluster. There often simply isn't enough resources to check old software for basi…

Currently we either get AI promo vulnerability dumps like from Chrome with pretty graphs that no one checks or false positives.

Where is this one now that was hyped everywhere?

https://news.ycombinator.com/item?id=49133889

The GitHub submitter could no longer reproduce the issue and the LKML post has no replies:

https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...

Re: SQLite Critical CVEs or LLM Slop?

#28
post #13
post #9

Earlier quoted context omitted.

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

Many orgs (esp w ISO27000) have a vulnerability management policy that involves patching at least critical CVEs within a short timeline. Tools like trivvy make it possible to do the scans…

I've been in such an org, & I've led initiatives to set up automated detection at very large scale. We started by issuing tickets to teams to resolve CVEs within varying timelines - ranging from a 24hr fix to 6 months - connected to the CVSS score. It wasn't viable.

- Firstly, you quickly realise how irrelevant CVSS scores are - initiatives like First's EPSS are designed to fix this but they aren't there yet

- Secondly, you need to begin implementing localised heuristics to determine exploitable code paths. This has generally been incredibly difficult to do reliably - LLMs have started to make it easier, but it's expensive.

- Lastly, you need to factor in consideration of actionable remediation pathways. A dependency upgrade for critical infrastructure might contain breaking changes that take months to fix, or two competing CVEs might be present in interdependent versions of transitive dependencies in your sbom tree.

Most orgs aren't applying any of the above three filters to reduce their CVE remediation burden, & even if they are, it's still too high to make zero a viable target.

In reality, most orgs aren't doing comprehensive detection to begin with - if you haven't discovered all of your CVEs, your remediation burden is going to be a lot more manageable.

Re: SQLite Critical CVEs or LLM Slop?

#29
post #17

Not validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.

That is exactly why many big projects are migrating to becoming CNA, so that randos can’t get assigned unqualified CVEs which nobody has looked at or validated.

Apparently RedHat is a CNA of last resort, so it might be possible to get your project under Redhat’s scope and go through them without having to be a CNA yourself.

Re: SQLite Critical CVEs or LLM Slop?

#30
just ran this article that ran a cve through gptzero.. through gptzero

-GPTZero AI Detection

-Model 4.8b

-We are moderately confident this text is a mix of AI and human

-63/88 Sentences likely AI generated

Post reply on HN