Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

191–200 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#191

Post by pixl97 »

ddevnyc wrote: post #117
cogman10 wrote: post #76

Earlier quoted context omitted.

Well that's the thing, to build the utilities required extra flags which were pretty rarely enabled. You can, in most distros, ultimately install minizip as a separate package but few do. It's not that useful of a utility. Most people will likely just grab the full blown "zip" application https://infozip.sourceforge.net/Zip.html . Though on linux, even more people are simply using tar with a zlib extension. That's th…

I think this presents a highly fascinating scenario: 1. A vulnerability that presents itself very rarely 2. If it presents itself, it is fatal to the organization So, few targets are affected, but the outsized effect means that the overall economic impact is not insignificant. How does one keep track of that? One method is to use a sort of "kirchhoff's law" approach, similar to google page rank. The electrical analog…

The hard part is accurately making that flow chart. You have your software composition you have to keep accurate. Your software configuration and it's changes. Environmental systems and its changes like VM hosts and network hardware. And network pathing and firewalling.

When does something become a ground can be very difficult to figure out with the above complexity.

Re: SQLite Critical CVEs or LLM Slop?

#192

Post by stronglikedan »

ChrisMarshallNY wrote: post #4

The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.

> I do know that LLMs have been discovering a lot of legit CVEs,

And that's great! Now the human in the loop can verify the accuracy of each finding and take responsibility for moving it forward.

> and I will lay odds that the blackhats are leveraging them to the max.

We just need more humans in the loop to do that more quickly, each putting their own reputation on the line with each CVE they determine to be legit.

AI doesn't stand for "accountability irrelevant", even though most people act like it does.

Re: SQLite Critical CVEs or LLM Slop?

#193

Post by pixl97 »

PunchyHamster wrote: post #87
post-it wrote: post #70

Earlier quoted context omitted.

But could an attacker with access to zip tools exploit it to get a root shell? Probably not, but maybe.

if the tool isn't ran it can't be exploited CVE should just be far more granular instead of flagging alert for anything using zlib

And when you chain another weak CVE to run the tool in a method that would have previously been a noop?

Security can be a major pain in the ass, and it's pretty often we see CVEs that were low valued suddenly become more urgent when someone finds a better way to use them as an exploit.

Re: SQLite Critical CVEs or LLM Slop?

#194

Post by wildfireday2 »

umarcyber wrote: post #190

It really is a shame that funding cuts at NIST has led directly to this. I don't know if there is an incentive for a non-profit to come out of the woods to do the validation and enrichment process, but boy do we need it. Like gortok has said, LLMs are just probabilistic text-prediction engines. They cannot automate the vulnerability validation lifecycle because they cannot account for improbable edge cases. They may…

Funding cuts at NIST did not lead directly to this. You must be new here or haven’t spent a lot of time directly interfaced with NIST on cybersecurity in decades past.

And NIST’s role with CVE’s has always been purely ministerial/clerical. It has never been their job to do even a cursory investigation into the vulnerability itself.

Re: SQLite Critical CVEs or LLM Slop?

#195

Post by ivan_gammel »

gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

You are right with the analysis, but wrong with the conclusions. Yes, LLM „thinking process“ is kinda non-deterministic in a sense that it does not follow logical reasoning and will not produce logically correct results in 100% cases. It has an error margin.

However, error margins are in the center of any engineering discipline. We cannot produce things measured with 100% accuracy. This is accepted fact. The focus is always not on eliminating errors, but on reducing them to acceptable minimum. With LLMs we should not expect an ideal logical thinker, but a process that may error sometimes, and we must design quality controls instead that push LLM outputs within acceptable margins. And it can work.

Re: SQLite Critical CVEs or LLM Slop?

#196

Post by pixl97 »

stackghost wrote: post #150
emj wrote: post #103

Earlier quoted context omitted.

You do not run it as root, all you get is the possibility to crash or execute code. Nodes that handle such things are all heavily locked down.

>all you get is the possibility to [...] execute code Anyone who's ever done even a beginner CTF knows that achieving code execution is a big deal.

Yep, once you have code execution you find a local privilege escalation and go from there.

Re: SQLite Critical CVEs or LLM Slop?

#197

Post by budsniffer952 »

gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

[flagged]

Re: SQLite Critical CVEs or LLM Slop?

#198

Post by gbnwl »

gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Every day I wake up and open HN.

“LLM has made legitimate mathematical discoveries” —> Wow the rate of progress is amazing. Highly upvoted.

“LLM does something not good” -> Does everyone else not realize LLMs are just dumb next token predictors? Highly upvoted.

So tired of this discourse and this site.

Re: SQLite Critical CVEs or LLM Slop?

#199

Post by pixl97 »

brabel wrote: post #95
jmull wrote: post #91

Earlier quoted context omitted.

Yes, the CVE system is broken. LLM-generated CVEs is making it even worse. I suppose all these fake issues and the many more that have absurdly elevated severities could be considered an attack on the system itself, stripping it of credibility.

That's the worst case scenario right there: so many fake reports that the real vulnerabilities can no longer be reported properly and fixed quickly. Assuming some LLMs really are capable of finding serious vulnerabilities (which in my experience is absolutely the case : just let a good model run on your security code base and see for yourself) that means that we can more easily find vulnerabilities, but it's much har…

Having your LLM look at the security really isn't a great option. Most users don't have unlimited budgets and can't dump the necessary time and funds needed to find these flaws.

On top of that, ability to prompt about flaws makes all the difference in the world. If you know what you're doing you can get better results. Problem, very few people know what they are doing.

Going back to the money/time problem, LLMs are not deterministic. Much like the day you forgot to drink coffee, running LLM systems against code can get different results where things are found or not found. That or two different things are found on two different runs (with the same code) because attention was stolen by the first issue it found.

So yea, in the LLM age proof is in the exploit pudding. Who cares if you're taken seriously, if you dump out an exploit that works the seriousness is now on the defenders time and people will scramble to fix it.

"But what about exploit embargos"... Well, you just found the exploit with an LLM, how many other nefarious groups do you think are already exploiting it? The age of sitting on problems is dead.

Re: SQLite Critical CVEs or LLM Slop?

#200

Post by kentm »

ivan_gammel wrote: post #195
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

You are right with the analysis, but wrong with the conclusions. Yes, LLM „thinking process“ is kinda non-deterministic in a sense that it does not follow logical reasoning and will not produce logically correct results in 100% cases. It has an error margin. However, error margins are in the center of any engineering discipline. We cannot produce things measured with 100% accuracy. This is accepted fact. The focus is…

Yes but the key here is doing proper risk assessment. "What is the consequence if the LLM gets this wrong?" "How do we verify the output?" "What are the legal ramifications for using the LLM in this way?" "Who is responsible when the LLM fails?" "Whats the expected accuracy here?" etc.

In the current AI mania, there's a lot of due diligence simply being ignored. Plenty of "Well humans make mistakes too!" going on here on HN too.

Post reply on HN