The federal government ought to be funding NIST sufficiently to actually do sufficient analysis. Do we care about funding civil "cyber" defense or not?
(Obviously the answer is not, we only care about funding offensive capabiltiies).
SQLite Critical CVEs or LLM Slop?
211–220 of 406 posts
Re: SQLite Critical CVEs or LLM Slop?
#212They couldn't even be bothered to use a non-AI image.
that's fine, I'm not sure they bothered to actually write the article either
Re: SQLite Critical CVEs or LLM Slop?
#213We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…
[flagged]
> We are not going back, period.
I didn't get this at all from the parent. They're simply stating that LLMs aren't entirely trustworthy, and that the responsibility is ultimately ours, not the LLM's.
Re: SQLite Critical CVEs or LLM Slop?
#214The duality of LLMs is real. On one hand, they are solving unsolved math problems, on the other hand they get fooled by the strawberry test or the car wash test.
We see this all the time on reddit and HN, where some users are building crazy things with LLM and others are swearing by how useless LLMs are. Some people claim SWE will be dead in a few years, and others are claiming the AI bubble is going to burst anytime now.
Re: SQLite Critical CVEs or LLM Slop?
#215We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…
Every day I wake up and open HN. “LLM has made legitimate mathematical discoveries” —> Wow the rate of progress is amazing. Highly upvoted. “LLM does something not good” -> Does everyone else not realize LLMs are just dumb next token predictors? Highly upvoted. So tired of this discourse and this site.
The rate of progress can be high and they can also be dumb next token predictors. Not sure why that is hard to understand.
These models can do a lot of things but they also can't do a lot of things. In order to use these models effectively you have to understand that they are next token predictors and how that allows it to do what they do.
Re: SQLite Critical CVEs or LLM Slop?
#216It really is a shame that funding cuts at NIST has led directly to this. I don't know if there is an incentive for a non-profit to come out of the woods to do the validation and enrichment process, but boy do we need it. Like gortok has said, LLMs are just probabilistic text-prediction engines. They cannot automate the vulnerability validation lifecycle because they cannot account for improbable edge cases. They may…
Funding cuts at NIST did not lead directly to this. You must be new here or haven’t spent a lot of time directly interfaced with NIST on cybersecurity in decades past. And NIST’s role with CVE’s has always been purely ministerial/clerical. It has never been their job to do even a cursory investigation into the vulnerability itself.
OP suggests something at NIST changed in 2024, where they stopped doing as much verification as they did before.
To be sure, the suggestion is not funding cuts, but an increase in workload with same funding.
Here is the 2024 NIST announcement OP links to: https://nvd.nist.gov/general/news/nvd-program-transition-ann...
> Currently, we are prioritizing analysis of the most significant vulnerabilities. In addition, we are working with our agency partners to bring on more support for analyzing vulnerabilities and have reassigned additional NIST staff to this task as well.
The OP described this as "NIST effectively hit pause on deep analysis. "
It does sound like they stopped doing something they used to do in 2024. I personally have definitely not spent a lot of time directly interfaced with NIST on cybersecurity in decades past, I know nothing about it, just what I read in OP.
Are you saying the OP was wrong to call what NIST used to do "deep analysis", and/or that the thing NIST stopped doing was "purely ministerial/clerical" in a way that it would not have caught fake reports anyway, contradicting the OP? Or other?
Again, to be sure, the OP's suggestion was not that this was caused by NIST funding cuts, but by "a massive surge in vulnerability reports,"
Re: SQLite Critical CVEs or LLM Slop?
#217The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult. But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.
> I do know that LLMs have been discovering a lot of legit CVEs, And that's great! Now the human in the loop can verify the accuracy of each finding and take responsibility for moving it forward. > and I will lay odds that the blackhats are leveraging them to the max. We just need more humans in the loop to do that more quickly, each putting their own reputation on the line with each CVE they determine to be legit. A…
We can shoot for 100% accountability by demanding 100% accuracy.
Of course what will really happen is anyone looking for problems will stop going to the source for fixes and will start gaining cred in the dark markets of exploits and problems, which is better for everyone, right?
Just because someone is using AI doesn't mean they didn't find a bug, it's more likely they just have no means of determining if it's a real bug or not, so to the maintainers tickets seem random. The problem comes in when clever attackers can take these untrustworthy tickets and run them through another LLM that determines if an exploit is possible and then uses the attacks it finds before anyone has a chance to patch.
Re: SQLite Critical CVEs or LLM Slop?
#218Earlier quoted context omitted.
Even more than that, a lot of vulnerabilities require untrusted input to exploit, but the library or executable is usually only used with trusted input. For example, suppose the chacha20 function had a vulnerability that was exploitable with a malicious secret key, but you only call it with your own, definitely not malicious, key. The vulnerability doesn't impact you, but your security policy might still require you…
I have concluded that even though I won't use that key, I still want the issue fixed. You never know if the key I randomly choose next year is the vulnerable one. Worse, you never know if someone else won't make a bug that allows outsiders to select a key...
But it doesn't warrant an emergency automatic update.
Re: SQLite Critical CVEs or LLM Slop?
#219Re: SQLite Critical CVEs or LLM Slop?
#220Earlier quoted context omitted.
> The vast majority of CVEs are not exploitable, basically noise. This is not true if you consider security-in-depth. Many of them are exploitable on their own but maybe not in combination with other issues that are as yet unknown or known but not patched everywhere. As a simple example a local privilege escalation issue that is not exploitable on a device that only I ever have access to, essentially becomes a remote…
This is the most common "not exploitable" CVE I have to deal with at work all the time. Things like buffer overruns when passing headers, but my server shouldn't be accessible to the public Internet anyway, and if it is, then we have much bigger problems. So yes it's technically exploitable IF other safeguards fail, but the reason it's irrelevant is that if those other safeguards fail, we are in deep shit anyway.
It's been proven over and over and over and over and over and over and over and over and over and over and over ......
that requiring access to a privileged network is NOT a sufficient security boundary! Someone you don't like always somehow ends up getting access to the network.