Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

181–190 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#181

Post by mandevil »

groundzeros2015 wrote: post #164
cogman10 wrote: post #56

Earlier quoted context omitted.

This is what I truly don't like about the CVE system. One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It w…

Also a buffer overflow in user space for a tool not serving internet traffic is not a big deal.

I would be careful with this line of thought: opening a malformed archive that gets you into user space is often the first step in a chain-of-attacks that ends up winning Pwn2Own, so I think that a 8-ish score makes sense for it. It won't be enough to do much on its own, but for the past decade or so it's been all about chaining enough small things together to get root, rather than single points of failure.

I keep feeling like the entire CVE system is creaking and about to fail- the signal-to-noise on that has gone so far down, the gate-keeper function that MITRE et al are supposed to be providing clearly has fallen apart, but we do need some way of tracking problems and alerting on bugs in commonly reused libraries. So I can see the problems with the current system, I just can't envision the new, better system to build to replace it.

Re: SQLite Critical CVEs or LLM Slop?

#183

Post by bwfan123 »

gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

> Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this usage of “AI”.

Brandolini's principle in action. It takes 10 times more energy to refute BS than to generate it. A related analogy to computing: it is easy to generate propositions, but hard to test if a given proposition is satisfiable or not, which curiously ties to P vs NP.

Re: SQLite Critical CVEs or LLM Slop?

#184

Post by buttercraft »

red75prime wrote: post #165
gortok wrote: post #151

Earlier quoted context omitted.

Turing-completeness is a necessary pre-requisite for being able to fulfill the requirements of a Turing machine, nothing more. In the same way that cell division is a necessary condition for life, but cell division does not mean a given life form itself is sentient. Intelligent life-forms can generate probabilistic outputs based on inputs, but being able to generate probabilistic outputs based on inputs is not what m…

OK. A more pointed question. What do you know about intelligence that allows you to exclude LLMs with CoT from the category of intelligent systems with certainty?

How do you know there's not a teapot orbiting the sun?

Re: SQLite Critical CVEs or LLM Slop?

#185

Post by pixl97 »

SkyBelow wrote: post #128
cogman10 wrote: post #56

Earlier quoted context omitted.

This is what I truly don't like about the CVE system. One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It w…

Isn't the real difficulty in how vulnerabilities can be mixed? A given vulnerability might be extremely hard to execute, but very damaging. Another vulnerability might be easier to execute, but it can't do much. But if the second one can be used to trigger the first one, you then have attack surface area of the second vulnerability with the damage of the first one. Even if some individual case can be shown to be safe…

Yea, this is where the problems start coming in.

In testing with LLMs with good exploit finding capabilities and a lack of guardrails on writing exploits, quite often LLMs will chain together a surprising stack of exploits to get what they want. They'll get access via a weak, but limited user. They'll search around laterally until they find something else. For example User A (weak exploit) -> User B -> User B bad configuration -> root (or application takeover).

They don't get bored like humans and they can be more robust than a set of scripts by far in catching errors when their scripts don't work or interesting things show up in their findings. The huggingface break down is a good example of just how much lateral movement an LLM can try with enough processing power behind it.

This is where the security onion gets messy. It can be hard to predict when compromising one layer actually bypasses many other layers of your security.

Re: SQLite Critical CVEs or LLM Slop?

#186

Post by pessimizer »

estearum wrote: post #162
gortok wrote: post #151

Earlier quoted context omitted.

Turing-completeness is a necessary pre-requisite for being able to fulfill the requirements of a Turing machine, nothing more. In the same way that cell division is a necessary condition for life, but cell division does not mean a given life form itself is sentient. Intelligent life-forms can generate probabilistic outputs based on inputs, but being able to generate probabilistic outputs based on inputs is not what m…

> but being able to generate probabilistic outputs based on inputs is not what makes us intelligent. ??? Of course it is. The brain is mechanically not capable of doing anything other than that. Do you believe the brain is something other than a bundle of probabilistic physical interactions? Or are brains not the source of what we call intelligence?

Yours is a controversial view. It is lazy and selfish to try to get other people to explain their case that it is not exclusively that, when saying that it is exclusively that is the weaker case, and you back it up with nothing but a snarky proclamation.

Are newly born babies reacting due to statistical probabilities that they have derived, or are they using something other than their brains?

Re: SQLite Critical CVEs or LLM Slop?

#187

Post by Joe_Cool »

elmer2 wrote: post #168
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Many people with no skills are taking advantage of the LLM craze to artificially inflate their own value. I see it every day on LinkedIn. People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

I heard the term "Sloperator" a few times. Quite fitting...

Re: SQLite Critical CVEs or LLM Slop?

#188

Post by beepbooptheory »

estearum wrote: post #162
gortok wrote: post #151

Earlier quoted context omitted.

Turing-completeness is a necessary pre-requisite for being able to fulfill the requirements of a Turing machine, nothing more. In the same way that cell division is a necessary condition for life, but cell division does not mean a given life form itself is sentient. Intelligent life-forms can generate probabilistic outputs based on inputs, but being able to generate probabilistic outputs based on inputs is not what m…

> but being able to generate probabilistic outputs based on inputs is not what makes us intelligent. ??? Of course it is. The brain is mechanically not capable of doing anything other than that. Do you believe the brain is something other than a bundle of probabilistic physical interactions? Or are brains not the source of what we call intelligence?

Why is the brain probabilistic instead of deterministic?

Re: SQLite Critical CVEs or LLM Slop?

#189

Post by tadfisher »

estearum wrote: post #162
gortok wrote: post #151

Earlier quoted context omitted.

Turing-completeness is a necessary pre-requisite for being able to fulfill the requirements of a Turing machine, nothing more. In the same way that cell division is a necessary condition for life, but cell division does not mean a given life form itself is sentient. Intelligent life-forms can generate probabilistic outputs based on inputs, but being able to generate probabilistic outputs based on inputs is not what m…

> but being able to generate probabilistic outputs based on inputs is not what makes us intelligent. ??? Of course it is. The brain is mechanically not capable of doing anything other than that. Do you believe the brain is something other than a bundle of probabilistic physical interactions? Or are brains not the source of what we call intelligence?

This deserves about as large of a "[citation needed]" as one could draw. Are you well-studied in neuroscience?

Re: SQLite Critical CVEs or LLM Slop?

#190

Post by umarcyber »

It really is a shame that funding cuts at NIST has led directly to this. I don't know if there is an incentive for a non-profit to come out of the woods to do the validation and enrichment process, but boy do we need it.

Like gortok has said, LLMs are just probabilistic text-prediction engines. They cannot automate the vulnerability validation lifecycle because they cannot account for improbable edge cases. They may not even get the version of the software right...

Post reply on HN