Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

271–280 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#271

Post by gspr »

adjfasn47573 wrote: post #243
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

> A human must verify everything an LLM presents as fact. Everything. I've thought about this for quite some time now. No. A human doesn't need to verify everything. And the argument is really simple: stochastic. Think of self-driving cars: We can show today - based on evidence and real data - that self-driving cars are safer than human drivers. That's a fact and the consequences are clear, more self-driving cars, le…

This reduction of everything to stochasticity is silly. Or, to put it differently: Do you accept a value with some error appearing in your bank account on salary day?

We have plenty of systems where complete accuracy is the only acceptable thing. Computers are great for such things. Until we all get caught up in a way of delusion and start writing those systems as natural prose passed through an improperly understood stochastic machine.

Re: SQLite Critical CVEs or LLM Slop?

#272

Post by ivan_gammel »

kentm wrote: post #200
ivan_gammel wrote: post #195

Earlier quoted context omitted.

You are right with the analysis, but wrong with the conclusions. Yes, LLM „thinking process“ is kinda non-deterministic in a sense that it does not follow logical reasoning and will not produce logically correct results in 100% cases. It has an error margin. However, error margins are in the center of any engineering discipline. We cannot produce things measured with 100% accuracy. This is accepted fact. The focus is…

Yes but the key here is doing proper risk assessment. "What is the consequence if the LLM gets this wrong?" "How do we verify the output?" "What are the legal ramifications for using the LLM in this way?" "Who is responsible when the LLM fails?" "Whats the expected accuracy here?" etc. In the current AI mania, there's a lot of due diligence simply being ignored. Plenty of "Well humans make mistakes too!" going on her…

The due diligence not being done is people putting cats in microwaves. It‘s not the dangerous part. The real danger is risk assessments coming to wrong conclusions, because it is still terra incognita. Talented engineers were in this situation before, doing mistakes with cars, airplanes, buildings etc.

Re: SQLite Critical CVEs or LLM Slop?

#273

Post by Plutoberth »

jrochkind1 wrote: post #211

The federal government ought to be funding NIST sufficiently to actually do sufficient analysis. Do we care about funding civil "cyber" defense or not? (Obviously the answer is not, we only care about funding offensive capabiltiies).

This is obviously impractical. With the volume of bug reports that are generated and such a wide breadth of software no single agency will be able to handle all reports. And honestly? It doesn't matter, even today. CVE should serve as a reasonably deduped identifier of specific vulnerabilities. It was never interesting and practical to care about 100% of vulnerabilities in a specific deployment, and it's not interesting today.

Re: SQLite Critical CVEs or LLM Slop?

#274

Post by pixl97 »

icedchai wrote: post #242
SoftTalker wrote: post #178

Earlier quoted context omitted.

So run apt full-upgrade and get the new bluetooth driver. Why bother with a fight over something that isn't even used? Just do the quickest thing to get it off your plate.

This might cause other problems, problems of the "if it's not broken, don't fix it" variety. Upgrading everything only to break something else, in a previously stable configuration, isn't worth it.

Really the days of "Lets run this stable configuration forever" are gone.

Getting rid of as much stuff in your OS and software stack as possible should be the security teams ultimate goal, so you have less to upgrade in the end. But actual security updates just come out at a tremendous rate, and you need a QA system that checks as much as it can before prod is upgraded.

Re: SQLite Critical CVEs or LLM Slop?

#275

Post by inigyou »

insanitybit wrote: post #134
inigyou wrote: post #116

Earlier quoted context omitted.

Or maybe it's ideologically motivated that other groups only identify certain bugs as security bugs?

Make a case then. My case is that Greg has, for decades, said that he hates the CVE system and rejected it and that the kernel objectively labels CVEs based on fixes (not the standard at all, CVEs are for vulns), etc. I can point to so many objective pieces of evidence to support my claim. You seem to have a vague, leading question. If you want to say something about "other groups" or ideologies, go ahead.

Malicious compliance is compliance.

Re: SQLite Critical CVEs or LLM Slop?

#276

Post by gbnwl »

unknownfuture wrote: post #261
gbnwl wrote: post #254

Earlier quoted context omitted.

Fair so let me be clear. I’m whining because the “next token predictor” reductionist point of view has been wrong and is only growing more wrong with time. Clearly these things can do things that actually matter. Do you disagree?

Even now you're engaging in this discussion as though I'm trying to litigate your point and that somehow forcing me to concede is, what, winning? I don't know. I get the impression you want me to concede that the particular points of view you disagree with aren't worthy of representation here on HN. I'm not going to do that.

I just prefer HN comments to be better reflections of reality. There is an unspoken expectation here that people here know what they’re talking about especially when it comes to technical matters. The rise of LLMs has given way to a HN branded populism that willingly denies reality as well. “Next token predictor” truthism is just so dumb and completely ignores the reality of what these tool are able to do. Smash that upvote button every time it feels good if you want but it’s just a meaningless take at this point. It won’t help you predict anything that’s coming.

Since we disagree on the present let’s informally do a “remind me 2 years” to this discussion and see what’s happened then.

Re: SQLite Critical CVEs or LLM Slop?

#277

Post by inigyou »

Yajirobe wrote: post #237
unknownfuture wrote: post #224

Earlier quoted context omitted.

Not sure what your point is? Those things can both be true. Or should the discourse in a diverse community like HN only reflect the positions you personally hold?

Tell me how a 'nExT toKeN prEdIcTor' can make breakthroughs in math or play a game of chess. These activities aren't pure symbol manipulation, they require actual understanding at some level.

By predicting next tokens

Re: SQLite Critical CVEs or LLM Slop?

#278

Post by inigyou »

gbnwl wrote: post #221
Jensson wrote: post #215

Earlier quoted context omitted.

The rate of progress can be high and they can also be dumb next token predictors. Not sure why that is hard to understand. These models can do a lot of things but they also can't do a lot of things. In order to use these models effectively you have to understand that they are next token predictors and how that allows it to do what they do.

Are they useful or not? Will they continue changing the world or not? People who choose one way or the other for describing them typically fall on one side or the other in these questions imo. What do you think? Will these next token predictors change the world or not?

Somewhat.

Yes, but not because they are useful.

Re: SQLite Critical CVEs or LLM Slop?

#279

Post by chrisjj »

gbnwl wrote: post #198
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Every day I wake up and open HN. “LLM has made legitimate mathematical discoveries” —> Wow the rate of progress is amazing. Highly upvoted. “LLM does something not good” -> Does everyone else not realize LLMs are just dumb next token predictors? Highly upvoted. So tired of this discourse and this site.

Opinions differ. This is not news.

Re: SQLite Critical CVEs or LLM Slop?

#280

Post by estearum »

polotics wrote: post #269
estearum wrote: post #162

Earlier quoted context omitted.

> but being able to generate probabilistic outputs based on inputs is not what makes us intelligent. ??? Of course it is. The brain is mechanically not capable of doing anything other than that. Do you believe the brain is something other than a bundle of probabilistic physical interactions? Or are brains not the source of what we call intelligence?

May I suggest the excellent & recent book: "The brain. In theory" by Romain Brette? This is going to elevate your thinking on this no end, if you're interested.

Looks interesting (added to my list, thank you for the recommendation), but not actually relevant to the topic at hand.

We know that the brain is a probabilistic input → output machine because the universe is a probabilistic input → output machine. The brain is made of universe. There are deterministic relationships (which at high sensitivity or complexity become easier to describe as probabilistic), and quantum relationships. That's it. The brain, like every other thing comprised of "universe" is comprised of those two types of relationships.

If Romain's book provides evidence of relationships in the brain that are neither quantum (therefore random) NOR classical (therefore deterministic), then 1) he would have already won at least one Nobel prize, and 2) anyone in this thread would be able to at least gesture toward what relationship that is.

Post reply on HN