Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

321–330 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#321

Post by gortok »

bigbuppo wrote: post #318
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

> A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can. The sort of person that's going to offload their thinking to AI is the exact sort of person that is not going to verify a…

For a long time I was anti-licensure in tech; now with the bar being lowered to next to nothing, it seems as if licensure is more important than ever — not to protect this trade (though it will do that, and that is a benefit), but because the sheer amount of irresponsibility in the usage of LLMs and “AI” in general begs for licensure and adoption of a regulatory body for software in general.

Re: SQLite Critical CVEs or LLM Slop?

#322

Post by lucideer »

mr_mitm wrote: post #42
lucideer wrote: post #28

Earlier quoted context omitted.

I've been in such an org, & I've led initiatives to set up automated detection at very large scale. We started by issuing tickets to teams to resolve CVEs within varying timelines - ranging from a 24hr fix to 6 months - connected to the CVSS score. It wasn't viable. - Firstly, you quickly realise how irrelevant CVSS scores are - initiatives like First's EPSS are designed to fix this but they aren't there yet - Second…

> - Firstly, you quickly realise how irrelevant CVSS scores are Even if you factor in the environmental score? I realize it's a lot more work, but it basically allows you to tune the score to get any value you want.

I could reword that CVSS is useless on its own - EPSS takes CVSS as an input afterall. The point here is that applying environmental scoring is the hard part. That's the algorithm that's containing the complexity.

Re: SQLite Critical CVEs or LLM Slop?

#324

Post by buttercraft »

red75prime wrote: post #241
buttercraft wrote: post #227

Earlier quoted context omitted.

The point is that you're asking someone to prove a negative.

It's a negative in your worldview where "intelligence can't be a computation" is an axiom or something close. To me this worldview is sloppy thinking.

I did not express my opinion or "worldview." I was simply responding to this:

> exclude LLMs with CoT from the category of intelligent systems with certainty

Re: SQLite Critical CVEs or LLM Slop?

#325

Post by unknownfuture »

gbnwl wrote: post #299
unknownfuture wrote: post #285

Earlier quoted context omitted.

> I just prefer HN comments to be better reflections of reality. You mean your particular version of it. It's interesting to see you consistently missing this point. You've decided LLMs are clearly more than just complex but mindless statistical models. You've decided that based on, it seems, the very impressive things these tools are capable of. Therefore if anyone claims they're just mindless stastical models--with…

It’s just a boring and unhelpful complaint that afaict largely serves to soothe the commenters ego rather than point at anything insightful that’s useful or predictive. Point me to your favorite “next token predictor” comment that was actually insightful or predictive. You have years of material to draw from.

https://news.ycombinator.com/item?id=49155075

"These models are probabilistic, you shouldn't blindly trust them in spaces where accuracy is really important" seems like pretty sound advice to me.

Re: SQLite Critical CVEs or LLM Slop?

#326

Post by ekholm_e »

afavour wrote: post #320
ryan_n wrote: post #264

Earlier quoted context omitted.

What fields do you see devs think they know about? I’ve never personally seen this with other devs I work with but obviously small sample size…

In my experience, product. There’s a pervasive view that product is easy, that anyone working in it is a dunderhead who gets in the way of engineers doing the really worthwhile work. In reality, it would just mean they rewrite the software five times and have zero customers. Oh, design too. I’ve met many engineers who simply don’t see any value in design.

This. I'm a former teacher. I left to get a PhD in educational psychology, and now I'm a data scientist for a large school division. The number of ed tech companies I interact with who clearly have little grasp of research-based best practices in education is staggering. But they can all build slick-looking websites and dashboards.

Re: SQLite Critical CVEs or LLM Slop?

#327

Post by estearum »

polotics wrote: post #293
estearum wrote: post #280

Earlier quoted context omitted.

Looks interesting (added to my list, thank you for the recommendation), but not actually relevant to the topic at hand. We know that the brain is a probabilistic input → output machine because the universe is a probabilistic input → output machine. The brain is made of universe. There are deterministic relationships (which at high sensitivity or complexity become easier to describe as probabilistic), and quantum rela…

You're a fast reader :-D No quantum bullshittery in there I promise. Equating "classical" with "deterministic" is however pushing it a bit too far, when no one and nothing can ever do even a very small fraction of said determination...

Quantum is not bullshit and yes, classical is deterministic.

That's why the brain cannot possibly be anything other than an input → output machine, which is functionally deterministic (with maybe some fully random components), but is easiest to describe as probabilistic.

In the same way that LLMs are functionally deterministic, but easiest to describe as probabilistic.

Re: SQLite Critical CVEs or LLM Slop?

#328

Post by vouwfietsman »

Jensson wrote: post #309
vouwfietsman wrote: post #303

Earlier quoted context omitted.

> I don't think LLM will ever stop being dumb without such an introspective part to them. The very second they add this part, it will "just be a dumb token predictor with introspection", mark my words.

If it keeps doing dumb things, yeah. But if that actually solves it then those opinions will quickly disappear when it replaces all human white collar work since it does it cheaper and better and faster. AGI is fairly easy to detect for this reason. Does this system make a majority feel you don't have to hire people anymore? If not its not AGI.

> But if that actually solves it then

Solves what? Chess? No thats not AI, its just a chess bot. Turing test? No, thats not AI, its just a dumb token predictor.

Why would the goalposts stop moving at AGI? I am 100% convinced it will somehow still lack the "gusto" or the "taste" or the "timbre" of real intelligence. You can see it in coding right now, AI has bad "taste" in coding, because really, we can't do a better job critiquing that which is obviously (on occasion) just plain good.

Re: SQLite Critical CVEs or LLM Slop?

#329

Post by estearum »

Jensson wrote: post #311
estearum wrote: post #283

Earlier quoted context omitted.

I am saying that there is no categorical distinction to be made on functional grounds between artificial intelligence and brains. You cannot say an AI model cannot be intelligent because it's a probability machine, when all available evidence points toward natural intelligence also being generated by probability machines (much more complex ones, called brains).

But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence. So, LLM are just next token predictors, brains are next token predictors + many other things in addition, and that makes people still feel LLM are dumb even when they solve a lot of problems using tokens.

> But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.

Like what?

What specific biological structure in the brain could be doing anything other than producing output as a function of 1) current electrical/chemical/thermal inputs and 2) previous electrical/chemical/thermal inputs?

Re: SQLite Critical CVEs or LLM Slop?

#330

Post by kentm »

ivan_gammel wrote: post #272
kentm wrote: post #200

Earlier quoted context omitted.

Yes but the key here is doing proper risk assessment. "What is the consequence if the LLM gets this wrong?" "How do we verify the output?" "What are the legal ramifications for using the LLM in this way?" "Who is responsible when the LLM fails?" "Whats the expected accuracy here?" etc. In the current AI mania, there's a lot of due diligence simply being ignored. Plenty of "Well humans make mistakes too!" going on her…

The due diligence not being done is people putting cats in microwaves. It‘s not the dangerous part. The real danger is risk assessments coming to wrong conclusions, because it is still terra incognita. Talented engineers were in this situation before, doing mistakes with cars, airplanes, buildings etc.

No, I'm sorry but I think thats a cop out. The fact that LLM are stochastic and can give incorrect answers is not particularly difficult to comprehend, and the risks that fall out of that are reasonably understandable. The issue is entirely down to bad choices by the people driving LLMs, because they are engaging with what they wish LLMs do instead of what they actually do.
Post reply on HN