Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

391–400 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#391

Post by gortok »

Jensson wrote: post #287
treszkai wrote: post #286

Earlier quoted context omitted.

> LLMs are text-prediction engines. They are not Artificial Intelligence, and shouldn’t not be treated in any form or fashion as if they possess intelligence. I agree that humans must verify LLM-produced facts, but strongly disagree with these kinds of "stochastic parrot therefore dumb" arguments. Yes, an LLM is a "stochastic parrot". No, that doesn't imply that it is dumb. Enough to look at how Terence Tao asks Chat…

The best way to describe the LLM intelligence is "an expert system that works the way people thought expert systems would work". You can encode a massive amount of skills into an LLM, and then the LLM uses those to navigate problems. But the LLM is still dumb where those skills doesn't have good coverage, since unlike the expert systems it maps fuzzily to its skills, and they are tuned to produce results over rejecti…

I’m not looking to argue about your position, but I think the inciting incident in the OP shows evidence that LLMs are “dumb” even when they have the sum total of written language and code at their training disposal. In this case I would expect, for your supposition to be true, that an LLM would not mistake a code comment for an actual vulnerability.

Re: SQLite Critical CVEs or LLM Slop?

#392

Post by [deleted] »

ryan_n wrote: post #264
Blackthorn wrote: post #259

Earlier quoted context omitted.

It is pretty funny to see the shoe on the other foot, since it's usually software engineers with unearned arrogance about other fields.

What fields do you see devs think they know about? I’ve never personally seen this with other devs I work with but obviously small sample size…

[deleted]

Re: SQLite Critical CVEs or LLM Slop?

#393

Post by emp17344 »

ryan_n wrote: post #264
Blackthorn wrote: post #259

Earlier quoted context omitted.

It is pretty funny to see the shoe on the other foot, since it's usually software engineers with unearned arrogance about other fields.

What fields do you see devs think they know about? I’ve never personally seen this with other devs I work with but obviously small sample size…

If HN is any indication, many devs have hilariously naive beliefs regarding economics and philosophy and will absolutely refuse to engage with actual arguments or experts within those fields.

Re: SQLite Critical CVEs or LLM Slop?

#394

Post by emp17344 »

red75prime wrote: post #241
buttercraft wrote: post #227

Earlier quoted context omitted.

The point is that you're asking someone to prove a negative.

It's a negative in your worldview where "intelligence can't be a computation" is an axiom or something close. To me this worldview is sloppy thinking.

We have reason to believe consciousness cannot be replicated via computation, and reason to suspect consciousness is an important aspect of cognition. Now, what reason do you have for believing intelligence could be replicated computationally?

Re: SQLite Critical CVEs or LLM Slop?

#395

Post by rolymath »

ryan_n wrote: post #264
Blackthorn wrote: post #259

Earlier quoted context omitted.

It is pretty funny to see the shoe on the other foot, since it's usually software engineers with unearned arrogance about other fields.

What fields do you see devs think they know about? I’ve never personally seen this with other devs I work with but obviously small sample size…

Devs like to pretend they know everything about everything.

Start a discussion about any topic, look at how many nerds will tell you that you are wrong.

Re: SQLite Critical CVEs or LLM Slop?

#397

Post by vouwfietsman »

Jensson wrote: post #339
vouwfietsman wrote: post #331

Earlier quoted context omitted.

> they will keep making strong statements about things they don't understand until they are fundamentally changed If ever there was a human quality. Also, your explanation of "dumb" is really favoring the anti-llm side, and its a very generous interpretation. I suspect what is much more likely meant, is that token predictors cannot be smart, not now nor in the future after improvements, because they are token predict…

> I suspect what is much more likely meant, is that token predictors cannot be smart, not now nor in the future after improvements, because they are token predictors and predicting tokens is not how intelligence works. Why do you think that? LLM are used as expert systems today, in order to quickly navigate problems by breaking them down and iterating between different well known possible solutions and paths to check…

> Why do you think that?

From (highly upvoted) parent:

> LLMs are text-prediction engines. They are not Artificial Intelligence, and shouldn’t not be treated in any form or fashion as if they possess intelligence > LLM-based “AI” ....

Hardly neutral statements, hardly accurate statements, yet highly upvoted. LLMs are AI, there is nothing to gain by pretending it isn't because of some secondary motive or opinion someone has.

> LLM has solved enough problems that almost nobody has the view you ridicule here

I think I have just shown you the parent literally claims LLMs are not AI, and do not possess intelligence.

> but there are still many who think LLM are thinking just like humans and that you can trust them just like humans.

I don't think many people think that, especially on HN, for two reasons:

1. Most high profile AI tools have immediately visible disclaimers saying, more or less, "AI makes mistakes". 2. People don't trust humans either, if anything people trust the AI more because it is not human.

People may be incorrectly worshipping LLMs, but they are doing it precisely because it is not human. If you put chatgpt behind a believable chatbox so people would actually think it was human, they would be much more skeptical.

Re: SQLite Critical CVEs or LLM Slop?

#398

Post by vouwfietsman »

Jensson wrote: post #335
vouwfietsman wrote: post #328

Earlier quoted context omitted.

> But if that actually solves it then Solves what? Chess? No thats not AI, its just a chess bot. Turing test? No, thats not AI, its just a dumb token predictor. Why would the goalposts stop moving at AGI? I am 100% convinced it will somehow still lack the "gusto" or the "taste" or the "timbre" of real intelligence. You can see it in coding right now, AI has bad "taste" in coding, because really, we can't do a better…

> Solves what? Chess? No thats not AI, its just a chess bot. Turing test? No, thats not AI, its just a dumb token predictor. You are moving the goalpost here if you think the chess AI was AGI. All those problems were evidence AI wasn't as smart as humans, not goalposts determining when AI is as smart as humans. The first turing test winner wasn't even an LLM, it was an expert system, so we already knew that test wasn…

> You are moving the goalpost here if you think the chess AI was AGI

Indeed I would be, but I don't, so I'm not.

What I'm saying is that nobody understands either creativity or intelligence, and people keep listing behavior that they think can only be displayed by something that has those qualities. Yet, whenever we build a system that displays this behavior, all of a sudden, those qualities are NOT attributed because we now understand how to achieve this, the curtain has fallen, the magic is gone, and we do not assign the qualities we believe are human to something that is not magic, lest we reduce ourselves to the machines in front of us, losing the things that make us unique.

That is the shifting of goalposts, its not me who's doing it. There's a wiki page for it, its called "the AI effect": https://en.wikipedia.org/wiki/AI_effect

Re: SQLite Critical CVEs or LLM Slop?

#399

Post by [deleted] »

Sammi wrote: post #388
ekholm_e wrote: post #326

Earlier quoted context omitted.

This. I'm a former teacher. I left to get a PhD in educational psychology, and now I'm a data scientist for a large school division. The number of ed tech companies I interact with who clearly have little grasp of research-based best practices in education is staggering. But they can all build slick-looking websites and dashboards.

The software is not the product. You can't get a software person to understand this unless they learn the domain they are working in properly.

[deleted]

Re: SQLite Critical CVEs or LLM Slop?

#400

Post by dns_snek »

traceroute66 wrote: post #386
dns_snek wrote: post #383

Earlier quoted context omitted.

> Over the last few years insurers have aggressively been adding "no vulnerability patch, no claim" exclusion clauses. If you take an even closer look- what exactly does that mean? It should stipulate patching real vulnerabilities in your system and not "all CVEs in all dependencies irrespective of their usage or applicability to your system", that's madness. It's like voiding your health insurance policy because a s…

> If you take an even closer look- what exactly does that mean? Here is one example: "Critical Vulnerability Exclusion We will not pay you under the cyber and data risks section of cover where your legal liability or any loss that you suffer arises from a cyber attack that exploits a critical vulnerability within your computer equipment. However this exclusion will only apply where a patch or fix for any critical vul…

Thanks, this seems to confirm my train of thought. To lose coverage the attack had to exploit that particular vulnerability and a patch had to have been available for over 3 weeks. A false CVE can't be exploited so you can't lose coverage by not patching it (even if a purported patch existed).
Post reply on HN