Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

311–320 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#311

Post by Jensson »

estearum wrote: post #283
beepbooptheory wrote: post #268

Earlier quoted context omitted.

Ok right, but then I don't see the original point. Or like, you don't seem to be appealing to a hard science/soft science distinction here if the question revolves around what conceptual model is most "useful" to us. Useful for what? For predicting future actions? That makes sense, but does actually that speak to op's (possibly still fraught) metaphysical/semantic point here? It just feels like you are speaking over…

I am saying that there is no categorical distinction to be made on functional grounds between artificial intelligence and brains. You cannot say an AI model cannot be intelligent because it's a probability machine, when all available evidence points toward natural intelligence also being generated by probability machines (much more complex ones, called brains).

But brains do much more than just predict tokens based on previously seen tokens. I think all the other things brains do are probably important for our intelligence.

So, LLM are just next token predictors, brains are next token predictors + many other things in addition, and that makes people still feel LLM are dumb even when they solve a lot of problems using tokens.

Re: SQLite Critical CVEs or LLM Slop?

#312

Post by inigyou »

clbrmbr wrote: post #154
dwedge wrote: post #47

Earlier quoted context omitted.

> Tools like trivvy make it possible to do the scans... Only if you didn't rip trivvy out of your organisation when it had two supply chain compromises within a month of each other earlier this year

Yikes. Man, there’s a market opening for someone to redistribute open source projects with supply chain assurances!

There's been a market for a while. In thinking of Azul Java, which is just OpenJDK but with someone to point the finger at, and costs money.

Re: SQLite Critical CVEs or LLM Slop?

#313

Post by Jensson »

jimmaswell wrote: post #310
Jensson wrote: post #301

Earlier quoted context omitted.

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it. Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective l…

I strongly suspect that if you dug deeply enough into how a human brain does reasoning, the answer would boil down to "it tries a bunch of pathways and picks the one that seems right". Pathways weighted by experience, but ultimately still guess-and-check. What would the alternative be? Maybe our ability to reason is not as mystical and special as our ego might hope it to be, and discomfort over LLM's bringing that to…

You have introspection, you can see a part of your thoughts, you know how that introspective part works since its what we call consciousness, you are conscious about it.

That introspection isn't an illusion, what your consciousness see of your thoughts obviously are tings that has been calculated and aggregated by the brain, so we know the brain calculates and aggregates those thoughts and feelings to produce its results. And we know LLM doesn't do that, it doesn't have a side system that does that sort of introspection.

Re: SQLite Critical CVEs or LLM Slop?

#314

Post by inigyou »

oxydite wrote: post #135

Damn, dunno why I've always assumed that if something gets a CVE, then some authority has reproduced and verified it. Shouldn't it be CNA's job? Why would anything get a number if it hadn't been verified?

It's really just a number allocation and that's all. You can get a number. That doesn't mean the number refers to something useful.

Re: SQLite Critical CVEs or LLM Slop?

#315

Post by inigyou »

insanitybit wrote: post #282
inigyou wrote: post #275

Earlier quoted context omitted.

Malicious compliance is compliance.

I don't know what you're trying to say at all. But no, the kernel is objectively not compliant because they label fixes with CVEs and not vulnerabilities. But even if they were compliant... what would that have to do with anything? Feels like a non sequitur. It's very hard to figure out what point you're trying to make.

Every bugfix implies a bug.

Re: SQLite Critical CVEs or LLM Slop?

#316

Post by jimmaswell »

Jensson wrote: post #313
jimmaswell wrote: post #310

Earlier quoted context omitted.

I strongly suspect that if you dug deeply enough into how a human brain does reasoning, the answer would boil down to "it tries a bunch of pathways and picks the one that seems right". Pathways weighted by experience, but ultimately still guess-and-check. What would the alternative be? Maybe our ability to reason is not as mystical and special as our ego might hope it to be, and discomfort over LLM's bringing that to…

You have introspection, you can see a part of your thoughts, you know how that introspective part works since its what we call consciousness, you are conscious about it. That introspection isn't an illusion, what your consciousness see of your thoughts obviously are tings that has been calculated and aggregated by the brain, so we know the brain calculates and aggregates those thoughts and feelings to produce its res…

It actually is an illusion though. The narrative the brain makes up after the fact for why we did something is not perfectly correlated with the actual reason. It's just very good at its job of spinning a convincing narrative, even to itself. This is demonstrated in split brain patients: they're given instructions to do something that are shown only to one eye, the one connected to the non-speaking hemisphere, and when asked why they did it, the speaking hemisphere confidently hallucinates a wrong answer that the person seems to fully believe.

Re: SQLite Critical CVEs or LLM Slop?

#317

Post by batshit_beaver »

adjfasn47573 wrote: post #243
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

> A human must verify everything an LLM presents as fact. Everything. I've thought about this for quite some time now. No. A human doesn't need to verify everything. And the argument is really simple: stochastic. Think of self-driving cars: We can show today - based on evidence and real data - that self-driving cars are safer than human drivers. That's a fact and the consequences are clear, more self-driving cars, le…

> Now back to AI for software creation. "Review is the bottleneck because EVERYTHING must be judged by a human." No. It doesn't. We just need to build AI review systems, that will do reviews better than (or at least as good as) humans. The human review quality bar is far below 100%. Far far far. If we can show (likely in the next 12-24 months I think) that AI review quality is consistently above the human review quality - again, based on evidence, based on real data - then that's it, then there's no good reason to have humans review the code.

Let me know once the majority of software engineering organizations start only checking in markdown files and let code be generated non-deterministically from these specs in CI. If this is not happening now, there’s clearly a sufficiently high level of distrust in blind LLM output (both code itself and reviews).

And even then, are you suggesting humans shouldn’t at least review the markdown specs? Why not have LLMs review the specs then? Is there, perhaps, some fundamental quality to human review process that is desirable here?

Re: SQLite Critical CVEs or LLM Slop?

#318

Post by bigbuppo »

gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

> A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can.

The sort of person that's going to offload their thinking to AI is the exact sort of person that is not going to verify anything because they've already offloaded their thinking to AI.

Re: SQLite Critical CVEs or LLM Slop?

#319

Post by Jensson »

jimmaswell wrote: post #316
Jensson wrote: post #313

Earlier quoted context omitted.

You have introspection, you can see a part of your thoughts, you know how that introspective part works since its what we call consciousness, you are conscious about it. That introspection isn't an illusion, what your consciousness see of your thoughts obviously are tings that has been calculated and aggregated by the brain, so we know the brain calculates and aggregates those thoughts and feelings to produce its res…

It actually is an illusion though. The narrative the brain makes up after the fact for why we did something is not perfectly correlated with the actual reason. It's just very good at its job of spinning a convincing narrative, even to itself. This is demonstrated in split brain patients: they're given instructions to do something that are shown only to one eye, the one connected to the non-speaking hemisphere, and wh…

> The narrative the brain makes up after the fact for why we did something is not perfectly correlated with the actual reason

But it does that introspection, we evolved to make it. If its not useful for anything we wouldn't have evolved it, it can't have been easy to evolve a consciousness so there has to be purpose for it.

Or do you think our consciousness is a magical ghost thing unrelated to the brains workings? I think its pretty obvious that our smarts in part depend on the computations that results in our conscious experience, you need a very strong argument as to why that wouldn't be the case. And broken brains being broken doesn't answer this.

Re: SQLite Critical CVEs or LLM Slop?

#320

Post by afavour »

ryan_n wrote: post #264
Blackthorn wrote: post #259

Earlier quoted context omitted.

It is pretty funny to see the shoe on the other foot, since it's usually software engineers with unearned arrogance about other fields.

What fields do you see devs think they know about? I’ve never personally seen this with other devs I work with but obviously small sample size…

In my experience, product. There’s a pervasive view that product is easy, that anyone working in it is a dunderhead who gets in the way of engineers doing the really worthwhile work. In reality, it would just mean they rewrite the software five times and have zero customers.

Oh, design too. I’ve met many engineers who simply don’t see any value in design.

Post reply on HN