Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

361–370 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#361

Post by Jensson »

estearum wrote: post #353
Jensson wrote: post #345

Earlier quoted context omitted.

> transforming an input into an output based on what inputs that part of the brain has previously been exposed to You changed the definition there, for it to be like an LLM it should be: > transforming an input into an output trying to mimic inputs that part of the brain has previously been exposed to Anyone can see how that limits you a lot, and why that makes it so much harder for LLM to learn things properly than…

Well no, you changed the definition. LLMs don't just mimic what they've been trained on. They mimic what they're rewarded for. Like a brain! Pre-training is just direct mimicry. A pre-trained LLM is very stupid and mostly useless. To become useful they are post-trained with a reward function.

Right, the CoT training step does work a bit more like normal training. But those do produce some very weird results, if you look at the "thoughts" CoT training leads to, I wouldn't say that CoT adds general intelligence, it teaches it to solve specific problems just like how any regular neural net training works.

The only general part of this training is the text prediction. All the other steps add specific skills to it and try to not hurt the generality as they do so, but we have no way to add general skill in the second step. So the general part is still the text predictor part. We know that training a neural net on specific tasks doesn't produce generality, we have tried that before, the text predictor is what let us get a glimpse of generality.

So then the text predictor try to solve problems by moving into the different specific skills there.

But yeah, you are right, without the CoT the text predictor wouldn't have any reliable skills to map to, so its still an important part of it. But the CoT doesn't add to the generality of the model, it just adds special skills it can utilize.

But, I agree what I am arguing now is a weaker statement. But I still believe its true, but I probably wont convince you of it.

Re: SQLite Critical CVEs or LLM Slop?

#362

Post by insanitybit »

inigyou wrote: post #315
insanitybit wrote: post #282

Earlier quoted context omitted.

I don't know what you're trying to say at all. But no, the kernel is objectively not compliant because they label fixes with CVEs and not vulnerabilities. But even if they were compliant... what would that have to do with anything? Feels like a non sequitur. It's very hard to figure out what point you're trying to make.

Every bugfix implies a bug.

I'm going to graciously give you one last chance to actually say something of substance before I stop responding entirely.

Re: SQLite Critical CVEs or LLM Slop?

#363

Post by fsflover »

Jensson wrote: post #347
p2detar wrote: post #343

Earlier quoted context omitted.

In all fairness humans can also be considered next token predictors. It could be said that’s how we communicate with one another today. Presently LLMs lack other things, like physical presence in the world and continuity of input sensory data.

Humans learn to be a next token predictor as a kid when they learn to speak, an LLM cannot learn to be a next token predictor or anything of the sort, we have no clue how you could have an LLM learn human language just based on a thousands conversations with a human. You don't see how that is very different? For an LLM to be as smart as a human it has to be able to learn like a human. Like you don't evaluate how smar…

> For an LLM to be as smart as a human it has to be able to learn like a human.

"For a plane to fly as well as a bird it has to be able to flap its wings".

"For a submarine to swim as well as a fish it has to be as light as fish".

Re: SQLite Critical CVEs or LLM Slop?

#364

Post by 27183 »

gortok wrote: post #321
bigbuppo wrote: post #318

Earlier quoted context omitted.

> A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can. The sort of person that's going to offload their thinking to AI is the exact sort of person that is not going to verify a…

For a long time I was anti-licensure in tech; now with the bar being lowered to next to nothing, it seems as if licensure is more important than ever — not to protect this trade (though it will do that, and that is a benefit), but because the sheer amount of irresponsibility in the usage of LLMs and “AI” in general begs for licensure and adoption of a regulatory body for software in general.

100%. I don't think "AI" has changed anything wrt to the responsibility of the tech industry in general. Tech has always been pretty much devoid of ethics or a sense of responsibility at the executive level (and therefore "management" more broadly). But before LLMs it was easier for ethical engineers to surreptitiously steer things towards responsible implementations. I suspect also competence correlates pretty strongly with intellectual honesty--narcissists and other personality defectives generally aren't terribly capable of the kind of introspection necessary to learn deeply. With the deluge of LLM slop, it's much harder for even a responsible manager to differentiate the competent from the incompetent, and now the dishonest, unethical folks have more leverage.

So now the problem is highly visible. There's only one solution I can think of, and that's to hold individuals (not corporations) responsible for professional malfeasance.

Re: SQLite Critical CVEs or LLM Slop?

#365

Post by icedchai »

pixl97 wrote: post #274
icedchai wrote: post #242

Earlier quoted context omitted.

This might cause other problems, problems of the "if it's not broken, don't fix it" variety. Upgrading everything only to break something else, in a previously stable configuration, isn't worth it.

Really the days of "Lets run this stable configuration forever" are gone. Getting rid of as much stuff in your OS and software stack as possible should be the security teams ultimate goal, so you have less to upgrade in the end. But actual security updates just come out at a tremendous rate, and you need a QA system that checks as much as it can before prod is upgraded.

I do agree with more frequent upgrades, but it has to be part of the culture. The longer you wait, the harder it gets. Unfortunately, I have worked in some heavily tech-debt-laden environments where upgrading anything required an act of god. I've logged on to production servers with 1500+ days of uptime at multiple companies. Nothing had been updated since well before that time. At one place, I recall encountering a 10+ year old dependency. On top of that, they were still using Python 2.7. This wasn't that long ago.

Re: SQLite Critical CVEs or LLM Slop?

#366

Post by 27183 »

fsflover wrote: post #363
Jensson wrote: post #347

Earlier quoted context omitted.

Humans learn to be a next token predictor as a kid when they learn to speak, an LLM cannot learn to be a next token predictor or anything of the sort, we have no clue how you could have an LLM learn human language just based on a thousands conversations with a human. You don't see how that is very different? For an LLM to be as smart as a human it has to be able to learn like a human. Like you don't evaluate how smar…

> For an LLM to be as smart as a human it has to be able to learn like a human. "For a plane to fly as well as a bird it has to be able to flap its wings". "For a submarine to swim as well as a fish it has to be as light as fish".

> "For a plane to fly as well as a bird it has to be able to flap its wings".

> "For a submarine to swim as well as a fish it has to be as light as fish".

These are false equivalences. The post you're responding to defined intelligence as learning rate. LLMs unequivocally do not learn. You can disagree with OP or agree, but what you have done is out of bounds. You're implicitly claiming that LLMs learn, albeit differently from humans. This is categorically false, unless you count training as some kind of "learning".

That's of course absurd, almost no user of an LLM also trains it. Instead they rely on queries submitted to pre-trained LLMs ("inference"). And don't bother yapping about context windows, it's just not anything like learning.

Re: SQLite Critical CVEs or LLM Slop?

#367

Post by blargey »

unknownfuture wrote: post #224
gbnwl wrote: post #198

Earlier quoted context omitted.

Every day I wake up and open HN. “LLM has made legitimate mathematical discoveries” —> Wow the rate of progress is amazing. Highly upvoted. “LLM does something not good” -> Does everyone else not realize LLMs are just dumb next token predictors? Highly upvoted. So tired of this discourse and this site.

Not sure what your point is? Those things can both be true. Or should the discourse in a diverse community like HN only reflect the positions you personally hold?

More that the “discourse” around anything “AI” is just another culture war at this point - dominated by facile taking points that are shallow, ill-considered, uninformed-and-uninformative, and re-re-recycled into worthless pulp by their respective tribal bubble.

And if someone’s first response to this comment is to try to peg which “side” I’m calling an idiot as if that’s the most important bit of info to dictate their response - that’s exactly what I’m talking about.

Re: SQLite Critical CVEs or LLM Slop?

#368

Post by red75prime »

buttercraft wrote: post #354
red75prime wrote: post #341

Earlier quoted context omitted.

Why do you think that intelligence of a Turing-complete system trained on human data can be likened to a Russell's teapot? At least, don't you think that the recent mathematical results of LLMs are a bit like a glimpse of something teapot-shaped in the orbit? (which makes it not a Russell's teapot, which. by definition, can't be observed). To me, it's an expected progression of ANNs' approximation of human cognitive…

> Why do you think that intelligence of a Turing-complete system trained on human data can be likened to a Russell's teapot? I don't? You are presenting opinions as if they are mine, but they are not.

I'm trying to understand what's going on in the head of people who are certain that LLMs can't be intelligent. And it proves to be a daunting task. Have you tried to reflect on your certainty? Where it comes from?

Re: SQLite Critical CVEs or LLM Slop?

#369

Post by thesmtsolver2 »

estearum wrote: post #280
polotics wrote: post #269

Earlier quoted context omitted.

May I suggest the excellent & recent book: "The brain. In theory" by Romain Brette? This is going to elevate your thinking on this no end, if you're interested.

Looks interesting (added to my list, thank you for the recommendation), but not actually relevant to the topic at hand. We know that the brain is a probabilistic input → output machine because the universe is a probabilistic input → output machine. The brain is made of universe. There are deterministic relationships (which at high sensitivity or complexity become easier to describe as probabilistic), and quantum rela…

This is a vacuous argument. If everything is a “probabilistic input output machine” then what exactly are you claiming?

You should look into mathematical definitions of probability. You will be surprised to find not all formal mathematical systems can fit inside probability spaces.

Re: SQLite Critical CVEs or LLM Slop?

#370

Post by estearum »

thesmtsolver2 wrote: post #369
estearum wrote: post #280

Earlier quoted context omitted.

Looks interesting (added to my list, thank you for the recommendation), but not actually relevant to the topic at hand. We know that the brain is a probabilistic input → output machine because the universe is a probabilistic input → output machine. The brain is made of universe. There are deterministic relationships (which at high sensitivity or complexity become easier to describe as probabilistic), and quantum rela…

This is a vacuous argument. If everything is a “probabilistic input output machine” then what exactly are you claiming? You should look into mathematical definitions of probability. You will be surprised to find not all formal mathematical systems can fit inside probability spaces.

Formal mathematical systems are not physical objects. The brain – which is presumably the source of natural intelligence – is.

By "everything" I was referring (obviously) to things in the universe. Mathematical systems are not things in the universe. They are not made of universe the way every object, including a brain, is.

And no, it's not vacuous at all. GP is trying to argue that a probabilistic input → output machine doesn't produce intelligence. But we have an existence proof of exactly that!

Post reply on HN