Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

301–310 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#301

Post by Jensson »

vouwfietsman wrote: post #292
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Does a dog possess intelligence? Does a bird? Does a cricket? An amoeba? I hate AI slop as much as the next guy but the amount of tribalism over AI is taking near-religious forms. Nobody knows what intelligence is, therefore we don't know what does or does not possess it, therefore we don't know whether LLMs currently, or in the future, possess it. Yes, LLMs can be stupid, guess what: so can I. That doesn't really ch…

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it.

Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective logic to produce that "I don't know", and its possible it actually does know if it followed another branch there so "I don't know" is often not even true.

If they had an introspective part that stops the prediction when its too unreliable it would no longer just be token prediction engine, and I believe we need such a part for them to become what I call smart. I don't think LLM will ever stop being dumb without such an introspective part to them.

And no, that introspective part is not a part of the token predictor. At least not in us humans, the feeling of certainty we have is not a prediction, it is bundled with our thoughts, so we get both "answer is a bear" and "certainty is low", we don't get just one of those as a "prediction".

Will LLM become smart as humans with such an introspective part? I don't know, but I think they will never become as smart as humans without one.

Note: The certainty score has to be per conclusion or response, not per token. You can't evaluate a responses validity by aggregating the weight of each token. Meaning its a logic engine, not token engine, that evaluates the certainty of a statement being correct or not instead of a token being correct or not. That is the level human thinking works at and seems to be dramatically more efficient.

Re: SQLite Critical CVEs or LLM Slop?

#302

Post by vouwfietsman »

Jensson wrote: post #215
gbnwl wrote: post #198

Earlier quoted context omitted.

Every day I wake up and open HN. “LLM has made legitimate mathematical discoveries” —> Wow the rate of progress is amazing. Highly upvoted. “LLM does something not good” -> Does everyone else not realize LLMs are just dumb next token predictors? Highly upvoted. So tired of this discourse and this site.

The rate of progress can be high and they can also be dumb next token predictors. Not sure why that is hard to understand. These models can do a lot of things but they also can't do a lot of things. In order to use these models effectively you have to understand that they are next token predictors and how that allows it to do what they do.

It can be a token predictor and still tell me exactly how my life will proceed from now until the indefinite future, or be the most intelligent conversational entity you have ever witnessed.

The issue is of course with using the word "dumb": they are next token predictors, no doubt about it, but whether LLms as a class of system are smart or dumb is entirely unknown and entirely variable in time.

To interact with them effectively you must know how they behave, just like you have to know how humans behave to interact with them effectively. If you disagree, find someone with autism and have a conversation with them.

Re: SQLite Critical CVEs or LLM Slop?

#303

Post by vouwfietsman »

Jensson wrote: post #301
vouwfietsman wrote: post #292

Earlier quoted context omitted.

Does a dog possess intelligence? Does a bird? Does a cricket? An amoeba? I hate AI slop as much as the next guy but the amount of tribalism over AI is taking near-religious forms. Nobody knows what intelligence is, therefore we don't know what does or does not possess it, therefore we don't know whether LLMs currently, or in the future, possess it. Yes, LLMs can be stupid, guess what: so can I. That doesn't really ch…

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it. Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective l…

> I don't think LLM will ever stop being dumb without such an introspective part to them.

The very second they add this part, it will "just be a dumb token predictor with introspection", mark my words.

Re: SQLite Critical CVEs or LLM Slop?

#304

Post by GeorgeWBasic »

dpacmittal wrote: post #214

The duality of LLMs is real. On one hand, they are solving unsolved math problems, on the other hand they get fooled by the strawberry test or the car wash test. We see this all the time on reddit and HN, where some users are building crazy things with LLM and others are swearing by how useless LLMs are. Some people claim SWE will be dead in a few years, and others are claiming the AI bubble is going to burst anytime…

The "AI bubble" refers to the massive overinvestment in AI, and doesn't imply that LLMs are going to go away after it pops.

Re: SQLite Critical CVEs or LLM Slop?

#307

Post by Terr_ »

vouwfietsman wrote: post #303
Jensson wrote: post #301

Earlier quoted context omitted.

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it. Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective l…

> I don't think LLM will ever stop being dumb without such an introspective part to them. The very second they add this part, it will "just be a dumb token predictor with introspection", mark my words.

Necessary is not the same as sufficient.

If I want my ox-cart to fly, I need to add wings, but that's probably not all that needs changing.

Re: SQLite Critical CVEs or LLM Slop?

#308

Post by Jensson »

vouwfietsman wrote: post #302
Jensson wrote: post #215

Earlier quoted context omitted.

The rate of progress can be high and they can also be dumb next token predictors. Not sure why that is hard to understand. These models can do a lot of things but they also can't do a lot of things. In order to use these models effectively you have to understand that they are next token predictors and how that allows it to do what they do.

It can be a token predictor and still tell me exactly how my life will proceed from now until the indefinite future, or be the most intelligent conversational entity you have ever witnessed. The issue is of course with using the word "dumb": they are next token predictors, no doubt about it, but whether LLms as a class of system are smart or dumb is entirely unknown and entirely variable in time. To interact with the…

The "dumb" part comes from how it behaves in contexts where it lacks a lot of data, or where the data is skewed. Since they are tuned to give a prediction anyway and just make something up since sometimes those made up things are useful they will produce dumb results.

So people call them dumb since like dumb people they make strong statements about things they don't understand. And it doesn't matter how much smart things you encode them with, they will keep making strong statements about things they don't understand until they are fundamentally changed.

But since LLM are very smart about things where they have extensive data they can still be used to reliable solve many problems and probably in the future where we understand that better almost completely replace most lawyer and doctors work etc, because a lot of what a frontline doctor or basis lawyer work is very repetitive and can be encoded with billions of examples and decision paths into an expert system framework the LLM will follow.

So people say LLM are dumb since LLM will always keep making dumb statements. This is the same way we call Elon Musk dumb for making a lot of dumb statements, he is a smart guy but he makes dumb statements so her is dumb.

Re: SQLite Critical CVEs or LLM Slop?

#309

Post by Jensson »

vouwfietsman wrote: post #303
Jensson wrote: post #301

Earlier quoted context omitted.

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it. Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective l…

> I don't think LLM will ever stop being dumb without such an introspective part to them. The very second they add this part, it will "just be a dumb token predictor with introspection", mark my words.

If it keeps doing dumb things, yeah. But if that actually solves it then those opinions will quickly disappear when it replaces all human white collar work since it does it cheaper and better and faster.

AGI is fairly easy to detect for this reason. Does this system make a majority feel you don't have to hire people anymore? If not its not AGI.

Re: SQLite Critical CVEs or LLM Slop?

#310

Post by jimmaswell »

Jensson wrote: post #301
vouwfietsman wrote: post #292

Earlier quoted context omitted.

Does a dog possess intelligence? Does a bird? Does a cricket? An amoeba? I hate AI slop as much as the next guy but the amount of tribalism over AI is taking near-religious forms. Nobody knows what intelligence is, therefore we don't know what does or does not possess it, therefore we don't know whether LLMs currently, or in the future, possess it. Yes, LLMs can be stupid, guess what: so can I. That doesn't really ch…

"Dumb next token predictor" keeps popping up since that is the core way they work. Since they aren't logic engines but prediction engines they will always return a result regardless what you ask it. Some predictions might be the tokens "I don't know", but that is based on the model mapping your text to those tokens by having seen many similar "I don't know" responses to such contexts, it didn't do any introspective l…

I strongly suspect that if you dug deeply enough into how a human brain does reasoning, the answer would boil down to "it tries a bunch of pathways and picks the one that seems right". Pathways weighted by experience, but ultimately still guess-and-check. What would the alternative be?

Maybe our ability to reason is not as mystical and special as our ego might hope it to be, and discomfort over LLM's bringing that to light is the root of some people's urge to continuously downplay and discredit them.

(As far as I can tell as a non-neuroscientist, the literature on how our default mode network and prefrontal cortex interact agrees with my assessment - free association and logical verification respectively)

Post reply on HN