Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

351–360 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#351

Post by ghostnshells »

I think this is expected with the increased usage of AI on both fronts (finding vulns and automating their remediation). Yes, there are and will be companies that help other companies validate CVEs before they raise any alarms, but that noise/signal battle has always been there.

I work as a sec eng at a biotech firm and have a tool I built that pulls our ITIL and runs a search on NVD, GHSA, etc. for enrichment and classification. But the number of CVEs I see for products we house is astronomical compared to the actual version of that product we are running (and that's only one value). But that's also a challenge when working with so many sources and having a small team try to tackle them.

The age-old validate, remediate, verify proves itself useful once again!

Re: SQLite Critical CVEs or LLM Slop?

#352

Post by rafterydj »

gortok wrote: post #321
bigbuppo wrote: post #318

Earlier quoted context omitted.

> A human must verify everything an LLM presents as fact. Everything. If you don’t, we all pay the price. LLMs do not remove the onus of responsibility on the human being, if anything they amplify it because LLMs can generate lots more output more quickly that needs to be verified than humans can. The sort of person that's going to offload their thinking to AI is the exact sort of person that is not going to verify a…

For a long time I was anti-licensure in tech; now with the bar being lowered to next to nothing, it seems as if licensure is more important than ever — not to protect this trade (though it will do that, and that is a benefit), but because the sheer amount of irresponsibility in the usage of LLMs and “AI” in general begs for licensure and adoption of a regulatory body for software in general.

This is unfortunately a feeling I share. It wasn't until LLMs have become nearly ubiquitous at this point, and there has been zero realistic technological response to the dangers they present. Not to mention I suspect there may be some psychological element to being exposed to interactions with AI models and their nonsense for hours a day. Not all of it is nonsense....but you won't ever know for sure.

Re: SQLite Critical CVEs or LLM Slop?

#353

Post by estearum »

Jensson wrote: post #345
estearum wrote: post #344

Earlier quoted context omitted.

Okay, then what part of the brain is doing anything other than transforming an input into an output based on what inputs that part of the brain has previously been exposed to?

> transforming an input into an output based on what inputs that part of the brain has previously been exposed to You changed the definition there, for it to be like an LLM it should be: > transforming an input into an output trying to mimic inputs that part of the brain has previously been exposed to Anyone can see how that limits you a lot, and why that makes it so much harder for LLM to learn things properly than…

Well no, you changed the definition. LLMs don't just mimic what they've been trained on. They mimic what they're rewarded for. Like a brain!

Pre-training is just direct mimicry. A pre-trained LLM is very stupid and mostly useless. To become useful they are post-trained with a reward function.

Re: SQLite Critical CVEs or LLM Slop?

#354

Post by buttercraft »

red75prime wrote: post #341
buttercraft wrote: post #324

Earlier quoted context omitted.

I did not express my opinion or "worldview." I was simply responding to this: > exclude LLMs with CoT from the category of intelligent systems with certainty

Why do you think that intelligence of a Turing-complete system trained on human data can be likened to a Russell's teapot? At least, don't you think that the recent mathematical results of LLMs are a bit like a glimpse of something teapot-shaped in the orbit? (which makes it not a Russell's teapot, which. by definition, can't be observed). To me, it's an expected progression of ANNs' approximation of human cognitive…

> Why do you think that intelligence of a Turing-complete system trained on human data can be likened to a Russell's teapot?

I don't? You are presenting opinions as if they are mine, but they are not.

Re: SQLite Critical CVEs or LLM Slop?

#355

Post by chlorion »

estearum wrote: post #256
tadfisher wrote: post #189

Earlier quoted context omitted.

This deserves about as large of a "[citation needed]" as one could draw. Are you well-studied in neuroscience?

Nothing in the universe is anything other than a chain of reactions, amigo. The brain is an object in the universe. The universe has quantum behaviors (fully random, not a source of intelligence) and it has deterministic behaviors (fully non-random). Many of those deterministic behaviors are so complex that they're easier to analyze and describe as probabilistic, which is where most brain input → output relationships…

Its baffling to me that people think the brain is somehow "outside" of the universe and somehow "magic" and not tied down by physics just like any other matter is. I don't understand how people come to this conclusion, except in the case of belief in magic or religion (which is belief in magic more or less).

Re: SQLite Critical CVEs or LLM Slop?

#356

Post by estearum »

chlorion wrote: post #355
estearum wrote: post #256

Earlier quoted context omitted.

Nothing in the universe is anything other than a chain of reactions, amigo. The brain is an object in the universe. The universe has quantum behaviors (fully random, not a source of intelligence) and it has deterministic behaviors (fully non-random). Many of those deterministic behaviors are so complex that they're easier to analyze and describe as probabilistic, which is where most brain input → output relationships…

Its baffling to me that people think the brain is somehow "outside" of the universe and somehow "magic" and not tied down by physics just like any other matter is. I don't understand how people come to this conclusion, except in the case of belief in magic or religion (which is belief in magic more or less).

Yeah, that's ultimately this whole conversation and people just don't have the courage to acknowledge their own beliefs.

The brain is a (very complex, incredible) input → output machine. That's it! It's incredible!

I don't understand why people are so afraid of this that they will believe otherwise with literally zero evidence whatsoever.

Re: SQLite Critical CVEs or LLM Slop?

#357

Post by iib »

clbrmbr wrote: post #154
dwedge wrote: post #47

Earlier quoted context omitted.

> Tools like trivvy make it possible to do the scans... Only if you didn't rip trivvy out of your organisation when it had two supply chain compromises within a month of each other earlier this year

Yikes. Man, there’s a market opening for someone to redistribute open source projects with supply chain assurances!

There is a market for that, indeed. Hardened container images and hardened CI actions have been a thing for a while, with some companies providing exactly that.

Re: SQLite Critical CVEs or LLM Slop?

#358

Post by ivan_gammel »

kentm wrote: post #330
ivan_gammel wrote: post #272

Earlier quoted context omitted.

The due diligence not being done is people putting cats in microwaves. It‘s not the dangerous part. The real danger is risk assessments coming to wrong conclusions, because it is still terra incognita. Talented engineers were in this situation before, doing mistakes with cars, airplanes, buildings etc.

No, I'm sorry but I think thats a cop out. The fact that LLM are stochastic and can give incorrect answers is not particularly difficult to comprehend, and the risks that fall out of that are reasonably understandable. The issue is entirely down to bad choices by the people driving LLMs, because they are engaging with what they wish LLMs do instead of what they actually do.

So you say that it is easy to understand what LLMs are, yet the issue is that people don‘t understand it. Either you put too many people in the bucket labelled „idiots“ or you are blowing that problem out of proportion, at the same time seeing only the tip of the iceberg.

Risks are not binary. Most people are not idiots. They do understand that LLMs aren‘t thinking humanly or 100% logically. You are talking about some sort of faith, I think this is more about trust, which is built on observations. Statistically, the outcomes of LLM work may hit the goal quite a lot. More of that in the beginning of the journey than in the end. For many reasonable people that becomes a trap, where minor, acceptable deviations accumulate into a fireball. Still, saying that LLMs are failure with inevitable fireball at the end and giving up is a trivial and stupid solution. Staying within razor-thin distance from that fireball and managing to deliver a working solution is what everyone tries at the moment. This is how it always happened and always worked. It‘s the art of engineering.

Re: SQLite Critical CVEs or LLM Slop?

#359

Post by Versipelle »

rowls66 wrote: post #156
MattPalmer1086 wrote: post #104

Earlier quoted context omitted.

In our organisation, my security team spend the majority of our effort weeding out unexploitable CVEs - which are the majority of them. But the volume is growing year on year - I don't know how sustainable it's going to be.

Sounds like a good job for AI. I do fear that the future will involve deploying AI do deal with the mountain of slop produced by other AI's.

I agree, this is a trend I'm seeing all over the place with generative AI: first, break a functioning, human-based workflow by flooding it with AI generated content. Blame the persons maintaining the workflow for being the bottleneck and hindering innovation. Then, implement generative AI in the workflow review or approval process to keep up the pace, irrespective of how valuable the extra generated stuff has been.

Re: SQLite Critical CVEs or LLM Slop?

#360

Post by everforward »

inigyou wrote: post #223
everforward wrote: post #113

Earlier quoted context omitted.

They’re not bad if you give up on being able to reproduce arcane bugs. Don’t pin versions, pull everything from latest, CVEs are just a rebuild. Certainly not an amazing idea, but it solves the problem of the day. The funnier, Kafakaesque problem of the day is interactions with mandatory cooldown periods on new versions because of supply chain risks. I’ve had a couple tickets get stuck because the CVE scanner says I…

> Don't pin versions, pull everything from latest Rebuild broke half your tests. People are exploiting the CVE against your system right now. Good luck!

You can’t fail unit tests you don’t have because your agent “fixed” them by deleting them. You have to go full galaxy brain for it to work lol
Post reply on HN