Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

171–180 of 406 posts

Re: SQLite Critical CVEs or LLM Slop?

#171

Post by minraws »

I think this will give a lot of ammunition for people on both sides, but the thing is we have made software so complex, ofc a lot of it is inherent, that we can't even tell if the vulnerability is hallucinated or not.

I find it getting closer and closer to the nonsensical 50 page mathematical proofs built with computer assited tools, that been driving reason for abstract proof systems so that we can break down such complicated examples into something simpler that we can feed into a system to help us verify.

I feel like for some reason we have built the output machine before we could even build the actual output verification machine. And aren't ready to accept that we should all move towards building some kind of system to better help verify the "truth" produced by these AI systems.

I am now very convinced that human built and proven to work verification systems for LLM outputs are more important than ever, but I just don't see any company or team working towards it.

Trillion dollar systems shipping to production code that is hardly understood by any one without any verification what really could go wrong. Honestly I really truly hope nothing goes wrong.

To note I am not anti-AI but I want to be able to have my LLM outputs be verified by more than just me under a tighter and tighter deadlines, accepting worse and worse slop just because it would less cost to me to accept the PR and then fix it in prod.

LLMs aren't sustainable software development, they are delusional slop factories if we aren't able to actually make sure the outputs have any amount of correctness.

With humans it worked because all 10 of us could in some ways verify each others outputs, and had trust in the fact that our timelines included the time for reviews.

Re: SQLite Critical CVEs or LLM Slop?

#172

Post by prh8 »

elmer2 wrote: post #168
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Many people with no skills are taking advantage of the LLM craze to artificially inflate their own value. I see it every day on LinkedIn. People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

Countless directors and managers are now cosplaying as engineers. I've seen so many myself and that's just my tiny slice of this engineering world

Re: SQLite Critical CVEs or LLM Slop?

#175

Post by jedbrooke »

nonexistent functions being referenced is pretty bad, I haven’t had that issue since like opus 4. Shows either complete lack of oversight and/or using a bargain basement model (so no even having financial skin in the game). Most likely both

Re: SQLite Critical CVEs or LLM Slop?

#176

Post by tsunamifury »

gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

[flagged]

Re: SQLite Critical CVEs or LLM Slop?

#177

Post by linuxhansl »

I almost feel that we have a new generation of the "script-kiddie". People without much (or any?) knowledge of software engineering using outside tooling to do "stuff" they wouldn't be able to do themselves.

Maybe it's a not a perfect analogy - in this case the intentions seemed to be laudable - but we will see much more of this, including from malicious actors.

Re: SQLite Critical CVEs or LLM Slop?

#178

Post by SoftTalker »

vrighter wrote: post #64
lucideer wrote: post #9

Earlier quoted context omitted.

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

I have been given a list by security. "We had an automated tool scan that machine. It reported these. Fix anything medium severity and above. Never mind that some of them involved vulnerabilities in some part of the bluetooth stack (servers in our datacenter don't even have bluetooth). But they just didn't care

So run apt full-upgrade and get the new bluetooth driver. Why bother with a fight over something that isn't even used? Just do the quickest thing to get it off your plate.

Re: SQLite Critical CVEs or LLM Slop?

#179

Post by Reptur »

elmer2 wrote: post #168
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Many people with no skills are taking advantage of the LLM craze to artificially inflate their own value. I see it every day on LinkedIn. People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

This isn't new due to AI, just amplified.

Re: SQLite Critical CVEs or LLM Slop?

#180

Post by eli »

elmer2 wrote: post #168
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Many people with no skills are taking advantage of the LLM craze to artificially inflate their own value. I see it every day on LinkedIn. People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

To be fair, "people with no skills inflating their own value" is what LinkedIn has always been like. But I guess LLMs are uniquely well positioned for that task.
Post reply on HN