Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SQLite Critical CVEs or LLM Slop?

research.jfrog.com

Post reply on HN
Print view View original

371–380 of 407 posts

Re: SQLite Critical CVEs or LLM Slop?

#371

Post by abustamam »

elmer2 wrote: post #168
gortok wrote: post #52

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are. LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credi…

Many people with no skills are taking advantage of the LLM craze to artificially inflate their own value. I see it every day on LinkedIn. People that previously have barely any experience in tech, now being hired in AI startups because they are good bullshitters.

I mean most startups and every billion dollar company are worth a lot because they are good bullshitters, in a way its kinda refreshing to see more people able to bullshit their way up a different ladder, even if it is a bit scary.

Re: SQLite Critical CVEs or LLM Slop?

#373

Post by andrewflnr »

27183 wrote: post #364
gortok wrote: post #321

Earlier quoted context omitted.

For a long time I was anti-licensure in tech; now with the bar being lowered to next to nothing, it seems as if licensure is more important than ever — not to protect this trade (though it will do that, and that is a benefit), but because the sheer amount of irresponsibility in the usage of LLMs and “AI” in general begs for licensure and adoption of a regulatory body for software in general.

100%. I don't think "AI" has changed anything wrt to the responsibility of the tech industry in general. Tech has always been pretty much devoid of ethics or a sense of responsibility at the executive level (and therefore "management" more broadly). But before LLMs it was easier for ethical engineers to surreptitiously steer things towards responsible implementations. I suspect also competence correlates pretty stron…

But as you've made clear, the root of the problem is not with the engineer.

Personally I think a corporate death penalty strikes a nice balance of holding companies and individuals responsible. If a company exercises sufficient malfeasance, it gets nationalized with no compensation for shareholders. Watch the incentives align real fast.

Re: SQLite Critical CVEs or LLM Slop?

#374

Post by Blackthorn »

ryan_n wrote: post #264
Blackthorn wrote: post #259

Earlier quoted context omitted.

It is pretty funny to see the shoe on the other foot, since it's usually software engineers with unearned arrogance about other fields.

What fields do you see devs think they know about? I’ve never personally seen this with other devs I work with but obviously small sample size…

Literally everything. Just pick a topic and you'll see SWEs on Hacker News prognosticating about it as if they knew everything while knowing nothing but the surface.

Re: SQLite Critical CVEs or LLM Slop?

#375

Post by 27183 »

andrewflnr wrote: post #373
27183 wrote: post #364

Earlier quoted context omitted.

100%. I don't think "AI" has changed anything wrt to the responsibility of the tech industry in general. Tech has always been pretty much devoid of ethics or a sense of responsibility at the executive level (and therefore "management" more broadly). But before LLMs it was easier for ethical engineers to surreptitiously steer things towards responsible implementations. I suspect also competence correlates pretty stron…

But as you've made clear, the root of the problem is not with the engineer. Personally I think a corporate death penalty strikes a nice balance of holding companies and individuals responsible. If a company exercises sufficient malfeasance, it gets nationalized with no compensation for shareholders. Watch the incentives align real fast.

Yeah that sounds great! A company guillotine is absolutely something I could get behind. Problem is I don't think there's any way it could possibly happen politically. Whereas we have a really strong existence proof for e.g. the Professional Engineer certification and other professional coherence mechanisms. That seems to work, albeit not without its own inefficiencies. I doubt anyone, though, would advocate for some kind of laissez-faire situation where the profit incentive aligns with the regulatory one. Historically that hasn't gone well. A lot of people died.

Re: SQLite Critical CVEs or LLM Slop?

#376

Post by estearum »

noncoml wrote: post #372
estearum wrote: post #350

Earlier quoted context omitted.

Of course we do. Unless the brain is an object unlike anything else in the known universe, then it's an input → output machine.

Not what what the parent post claimed.

The brain is the source of intelligence

The brain solely does probabilistic conversion of inputs to outputs

Therefore the thing that makes us intelligent is our probabilistic conversion of inputs to outputs

Ergo GP's assertion is wrong

Re: SQLite Critical CVEs or LLM Slop?

#377

Post by adjfasn47573 »

batshit_beaver wrote: post #317
adjfasn47573 wrote: post #243

Earlier quoted context omitted.

> A human must verify everything an LLM presents as fact. Everything. I've thought about this for quite some time now. No. A human doesn't need to verify everything. And the argument is really simple: stochastic. Think of self-driving cars: We can show today - based on evidence and real data - that self-driving cars are safer than human drivers. That's a fact and the consequences are clear, more self-driving cars, le…

> Now back to AI for software creation. "Review is the bottleneck because EVERYTHING must be judged by a human." No. It doesn't. We just need to build AI review systems, that will do reviews better than (or at least as good as) humans. The human review quality bar is far below 100%. Far far far. If we can show (likely in the next 12-24 months I think) that AI review quality is consistently above the human review qual…

This is weird I didn't imply or say any of these things that you're arguing against right now.

Re: SQLite Critical CVEs or LLM Slop?

#378

Post by andrewflnr »

27183 wrote: post #375
andrewflnr wrote: post #373

Earlier quoted context omitted.

But as you've made clear, the root of the problem is not with the engineer. Personally I think a corporate death penalty strikes a nice balance of holding companies and individuals responsible. If a company exercises sufficient malfeasance, it gets nationalized with no compensation for shareholders. Watch the incentives align real fast.

Yeah that sounds great! A company guillotine is absolutely something I could get behind. Problem is I don't think there's any way it could possibly happen politically. Whereas we have a really strong existence proof for e.g. the Professional Engineer certification and other professional coherence mechanisms. That seems to work, albeit not without its own inefficiencies. I doubt anyone , though, would advocate for som…

PEs are demonstrably a partial solution at best. Fields with certified engineers still have financially motivated disasters (boeing goes the door plug).

For better or worse, I think we'll see more political support for increased oversight of companies on the next free decades. I only hope it manifests in a useful way instead of a self-defeating way.

Re: SQLite Critical CVEs or LLM Slop?

#379

Post by 27183 »

andrewflnr wrote: post #378
27183 wrote: post #375

Earlier quoted context omitted.

Yeah that sounds great! A company guillotine is absolutely something I could get behind. Problem is I don't think there's any way it could possibly happen politically. Whereas we have a really strong existence proof for e.g. the Professional Engineer certification and other professional coherence mechanisms. That seems to work, albeit not without its own inefficiencies. I doubt anyone , though, would advocate for som…

PEs are demonstrably a partial solution at best. Fields with certified engineers still have financially motivated disasters ( boeing goes the door plug). For better or worse, I think we'll see more political support for increased oversight of companies on the next free decades. I only hope it manifests in a useful way instead of a self-defeating way.

It'll have to be pretty much all the ways. One way will stick. The rest will be forgotten failures.

Re: SQLite Critical CVEs or LLM Slop?

#380

Post by adjfasn47573 »

gspr wrote: post #271
adjfasn47573 wrote: post #243

Earlier quoted context omitted.

> A human must verify everything an LLM presents as fact. Everything. I've thought about this for quite some time now. No. A human doesn't need to verify everything. And the argument is really simple: stochastic. Think of self-driving cars: We can show today - based on evidence and real data - that self-driving cars are safer than human drivers. That's a fact and the consequences are clear, more self-driving cars, le…

This reduction of everything to stochasticity is silly. Or, to put it differently: Do you accept a value with some error appearing in your bank account on salary day? We have plenty of systems where complete accuracy is the only acceptable thing. Computers are great for such things. Until we all get caught up in a way of delusion and start writing those systems as natural prose passed through an improperly understood…

I've never said or implied what you're arguing against right now.

An LLM building software benefits from using a formal language, formal tests or having a formal specification in a similar way we do. Everything you can formalize into something, so you have certainty about stuff, is always a benefit.

I wasn't comparing exact science with LLMs. I was comparing current messy review processes of code performed by humans with future messy review processes of code likely performed by LLMs.

If you find a way to convert code reviews into a fully formalized process, then this is clearly the winner. If the LLMs find a way to do that, same.

If you find a way to formalize the process of driving through any street in any situation, then this is clearly the winner. Until then, driving stays messy and stochastic.

Post reply on HN