Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

131–140 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#131

Post by dist-epoch »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

Mostly because Microsoft shut down kernel access, wrote it's own generic drivers for "simple" devices (USBs, printers, sound cards, ...) and made "heavy" drivers submit to their WHQL quality control to be signed to run.

Re: Beliefs that are true for regular software but false when applied to AI

#132

Post by Izkata »

meonkeys wrote: post #46
nlawalker wrote: post #28

Where did "can't you just turn it off?" in the title come from? It doesn't appear anywhere in the actual title or the article, and I don't think it really aligns with its main assertions.

It shows up at https://boydkane.com under the link "Why your boss isn't worried about advanced AI". Must be some kind of sub-heading, but not part of the actual article / blog post. Presumably it's a phrase you might hear from a boss who sees AI as similar to (and as benign/known/deterministic as) most other software, per TFA

It's a sci-fi thing, think of it along the lines of "What do you mean Skynet has gone rogue? Can't you just turn it off?"

(I think something along these lines was actually in the Terminator 3 movie, the one where Skynet goes live for the first time).

Agreed though, no relation to the actual post.

Re: Beliefs that are true for regular software but false when applied to AI

#133

Post by kstrauser »

binarymax wrote: post #94
kstrauser wrote: post #65

Earlier quoted context omitted.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

You know, I had spent a good amount of years not having even a single thought about rational rose, and now that’s all over.

I do apologize. I couldn't bear this burden alone.

Re: Beliefs that are true for regular software but false when applied to AI

#134

Post by ponector »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

I guess that is because you run it on old hardware. When I've bought my Asus ROG expensive laptop I had bsod almost daily. A year later with all updates I had bsod once in a month on the same device and windows installation.

Re: Beliefs that are true for regular software but false when applied to AI

#135

Post by rkomorn »

krior wrote: post #106
malfist wrote: post #85

Earlier quoted context omitted.

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

Gone? I had two last year, lets not overstate things.

My anecdata is that my current PC is four years old, with the same OS install, and I can't even recall if I've seen one BSoD.

Re: Beliefs that are true for regular software but false when applied to AI

#136

Post by ben_w »

tyg13 wrote: post #101
naasking wrote: post #95

Earlier quoted context omitted.

> The problem with LLMs is that they don't have a process to guarantee that a solution is correct Neither do we. > They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. As do we, and so you can correctly reframe the issue as "there's a gap between the quality of AI heuristics and the quality of human heuristics". That the gap is still shrin…

I'll never doubt the ability of people like yourself to consistently mischaracterize human capabilities in order to make it seem like LLMs' flaws are just the same as (maybe even fewer than!) humans. There are still so many obvious errors (noticeable by just using Claude or ChatGPT to do some non-trivial task) that the average human would simply not make. And no, just because you can imagine a human stupid enough to…

> doesn't mean that LLMs are somehow human in their flaws.

I don't believe anyone is suggesting that LLMs flaws are perfectly 1:1 aligned with human flaws, just that both do have flaws.

> If the gap is getting smaller, surely soon it will be zero, right?

The gap between y=x^2 and y=-x^2-1 gets closer for a bit, fails to ever become zero, then gets bigger.

The difference between any given human (or even all humans) and AI will never be zero: Some future AI that can only do what one or all of us can do, can be trivially glued to any of that other stuff where AI can already do better, like chess and go (and stuff simple computers can do better, like arithmetic).

Re: Beliefs that are true for regular software but false when applied to AI

#137

Post by naasking »

tyg13 wrote: post #101
naasking wrote: post #95

Earlier quoted context omitted.

> The problem with LLMs is that they don't have a process to guarantee that a solution is correct Neither do we. > They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. As do we, and so you can correctly reframe the issue as "there's a gap between the quality of AI heuristics and the quality of human heuristics". That the gap is still shrin…

I'll never doubt the ability of people like yourself to consistently mischaracterize human capabilities in order to make it seem like LLMs' flaws are just the same as (maybe even fewer than!) humans. There are still so many obvious errors (noticeable by just using Claude or ChatGPT to do some non-trivial task) that the average human would simply not make. And no, just because you can imagine a human stupid enough to…

> I'll never doubt the ability of people like yourself to consistently mischaracterize human capabilities

Ditto for your mischaracterizations of LLMs.

> There are still so many obvious errors (noticeable by just using Claude or ChatGPT to do some non-trivial task) that the average human would simply not make.

Firstly, so what? LLMs also do things no human could do.

Secondly, they've learned from unimodal data sets which don't have the rich semantic content that humans are exposed to (not to mention born with due to evolution). Questions that cross modal boundaries are expected to be wrong.

> If the gap is getting smaller, surely soon it will be zero, right?

Quantify "soon".

Re: Beliefs that are true for regular software but false when applied to AI

#138

Post by alganet »

SalientBlue wrote: post #83
alganet wrote: post #54

Earlier quoted context omitted.

He is. Maybe he's just running with the pack, but that doesn't matter either. The fact is, we kind of know how to prevent problems in AI systems: - Good benchmarks. People said several times that LLMs display erratic behavior that could be prevented. Instead of adjusting the benchmarks (which would slow down development), they ignored the issues. - Accountability frameworks. Who is responsible when an AI fails? How t…

>But it's a different we already know 'we' is the operative word here. 'We', meaning technical people who have followed this stuff for years. The target audience of this article are not part of this 'we' and this stuff IS completely new _for them_. The target audience are people who, when confronted with a problem with an LLM, think it is perfectly reasonable to just tell someone to 'look at the code' and 'fix the bu…

Let's pretend I'm the audience, and imagine that in the past I said those things ("fix the bug" and "look at the code").

What should I say now? "AI works in mysterious ways"? Doesn't sound very useful.

Also, should I start parroting innacurate outdated generalizations about regular software?

The post doesn't teach anything useful for a beginner audience. It's bamboozling them. I am amazed that you used the audience perspective as a defense of some kind. It only made it worse.

Please, please, take a moment to digest my critique properly. Think about what you just said and what that implies. Re-read the thread if needed.

Re: Beliefs that are true for regular software but false when applied to AI

#139

Post by Podrod »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

They're definitely not gone.

Re: Beliefs that are true for regular software but false when applied to AI

#140

Post by __loam »

freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

I'm happy they ate shit here because I like my mac not getting co-pilot bullshit forced into it, but apparently Apple had two separate teams competing against each other on this topic. Supposedly a lot of politics got in the way of delivering on a good product combined with the general difficulty of building LLM products.
Post reply on HN