Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

101–110 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#101

Post by tyg13 »

naasking wrote: post #95
coliveira wrote: post #38

Earlier quoted context omitted.

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…

> The problem with LLMs is that they don't have a process to guarantee that a solution is correct Neither do we. > They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. As do we, and so you can correctly reframe the issue as "there's a gap between the quality of AI heuristics and the quality of human heuristics". That the gap is still shrin…

I'll never doubt the ability of people like yourself to consistently mischaracterize human capabilities in order to make it seem like LLMs' flaws are just the same as (maybe even fewer than!) humans. There are still so many obvious errors (noticeable by just using Claude or ChatGPT to do some non-trivial task) that the average human would simply not make.

And no, just because you can imagine a human stupid enough to make the same mistake, doesn't mean that LLMs are somehow human in their flaws.

> the gap is still shrinking though

I can tell this human is fond of extrapolation. If the gap is getting smaller, surely soon it will be zero, right?

Re: Beliefs that are true for regular software but false when applied to AI

#102

Post by chasing0entropy »

70 years ago we were fascinated by the concept of converting analog to a perfect digital copy. In reality, that goal was a pipe drea!m and the closest we can ever get is a near identical facimile to which data fits... But it's still quite easy to determine digital from true analog with rudimentary means.

Human thought is analog. It is based on chemical reactions, time, and unpredictably (effectively) random physical characteristics. AI is an attempt to turn that which is purely digital into an rational analog thought equivalent.

No matter how much effort, money, power, and rare mineral eating TPUs will - ever - produce true analog data.

Re: Beliefs that are true for regular software but false when applied to AI

#103

Post by tobias3 »

nancyminusone wrote: post #16
xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

To me, the greatest threat is information pollution. Primary sources will be diluted so heavily in an ocean of generated trash that you might as well not even bother to look through any of it.

And it imitates all the unimportant bits perfectly (like spelling, grammar, word choice) while failing at the hard to verify important bits (truth, consistency, novelty)

Re: Beliefs that are true for regular software but false when applied to AI

#104

Post by wlesieutre »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

And the "cooperative multitasking" in old operating systems where one program locking up meant the whole system was locked up

Re: Beliefs that are true for regular software but false when applied to AI

#105

Post by Yoric »

noir_lord wrote: post #75
kstrauser wrote: post #65

Earlier quoted context omitted.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

I grew up in the same era and I recall crashes being less frequent.

There were plenty of other issues, including the fact that you had to adjust the right IRQ and DMA for your Sound Blaster manually, both physically and in each game, or that you needed to "optimize" memory usage, enable XMS or EMS or whatever it was at the time, or that you spent hours looking at the nice defrag/diskopt playing with your files, etc.

More generally, as you hint to, desktop operating systems were crap, but the software on top of it was much more comprehensively debugged. This was presumably a combination of two factors: you couldn't ship patches, so you had a strong incentive to debug it if you wanted to sell it, and software had way fewer features.

Come to think about it, early browsers kept crashing and taking down the entire OS, so maybe I'm looking at it with rosy glasses.

Re: Beliefs that are true for regular software but false when applied to AI

#106

Post by krior »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

Gone? I had two last year, lets not overstate things.

Re: Beliefs that are true for regular software but false when applied to AI

#107

Post by wmf »

nlawalker wrote: post #28

Where did "can't you just turn it off?" in the title come from? It doesn't appear anywhere in the actual title or the article, and I don't think it really aligns with its main assertions.

Turning AI off comes up a lot in existential risk discussions so I was surprised the article isn't about that.

Re: Beliefs that are true for regular software but false when applied to AI

#108

Post by largbae »

chasing0entropy wrote: post #102

70 years ago we were fascinated by the concept of converting analog to a perfect digital copy. In reality, that goal was a pipe drea!m and the closest we can ever get is a near identical facimile to which data fits... But it's still quite easy to determine digital from true analog with rudimentary means. Human thought is analog. It is based on chemical reactions, time, and unpredictably (effectively) random physical…

This is all true. But digital audio and video media has captured essentially all economic value outside of live performance. So it seems likely that we will find a "good enough" in this domain too.
Post reply on HN