Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

441–450 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#441

Post by beyarkay »

dkersten wrote: post #20
SalientBlue wrote: post #9

Earlier quoted context omitted.

You should read the footnote marked [1] after "a note for technical folk" at the beginning of the article. He is very consciously making sweeping generalizations about how software works in order to make things intelligible to non-technical readers.

Sure, but: > these claims mostly hold, but they break down when applied to distributed systems, parallel code, or complex interactions between software systems and human processes The claims the GP quoted DON’T mostly hold, they’re just plain wrong. At least the last two, anyway.

Say more? I stand by my statement, but you're not specific enough for me to explain why I believe I'm correct.

Re: Beliefs that are true for regular software but false when applied to AI

#442

Post by beyarkay »

alganet wrote: post #33
SalientBlue wrote: post #9

Earlier quoted context omitted.

You should read the footnote marked [1] after "a note for technical folk" at the beginning of the article. He is very consciously making sweeping generalizations about how software works in order to make things intelligible to non-technical readers.

Does that really matter? He is trying to lax the general public perception around AIs shortcomings. He's giving AI a break, at the expense of regular developers. This is wrong on two fronts: First, because many people foresaw the AI shortcomings and warned about them. This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks. Again, lo…

> He is trying to lax the general public perception around AIs shortcomings

This is not at all what I'm trying to do. This same essay is cross-posted on LessWrong[1] because I think ASI is the most dangerous problem of our time.

> This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks

I'm not sure how I can say "your intuitions are wrong and you should be careful" and have that be misintepreted as "ignore the problems around AI"

[1]: https://www.lesswrong.com/posts/ZFsMtjsa6GjeE22zX/why-your-b...

Re: Beliefs that are true for regular software but false when applied to AI

#443

Post by beyarkay »

rester324 wrote: post #183
alganet wrote: post #5

> here are some example ideas that are perfectly true when applied to regular software Hm, I'm listening, let's see. > Software vulnerabilities are caused by mistakes in the code That's not exactly true. In regular software, the code can be fine and you can still end up with vulnerabilities. The platform in which the code is deployed could be vulnerable, or the way it is installed make it vulnerable, and so on. > Bug…

I thought this blog post was a parody. And to my surprise both the author and the audience takes it seriously. Weird

Why did you think it a parody?

Re: Beliefs that are true for regular software but false when applied to AI

#444

Post by beyarkay »

nlawalker wrote: post #63
meonkeys wrote: post #46

Earlier quoted context omitted.

It shows up at https://boydkane.com under the link "Why your boss isn't worried about advanced AI". Must be some kind of sub-heading, but not part of the actual article / blog post. Presumably it's a phrase you might hear from a boss who sees AI as similar to (and as benign/known/deterministic as) most other software, per TFA

Ah, thanks for that! >Presumably it's a phrase you might hear from a boss who sees AI as similar to (and as benign/known/deterministic as) most other software, per TFA Yeah I get that, but I think that given the content of the article, "can't you just fix the code?" or the like would have been a better fit.

Yeah this is my mistake, the phrase "can't you just turn it off" was in several drafts but got edited out, and I missed that during the publishing of the essay.

Re: Beliefs that are true for regular software but false when applied to AI

#445

Post by beyarkay »

kazinator wrote: post #2

> AIs will get more reliable over time, like old software is more reliable than new software. :) Was that a humam Freudian slip, or artificial one? Yes, old software is often more reliable than new.

I was maybe a little unclear with the phrasing, I meant to say "software that's been around for 20 years is more reliable than software that's been around for 2 months"

Re: Beliefs that are true for regular software but false when applied to AI

#446

Post by beyarkay »

crottypeter wrote: post #298
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

But at the time that software was "new" and unreliable.

ah yes, one person understands the point I was trying to make (:

Re: Beliefs that are true for regular software but false when applied to AI

#447

Post by beyarkay »

gridspy wrote: post #164
kstrauser wrote: post #65

Earlier quoted context omitted.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

I think old in this sense is "released" rather than "beta" - it takes time to make any software reliable. Many of the examples here further prove that young software is unreliable. Remember when debuggers were young? Remember when OSes were young? Remember when multi-tasking CPUs were young? Etc...

Yes this is what I was trying to say. Code that's been in production for 20 years has less bugs than code that's been around for 20 minutes

Re: Beliefs that are true for regular software but false when applied to AI

#448

Post by beyarkay »

jayd16 wrote: post #220
kstrauser wrote: post #65

Earlier quoted context omitted.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

You misunderstand. They are explicitly referring to the survivors that have been iterated on and chosen for being good. They're NOT saying all software in the past was better.

correct.

Re: Beliefs that are true for regular software but false when applied to AI

#449

Post by beyarkay »

izzydata wrote: post #27
joomla199 wrote: post #23

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Sounds more like survivorship bias. All the bad codebases were thrown out and only the good ones lasted a long time.

well, yes, exactly. I'm not trying to claim that old code is more reliable just because it was written a long time ago, I'm making the claim that old code is more reliable because of the survivorship bias. If code was first written 20 years ago and is still in production, unchanged, I can be relatively certain there's no stop-the-world bugs in those lines. (this says nothing about how pretty the code is, though).

Re: Beliefs that are true for regular software but false when applied to AI

#450

Post by beyarkay »

topaz0 wrote: post #347
izzydata wrote: post #27

Earlier quoted context omitted.

Sounds more like survivorship bias. All the bad codebases were thrown out and only the good ones lasted a long time.

Survivorship bias is real, but is missing the important piece of the story when it comes to software, which doesn't just survive but is also maintained. Sure you may choose to discard/replace low quality software and keep high quality software in operation, which leads to survivorship bias, but the point here is that you also have a chance to find and fix issues in the one that survived, even if those issues weren't…

well said, yes this is the point I was (apparently failing) to make
Post reply on HN