Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

41–50 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#41

Post by shakna »

prasadjoglekar wrote: post #35
giancarlostoro wrote: post #25

Earlier quoted context omitted.

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

It actually might. Older code running in production is almost automatically regression tested with each new fix. It might not be pretty, but it's definitely more reliable for solving real problems.

The list of bugs tagged regression at work certainly suggests it gets tested... But fixing those regressions...? That's a lot of dev time for things that don't really have time allocated for them.

Re: Beliefs that are true for regular software but false when applied to AI

#42

Post by SalientBlue »

alganet wrote: post #33
SalientBlue wrote: post #9

Earlier quoted context omitted.

You should read the footnote marked [1] after "a note for technical folk" at the beginning of the article. He is very consciously making sweeping generalizations about how software works in order to make things intelligible to non-technical readers.

Does that really matter? He is trying to lax the general public perception around AIs shortcomings. He's giving AI a break, at the expense of regular developers. This is wrong on two fronts: First, because many people foresaw the AI shortcomings and warned about them. This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks. Again, lo…

I don't think he's doing that at all. The article is pointing out to non-technical people how AI is different than traditional software. I'm not sure how you think it's giving AI a break, as it's pointing out that it is essentially impossible to reason about. And it's not at the expense of regular developers because it's showing how regular software development is different than this. It makes two buckets, and puts AI in one and non-AI in the other.

Re: Beliefs that are true for regular software but false when applied to AI

#43

Post by jongjong »

This article makes a solid case. The worst kinds of bugs in software are not the most obvious ones like syntax errors, they are the ones where the code appears to be working correctly, until some users do something slightly unusual after a few weeks of some code change being deployed and it breaks spectacularly but the bug only affects a small fraction of users so developers cannot reproduce the issue... And the cose change happened such time ago that the guilty code isn't even suspected.

Re: Beliefs that are true for regular software but false when applied to AI

#44

Post by wvenable »

izzydata wrote: post #27
joomla199 wrote: post #23

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Sounds more like survivorship bias. All the bad codebases were thrown out and only the good ones lasted a long time.

In my experience actively maintained but not heavily modified applications tend towards stability over time. It don't even matter if they are good or bad codebases -- even a bad code will become less buggy over time if someone is working on bug fixes.

New code is the source of new bugs. Whether that's an entirely new product, a new feature on an existing project, or refactoring.

Re: Beliefs that are true for regular software but false when applied to AI

#46

Post by meonkeys »

nlawalker wrote: post #28

Where did "can't you just turn it off?" in the title come from? It doesn't appear anywhere in the actual title or the article, and I don't think it really aligns with its main assertions.

It shows up at https://boydkane.com under the link "Why your boss isn't worried about advanced AI". Must be some kind of sub-heading, but not part of the actual article / blog post.

Presumably it's a phrase you might hear from a boss who sees AI as similar to (and as benign/known/deterministic as) most other software, per TFA

Re: Beliefs that are true for regular software but false when applied to AI

#47

Post by hedora »

pavel_lishin wrote: post #26
SalientBlue wrote: post #9

Earlier quoted context omitted.

You should read the footnote marked [1] after "a note for technical folk" at the beginning of the article. He is very consciously making sweeping generalizations about how software works in order to make things intelligible to non-technical readers.

But are those sweeping generalizations true? > I’m also going to be making some sweeping statements about “how software works”, these claims mostly hold, but they break down when applied to distributed systems, parallel code, or complex interactions between software systems and human processes. I'd argue that this describes most software written since, uh, I hesitate to even commit to a decade here.

At least the 1950’s. That’s when stuff like asynchrony and interrupts were worked out. Dijkstra wrote at length about this in reference to writing code that could drive a teletype (which had fundamentally non-deterministic timings).

If you include analog computers, then there are some WWII targeting computers that definitely qualify (e.g., on aircraft carriers).

Re: Beliefs that are true for regular software but false when applied to AI

#48

Post by Animats »

toomuchtodo wrote: post #19
ljlolel wrote: post #15

Earlier quoted context omitted.

CEOs (even most VCs) are labor too

Labor competes for compensation, CEOs compete for status (above a certain enterprise size, admittedly). Show me a CEO willingly stepping down to be replaced by generative AI. Jamie Dimon will be so bold to say AI will bring about a 3 day week (because it grabs headlines [1]) but he isn't going to give up the status of running JPMC; it's all he has besides the wealth, which does not appear to be enough. The feeling of…

That's the market's job. Once AI CEOs start outperforming human CEOs, investment will flow to the winners. Give it 5-10 years.

(Has anyone tried an LLM on an in-basket test? [1] That's a basic test for managers.)

[1] https://en.wikipedia.org/wiki/In-basket_test

Re: Beliefs that are true for regular software but false when applied to AI

#49

Post by prometheus76 »

pcdevils wrote: post #8
xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

For one thing, we'd make shit batteries.

They farm you for attention, not electricity. Attention (engagement time) is how they quantify "quality" so that it can be gamed with an algorithm.

Re: Beliefs that are true for regular software but false when applied to AI

#50

Post by SalientBlue »

pavel_lishin wrote: post #26
SalientBlue wrote: post #9

Earlier quoted context omitted.

You should read the footnote marked [1] after "a note for technical folk" at the beginning of the article. He is very consciously making sweeping generalizations about how software works in order to make things intelligible to non-technical readers.

But are those sweeping generalizations true? > I’m also going to be making some sweeping statements about “how software works”, these claims mostly hold, but they break down when applied to distributed systems, parallel code, or complex interactions between software systems and human processes. I'd argue that this describes most software written since, uh, I hesitate to even commit to a decade here.

For the purposes of the article, which is to demonstrate how developing an LLM is completely different from developing traditional software, I'd say they are true enough. It's a CS 101 understanding of the software development lifecycle, which for non-technical readers is enough to get the point across. An accurate depiction of software development would only obscure the actual point for the lay reader.
Post reply on HN