Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

31–40 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#31

Post by 1313ed01 »

giancarlostoro wrote: post #25
joomla199 wrote: post #23

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

Old code that has been maintained (bugfixed), but not messed with too much (i.e. major rewrites or new features) is almost certain to be better than most other code though?

Re: Beliefs that are true for regular software but false when applied to AI

#32

Post by tptacek »

It would help if this piece was clearer about the context in which "AI bugs" reveal themselves. As an argument for why you shouldn't have LLMs making unsupervised real-time critical decisions, these points are all well taken. AI shouldn't be controlling the traffic lights in your town. We may never reach a point where it can. But among technologists, the major front on which these kinds of bugs are discussed is coding agents, and almost none of these points apply directly to coding agents: agent coding is (or should be) a supervised process.

Re: Beliefs that are true for regular software but false when applied to AI

#33

Post by alganet »

SalientBlue wrote: post #9
alganet wrote: post #5

> here are some example ideas that are perfectly true when applied to regular software Hm, I'm listening, let's see. > Software vulnerabilities are caused by mistakes in the code That's not exactly true. In regular software, the code can be fine and you can still end up with vulnerabilities. The platform in which the code is deployed could be vulnerable, or the way it is installed make it vulnerable, and so on. > Bug…

You should read the footnote marked [1] after "a note for technical folk" at the beginning of the article. He is very consciously making sweeping generalizations about how software works in order to make things intelligible to non-technical readers.

Does that really matter?

He is trying to lax the general public perception around AIs shortcomings. He's giving AI a break, at the expense of regular developers.

This is wrong on two fronts:

First, because many people foresaw the AI shortcomings and warned about them. This "we can't fix a bug like in regular software" theatre hides the fact that we can design better benchmarks, or accountability frameworks. Again, lots of people foresaw this, and they were ignored.

Second, because it puts the strain on non-AI developers. It blamishes all the industry, putting together AI with non-AI in the same bucket, as if AI companies stumbled on this new thing and were not prepared for its problems, when the reality is that many people were anxious about the AI companies practices not being up to standard.

I think it's a disgraceful take, that only serves to sweep things under a carpet.

Re: Beliefs that are true for regular software but false when applied to AI

#34

Post by wsc981 »

izzydata wrote: post #27
joomla199 wrote: post #23

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Sounds more like survivorship bias. All the bad codebases were thrown out and only the good ones lasted a long time.

Basically the Lindy Effect: https://en.wikipedia.org/wiki/Lindy_effect

Re: Beliefs that are true for regular software but false when applied to AI

#35

Post by prasadjoglekar »

giancarlostoro wrote: post #25
joomla199 wrote: post #23

Earlier quoted context omitted.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

It actually might. Older code running in production is almost automatically regression tested with each new fix. It might not be pretty, but it's definitely more reliable for solving real problems.

Re: Beliefs that are true for regular software but false when applied to AI

#36

Post by wrs »

My current method for trying to break through this misconception is informing people that nobody knows how AI works. Literally. Nobody knows. (Note that knowing how to make something is not the same as knowing how it works. Take humans as an obvious example.)

Re: Beliefs that are true for regular software but false when applied to AI

#37

Post by eptcyka »

1313ed01 wrote: post #31
giancarlostoro wrote: post #25

Earlier quoted context omitted.

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

Old code that has been maintained (bugfixed), but not messed with too much (i.e. major rewrites or new features) is almost certain to be better than most other code though?

I’ve read parts of macOS’ open source code that surely has been around for a while, maintained and absolute rubbish.

Re: Beliefs that are true for regular software but false when applied to AI

#38

Post by coliveira »

drsupergud wrote: post #11

> bugs are usually caused by problems in the data used to train an AI This also is a misunderstanding. The LLM can be fine, the training and data can be fine, but because the LLMs we use are non-deterministic (at least in regard to their being intentional attempts at entropy to avoid always failing certain scenarios) current algorithms are inherently by-design not going to always answer every question correctly that…

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical thinking.

Re: Beliefs that are true for regular software but false when applied to AI

#39

Post by worldsayshi »

xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

> power resides where men believe it resides

And also where people believe that others believe it resides. Etc...

If we can find new ways to collectively renegotiate where we think power should reside we can break the cycle.

But we only have time to do this until people aren't a significant power factor anymore. But that's still quite some time away.

Re: Beliefs that are true for regular software but false when applied to AI

#40

Post by DSMan195276 »

1313ed01 wrote: post #31
giancarlostoro wrote: post #25

Earlier quoted context omitted.

Age of Code does not automatically equal quality of code, ever. Good code is maintained by good developers. A lot of bad code is pushed out by management, and other situations, or just bad devs. This is a can of worms you're talking your way into.

Old code that has been maintained (bugfixed), but not messed with too much (i.e. major rewrites or new features) is almost certain to be better than most other code though?

"Bugfixes" doesn't mean the code actually got better, it just means someone attempted to fix a bug. I've seen plenty of people make code worse and more buggy by trying to fix a bug, and also plenty of old "maintained" code that still has tons of bugs because it started from the wrong foundation and everyone kept bolting on fixes around the bad part.
Post reply on HN