Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

81–90 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#81

Post by fidotron »

hn_acc1 wrote: post #61
pixl97 wrote: post #52

Earlier quoted context omitted.

>that baffles me about the managerial globalists It's no different from those on HN that yell loudly that unions for programmers are the worst idea ever... "it will never be me" is all they can think, then they are protesting in the streets when it is them, but only after the hypocrisy of mocking those in the street protesting today.

Agreed. My dad was raised strongly fundamentalist, and in North America, that included (back then) strongly resisting unions. In hindsight, I've come to realize that my parent's weren't maybe even of average intelligence, and definitely of above-average gullibility. Unionized software engineers would solve a lot of the "we always work 80 hour weeks for 2 months at the end of a release cycle" problems, the "you're too…

Software "engineers" totally miss a key thing in other engineering professions as well, which is organizations to enforce some pretense of ethical standards to help push back against requests from product. Those orgs often look a lot like unions.

Re: Beliefs that are true for regular software but false when applied to AI

#82

Post by [deleted] »

conception wrote: post #68
toomuchtodo wrote: post #19

Earlier quoted context omitted.

Labor competes for compensation, CEOs compete for status (above a certain enterprise size, admittedly). Show me a CEO willingly stepping down to be replaced by generative AI. Jamie Dimon will be so bold to say AI will bring about a 3 day week (because it grabs headlines [1]) but he isn't going to give up the status of running JPMC; it's all he has besides the wealth, which does not appear to be enough. The feeling of…

Spoiler there’s no reason we couldn’t work three days a week now. And 100 might be pushing it, but having life expectancy to 90 as well within our grass today as well. We have just decided not to do that.

[deleted]

Re: Beliefs that are true for regular software but false when applied to AI

#83

Post by SalientBlue »

alganet wrote: post #54
SalientBlue wrote: post #42

Earlier quoted context omitted.

I don't think he's doing that at all. The article is pointing out to non-technical people how AI is different than traditional software. I'm not sure how you think it's giving AI a break, as it's pointing out that it is essentially impossible to reason about. And it's not at the expense of regular developers because it's showing how regular software development is different than this. It makes two buckets, and puts A…

He is. Maybe he's just running with the pack, but that doesn't matter either. The fact is, we kind of know how to prevent problems in AI systems: - Good benchmarks. People said several times that LLMs display erratic behavior that could be prevented. Instead of adjusting the benchmarks (which would slow down development), they ignored the issues. - Accountability frameworks. Who is responsible when an AI fails? How t…

>But it's a different we already know

'we' is the operative word here. 'We', meaning technical people who have followed this stuff for years. The target audience of this article are not part of this 'we' and this stuff IS completely new _for them_. The target audience are people who, when confronted with a problem with an LLM, think it is perfectly reasonable to just tell someone to 'look at the code' and 'fix the bug'. You are not the target audience and you are arguing something entirely different.

Re: Beliefs that are true for regular software but false when applied to AI

#84

Post by vladms »

coliveira wrote: post #38
drsupergud wrote: post #11

> bugs are usually caused by problems in the data used to train an AI This also is a misunderstanding. The LLM can be fine, the training and data can be fine, but because the LLMs we use are non-deterministic (at least in regard to their being intentional attempts at entropy to avoid always failing certain scenarios) current algorithms are inherently by-design not going to always answer every question correctly that…

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…

> Any programming or mathematical question has several correct answers.

Huh? If I need to sort the list of integer number of 3,1,2 in ascending order the only correct answer is 1,2,3. And there are multiple programming and mathematical questions with only one correct answer.

If you want to say "some programming and mathematical questions have several correct answers" that might hold.

Re: Beliefs that are true for regular software but false when applied to AI

#85

Post by malfist »

noir_lord wrote: post #75
kstrauser wrote: post #65

Earlier quoted context omitted.

Holy survivorship bias, Batman. If you think modern software is unreliable, let me introduce you to our friend, Rational Rose.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

Re: Beliefs that are true for regular software but false when applied to AI

#86

Post by redblacktree »

vladms wrote: post #84
coliveira wrote: post #38

Earlier quoted context omitted.

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…

> Any programming or mathematical question has several correct answers. Huh? If I need to sort the list of integer number of 3,1,2 in ascending order the only correct answer is 1,2,3. And there are multiple programming and mathematical questions with only one correct answer. If you want to say "some programming and mathematical questions have several correct answers" that might hold.

What about multiple notational variations?

1, 2, 3

1,2,3

[1,2,3]

1 2 3

etc.

Re: Beliefs that are true for regular software but false when applied to AI

#87

Post by alecbz »

surgical_fire wrote: post #80
xutopia wrote: post #4

The most likely danger with AI is concentrated power, not that sentient AI will develop a dislike for us and use us as "batteries" like in the Matrix.

"AI will take over the world". I hear that. Then I try to use AI for simple code task, writing unit tests for a class, very similar to other unit tests. If fails miserably. Forgets to add an annotation and enters in a death loop of bullshit code generation. Generates test classes that tests failed test classes that test failed test classes and so on. Fascinating to watch. I wonder how much CO2 it generated while fryi…

Regularly trying to use LLMs to debug coding issues has convinced me that we're _nowhere_ close to the kind of AGI some are imagining is right around the corner.

Re: Beliefs that are true for regular software but false when applied to AI

#88

Post by james_marks »

joomla199 wrote: post #23
kazinator wrote: post #2

> AIs will get more reliable over time, like old software is more reliable than new software. :) Was that a humam Freudian slip, or artificial one? Yes, old software is often more reliable than new.

Neither, you’re reading it wrong. Think of it as codebases getting more reliable over time as they accumulate fixes and tests. (As opposed to, say, writing code in NodeJS versus C++)

I’ve always called this “Work Hardening”, as in, the software has been improved over time by real work being done with it.

Re: Beliefs that are true for regular software but false when applied to AI

#89

Post by kazinator »

malfist wrote: post #85
noir_lord wrote: post #75

Earlier quoted context omitted.

Agreed. Or debuggers that would take out the entire OS. Or a bad driver crashing everything multiple times a week. Or a misbehaving process not handing control back to the OS. I grew up in the era of 8 and 16 bit micros and early PCs, they where hilariously less stable than modern machines while doing far less, there wasn’t some halcyon age of near perfect software, it’s always been a case of things been good enough…

Remember BSODs? Used to be a regular occurrence, now they're so infrequent they're gone from windows 11

I remember Linux being remarkable reliable throughout its entire life in spite of being rabidly worked on.

Windows is only stabilizing because it's basically dead. All the activity is in the higher layers, where they are racking their brains on how to enshittify the experience, and extract value out of the remaining users.

Re: Beliefs that are true for regular software but false when applied to AI

#90

Post by Nasrudith »

CollinEMac wrote: post #59

> It’s entirely possible that some dangerous capability is hidden in ChatGPT, but nobody’s figured out the right prompt just yet. This sounds a little dramatic. The capabilities of ChatGPT are known. It generates text and images. The qualities of the content of the generated text and images is not fully known.

Plus there is the 'monkeys with typewriters' problem with both danger and hypothetical good. In contrast, ChatGPT may technically reply to the right prompt with a universal cancer cure/vaccine. Psuedorandomly generating it wouldn't help as you wouldn't recognize it from all of the other queries of things we don't know of as true or false.

Likewise what to ask it for how to make some sort of horrific toxic chemical, nuclear bomb, or similar isn't much good if you cannot recognize it and dangerous capability depends heavily on what you have available to you. Any idiot can be dangerous with C4 and detonator or bleach and ammonia. Even if ChatGPT could give entirely accurate instructions on how to build an atomic bomb it wouldn't do much good because you wouldn't be able to source the tools and materials without setting off red flags.

Post reply on HN