Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Beliefs that are true for regular software but false when applied to AI

boydkane.com

Post reply on HN
Print view View original

301–310 of 461 posts

Re: Beliefs that are true for regular software but false when applied to AI

#301

Post by karuko24 »

> bugs are usually caused by problems in the data used to train an AI

I think a fundamental problem is that many people assume that an LLM's failure to correctly perform a task is a bug that can be fixed somehow. Often times, the reason for that failure is simply a property of the AI systems we have at the moment.

When you accidentally drop a glass and it breaks, you don't say that it's a bug in gravity. Instead, you accept that it's a part of the system you're working with. The same applies to many categories of failures in AI systems: we can try to reduce them, but unless the nature of the system fundamentally changes (and we don't know if or when that will happen), we won't be able to get rid of them.

"Bug" carries an implication of "fixable" and that doesn't necessarily apply to AI systems.

Re: Beliefs that are true for regular software but false when applied to AI

#302

Post by OskarS »

vladms wrote: post #84
coliveira wrote: post #38

Earlier quoted context omitted.

This is not necessarily a problem. Any programming or mathematical question has several correct answers. The problem with LLMs is that they don't have a process to guarantee that a solution is correct. They will give a solution that seems correct under their heuristic reasoning, but they arrived at that result in a non-logical way. That's why LLMs generate so many bugs in software and in anything related to logical t…

> Any programming or mathematical question has several correct answers. Huh? If I need to sort the list of integer number of 3,1,2 in ascending order the only correct answer is 1,2,3. And there are multiple programming and mathematical questions with only one correct answer. If you want to say "some programming and mathematical questions have several correct answers" that might hold.

No, but if you phrase it like "there are multiple correct answers to the question 'I have a list of integers, write me a computer program that sorts it'", that is obviously true. There's an enormous variety of different computer programs that you can write that sorts a list.

Re: Beliefs that are true for regular software but false when applied to AI

#303

Post by baq »

cwillu wrote: post #283
harrisonjackson wrote: post #218

Earlier quoted context omitted.

The Ring app notification summaries still scare me. > "A bunch of people right outside your house!!!" because it aggregates multiple single person walking by notifications that way...

Unrelated, but am I the only person who finds the concept of “getting notifications for somebody walking by a house” to be really creepy?

it isn't creepy, it's super annoying if you don't live in the woods. got a ring doorbell and turned them off a few hours after installation, it was driving me nuts.

Re: Beliefs that are true for regular software but false when applied to AI

#304

Post by gwd »

karuko24 wrote: post #301

> bugs are usually caused by problems in the data used to train an AI I think a fundamental problem is that many people assume that an LLM's failure to correctly perform a task is a bug that can be fixed somehow. Often times, the reason for that failure is simply a property of the AI systems we have at the moment. When you accidentally drop a glass and it breaks, you don't say that it's a bug in gravity. Instead, you…

But it's worse than that. Even if in theory the system could be fixed, we don't actually know how to fix it for real, the way we can fix a normal computer program.

The reason we can't fix them is because we have no idea how they work; and the reason we have no idea how they work is this:

1. The "normal" computer program, which we do understand, implement a neural network

2. This neural network is essentially a different kind of processor. The "actual" computer program for modern deep learning systems is the weights. That is, weights : neural net :: machine language : normal cpu

3. We don't program these weights; we literally summon them out of the mathematical aether by the magic of back-propagation and gradient descent.

This summoning is possible because the "processor" (the neural network architecture) has been designed to be differentiable: for every node we can calculate the slope of the curve with respect to the result we wanted, so we know "The final output for this particular bit was 0.7, but we wanted it to be 1. If this weight in the middle of the network were just a little bit lower, then that particular output would have been a little bit higher, so we'll bump it down a bit."

And that's fundamentally why we can't verify their properties or "fix" them the way we can fix normal computer programs: Because what we program is the neural network; the real program, which runs on top of that network, is summoned and not written.

Re: Beliefs that are true for regular software but false when applied to AI

#305

Post by Cthulhu_ »

cwillu wrote: post #283
harrisonjackson wrote: post #218

Earlier quoted context omitted.

The Ring app notification summaries still scare me. > "A bunch of people right outside your house!!!" because it aggregates multiple single person walking by notifications that way...

Unrelated, but am I the only person who finds the concept of “getting notifications for somebody walking by a house” to be really creepy?

Well yeah, but that's in part a problem with always-on doorbell cameras. On paper they're illegal in many countries (privacy laws, you can't just put up a camera and record anyone out in public), in practice the police asks people to put their doorbell cameras in a registry so they can request footage if needs be.

Anyway, I get wanting to see who's ringing your doorbell in e.g. apartment buildings, and that extending to a house, especially if you have a bigger one. But is there a reason those cameras need to be on all the time?

Re: Beliefs that are true for regular software but false when applied to AI

#306

Post by dingdingdang »

rldjbpin wrote: post #286
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

> to perform up to Apple's typical standards of polish and control. i no longer believe they have kept on to the standards in general. the ux/ui used to be a top priority, but the quality control has certainly gone down over the years [1]. the company is now driven by supply chain and business-minded optimizations than what to give to the end user. at the same time, what one can do using AI has large correlation with…

>> to perform up to Apple's typical standards of polish and control.

>i no longer believe they have kept on to the standards in general.

One 100% agree with this, if I compare AI's ability to speed up the baseline for me in terms of programming Golang (hard/tricky tasks clearly still require human input - watch out for I/O ops) with Apple's lack of ability to integrate it in even the simplest of ways.. things are just peculiar on the Apple front. Bit similar to how MS seems to be gradually loosing the ability to produce a version of Windows that people want to run due to organisational infighting.

Re: Beliefs that are true for regular software but false when applied to AI

#307

Post by highfrequency »

Fortunately, we can have LLMs write code and keep all the benefits of normal software (determinism, reproducibility, permanent bug fixes etc.)

I don’t think anyone is advocating for web apps to take the form of an LLM prompt with the app getting created on the fly every time someone goes to the url.

Re: Beliefs that are true for regular software but false when applied to AI

#308

Post by bombcar »

mikkupikku wrote: post #299
remexre wrote: post #205

Earlier quoted context omitted.

It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...

Maybe they remembered that a lot of people aren't actually good writers. My brother will send 1000 word emails that meander through subjects like what he ate for breakfast to eventually get to the point of scheduling a meeting about negotiating a time for help with moving a sofa. Mind you, I see him several times a week so he's not lonely, this is just the way he writes. Then he complains endlessly about his coworker…

There’s a thread here that could be pulled - something about using AI to turn everyone into exactly who you want to communicate with in the way you want.

Probably a sci-fi story about it, if not, it should be written.

Re: Beliefs that are true for regular software but false when applied to AI

#309

Post by bombcar »

ludicrousdispla wrote: post #262
remexre wrote: post #205

Earlier quoted context omitted.

It does feel like somebody forgot that "from the first sentence or two of the email, you can tell what it's about" was already a rule of good writing...

I think people read texts because they want to read them, and when they don't want to read the texts they are also not even interested in reading the summaries. Why do I think this? ...in the early 2000's my employer had a company wide license for a document summarizer tool that was rather accurate and easy to use, but nobody ever used it.

The obvious use case is “I don’t want to read this but I am required to read this (job)” - the fact that people don’t want to use it even there is telling, imo.

Re: Beliefs that are true for regular software but false when applied to AI

#310

Post by bombcar »

arethuza wrote: post #279
freetime2 wrote: post #122

For a real world example of the challenges of harnessing LLMs, look at Apple. Over a year ago they had a big product launch focused on "Apple Intelligence" that was supposed to make heavy use of LLMs for agentic workflows. But all we've really gotten since then are a couple of minor tools for making emojis, summarizing notifications, and proof reading. And they even had to roll back the notification summaries for a w…

My wife was in China recently and was sending back pictures of interesting things - one came in while I was driving and my iPhone read out a description of the picture that had been sent - "How cool is that!" I thought. However, when I stopped driving and looked at the picture the AI generated description was pretty poor - it wasn't completely wrong but it really wasn't what I was expecting given the description.

It’s been surprisingly accurate at times “a child holding an apple” in a crowded picture, and then sometimes somewhat wrong.

What really kills me is “a screenshot of a social media post” come on it’s simple OCR read the damn post to me you stupid robot! Don’t tell me you can’t, OCR was good enough in the 90s!

Post reply on HN