Scientists Increasingly Can’t Explain How AI Works • Hacker News Board

Re: Scientists Increasingly Can’t Explain How AI Works

#151

Post by omginternets » Wed, Nov 02, 2022, 4:15 PM UTC

science4sail wrote: post #99

omginternets wrote: post #53

Earlier quoted context omitted.

There's the obvious problem of knowing if and when it might suddenly start failing empirically.

Isn't that also a problem with humans? A human pilot might suddenly go insane and start murdering people (even though empirically, it's an improbable event). https://en.wikipedia.org/wiki/Germanwings_Flight_9525

If your point is just that machines can be more reliable than humans, that's not a terribly interesting argument. Humans are often held accountable for the behavior of the machines they oversee. This is no exception.

Re: Scientists Increasingly Can’t Explain How AI Works

#152

Post by loyalcinnamon » Wed, Nov 02, 2022, 4:15 PM UTC

dinosaurdynasty wrote: post #121

kriro wrote: post #40

I don't see a problem with not being able to explain how something works as long as it's not failing empirically. Scientists also cannot explain how humans work in detail and yet humans are "allowed" to do many things and make many decisions that cannot be fully explained. The question for me is...if I have an AI system that outperforms humans empirically, why do I need to understand how it works to use it? In fact i…

One of the problems of highly capable AI ("outperforms humans empirically" in enough tasks) versus most physical systems is that it can plan and it can lie. One situation AI safety researches worry about is "AI wants X; human wants Y; AI knows human wants Y; AI does Y in training/experiments; once AI is in place to do X instead without human being able to stop AI AI does X"

Hmm, well, instead of an ai we put a human, they could do that exact thing too. So what's stoping a human? Punishment?

Does this mean we need to make the ai understand punishment and/or consequences?

Re: Scientists Increasingly Can’t Explain How AI Works

#153

Post by ripe » Wed, Nov 02, 2022, 4:21 PM UTC

joxel wrote: post #13

It's quite easy to explain. You take a bunch of tensors and multiply them by a bunch of tensors. Humans can't wrap their heads around multiple tensors being multiplied together and never will. It's not a problem with AI. It's a problem with humans. It's not AIs fault that we can understand F=ma but can't understand 50 tensors being stacked.

> tensors multiplied by tensors

This argument is ignoring the problem.

What a neural network cannot do is to explain which invisible "rules" it has learned.

For example, when trying to classify movie reviews into good and bad [1], two invisible rules that it learned to apply were:

1. The word "horrible" indicates a bad review.

2. The term "Daniel Day Lewis" indicates a good review.

A human would judge the first rule to be reasonable, the second rule to be a mistake.

[1] https://www.science.org/content/article/how-ai-detectives-ar...

Re: Scientists Increasingly Can’t Explain How AI Works

#154

Post by scarecrowbob » Wed, Nov 02, 2022, 4:21 PM UTC

magic_hamster wrote: post #138

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

I must be in the minority but I really dislike voice commands. Every time I try it I feel stupid talking to appliances. But even if I didn't mind talking to lightbulbs and thermostats it would still bother me for several reasons: I don't like being listened to by these corporations, I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch.

I don't prefer it either, and I am glad you're not in a position to understand why it is better than a switch.

However, my dad is pretty far into Parkinsons and doesn't have a lot of mobility. It's nice that he can turn off the lights when he goes to bed, or turn on the radio from his chair.

I hope to never be in a position where these tools are big upgrades for me, but my assumption is that we all will if we live long enough.

Re: Scientists Increasingly Can’t Explain How AI Works

#155

Post by fumeux_fume » Wed, Nov 02, 2022, 4:30 PM UTC

mdp2021 wrote: post #87

Now for a small elephant which entered the room: the article recites that it would be > worth mentioning that the white-box / black-box terminology is in itself part of a long history of racially coded terms in science; researchers have pushed to change "blacklist" to "blocklist," for example Very plainly, the idea of "black-box" comes from the clear, basic and original, notion and experience that "in the dark, you c…

W/r/t to the term "black-box" being racially coded, I regard that as cheap virtue signalling and a little insulting to the people it's meant to "protect". It's not a hill I'm going to die on, but I don't view that behavior as part of an actual solution towards an equitable society. I've noticed that in my line of work, "black-box" often gets used to describe a process that isn't understood thoroughly by the end-users or stakeholders. I once heard a VP complain that an ARIMA model was a black-box. Had a good chuckle at that.

Re: Scientists Increasingly Can’t Explain How AI Works

#156

Post by 6d6b73 » Wed, Nov 02, 2022, 4:33 PM UTC

In other words - ai doesnt work and nobody knows why

Re: Scientists Increasingly Can’t Explain How AI Works

#157

Post by [deleted] » Wed, Nov 02, 2022, 4:35 PM UTC

magic_hamster wrote: post #138

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

I must be in the minority but I really dislike voice commands. Every time I try it I feel stupid talking to appliances. But even if I didn't mind talking to lightbulbs and thermostats it would still bother me for several reasons: I don't like being listened to by these corporations, I don't like exposing my electronics to the internet, and I still don't see how any of this stuff is better than flicking a switch.

[deleted]

Re: Scientists Increasingly Can’t Explain How AI Works

#158

Post by lastofthemojito » Wed, Nov 02, 2022, 4:36 PM UTC

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

At one point I worked alongside some folks doing machine translation work and it was interesting seeing the older rules-based translation versus the new hotness neural translation. Often the neural models did better overall, but the inscrutability of those models meant sometimes people had no idea why weird output occurred and fixing such cases wasn't as simple as slapping on one more if-statement. It felt like a lower-stakes version of Tesla self-driving - they train it more and more, and some people report improvements, and other folks report regressions and it's not like you can go in and edit the if(carStoppedAhead) statement.

Re: Scientists Increasingly Can’t Explain How AI Works

#159

Post by JacobiX » Wed, Nov 02, 2022, 4:40 PM UTC

visarga wrote: post #143

saiya-jin wrote: post #125

Earlier quoted context omitted.

OP just described ridiculously bad behavior that consistently changes over time, not randomly random results. That's more a case of bad engineering (ie light sensor interfering with microphone data feed) or some stupid management decision then 'AI is complicated but we don't have anything better'

I think it was management that chose to go with a cheaper model, not the engineers. They got to think about profits. You do realise that a neural net, given the same input, will always give the same output, if you set your random seeds of course. It doesn't depend on time of day unless you degrade the service when it is overloaded - say, you use a different model in the evening that is 10x faster but 2x worse.

Not sure about the cheaper model part, because one cannot transform trivially a big SOTA langage modeling models into a digital assistant. So maybe they are not using a cheap ML model after all ... Also, for a digital assistant the input is not only the spoken commands, it can also be: time of the day, weather, location, etc

Re: Scientists Increasingly Can’t Explain How AI Works

#160

Post by Ragnarork » Wed, Nov 02, 2022, 4:40 PM UTC

> Scientists do not attempt to decipher the “black box,” or the opaque processes that the system undertakes, as long as they receive the outputs they are looking for.

This seems a bit disingenuous as there's a whole research field (explainability) that does exactly that. This is not the widest of the whole machine learning domain, but it's there and there's significant work around it...