Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

Post reply on HN
Print view View original

111–120 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#111

Post by visarga »

HPsquared wrote: post #110
FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

The explanations used to explain or justify human behaviour are just as speculative though, really. We can try and hypothesize, but that's it - there is no engineering design specification for the brain, nor even a debugger.

We practically invent our explanations post facto, first decide, then verbalise.

Re: Scientists Increasingly Can’t Explain How AI Works

#112

Post by zmgsabst »

diputsmonro wrote: post #75
fallingknife wrote: post #61

Earlier quoted context omitted.

What people want is for the models to show no differences between certain groups of people. That is the standard they use for "no bias." But that itself is bias, so really they just want the model output to conform to their bias that there isn't any difference.

So what you're really saying is, it's perfectly okay for AI to have negative preconceptions and concrete impacts to certain groups of people, as long as it agrees with you on which groups are bad. How would you feel about AI models that classify all modern white people as racists based on overwhelming historical data about slavery, the KKK, etc.? Hey man, data doesn't lie.

Isn’t the media doing exactly that?

They fixate on white racists while ignoring (eg) the Christmas parade attack — leading to society developing a bias in their beliefs about racism.

I always take these criticism of AI as confession-through-projection of the misdeeds people have engaged in already.

Re: Scientists Increasingly Can’t Explain How AI Works

#113

Post by karpierz »

tromp wrote: post #101
karpierz wrote: post #88

Earlier quoted context omitted.

That's punting the problem of explaining down the line. All the PV tells you is the computer is making this move because it thinks that 20 moves down the line, you'll end up in some desirable position. It doesn't abstract or summarize any understanding. If you say, "well, what if they do Kf3 instead", all it can show is another PV. It can't tell you "knights on the rim are dim", or "this pawn needs to move now to pre…

It doesn't provide a clear explanation in those terms, and in some fraction of positions, there is no simple such explanation. But I believe that in a decent fraction of cases, a human chess player probing PVs will gain a reasonable understanding of the winning or drawing strategy involved, and that such interactive PV probing is often a better form of explanation than a concise non-interactive textual one could offe…

I agree that a human can use the PV to get insight into why a move might be good. I also agree that non-interactive explanations provided by a chess engine will often be inferior. I think one way to break it down might be that the PV (or sets of PVs) can provide a useful explanation of tactical value (Kf3 leads to mate in 6), but struggles to explain strategic value (Kf3 leads to the positioning opening up, which broadly makes it harder for you to press your advantage).

> There is no equivalent to that in neural networks.

The first example that comes to mind is using backpropagation to determine how much influence each value had on the output result, and then rendering it in a human-readable way. Stuff like this:

https://miro.medium.com/max/4800/0*Y3Yi7cEueF0XLZP-

https://miro.medium.com/max/1100/1*IPhQ12OKnxH31AQNja6FnQ.pn...

Re: Scientists Increasingly Can’t Explain How AI Works

#114

Post by rwmj »

kriro wrote: post #40

I don't see a problem with not being able to explain how something works as long as it's not failing empirically. Scientists also cannot explain how humans work in detail and yet humans are "allowed" to do many things and make many decisions that cannot be fully explained. The question for me is...if I have an AI system that outperforms humans empirically, why do I need to understand how it works to use it? In fact i…

Eventually it'll come up in a court case or similar scenario: Your AI killed a particular patient (never mind that it saved a thousand others). Can you explain to a judge why that patient was killed? Can you ensure that the changes you have made since mean it will not kill another patient? Those are going to be "interesting" questions to have to deal with in a legal or regulatory environment.

Re: Scientists Increasingly Can’t Explain How AI Works

#115

Post by visarga »

randcraw wrote: post #50
FartyMcFarter wrote: post #33

Earlier quoted context omitted.

> And of course, "understanding" the engine is required to advance it. I don't think so. People are clearly making huge advances in machine learning even though they can't explain the systems made 10 years ago, let alone more recent ones. There's a lot of trial and error / cargo-culting involved. Some researchers go as far as comparing machine learning research to Alchemy: https://www.science.org/content/article/ai-r…

But an AI must “understand its reasoning” in order to revise it efficiently to learn from its mistakes. If revision is impossible, the only way to correct its reasoning is to re-train the system from scratch. But in doing that, you can only hope the problem is fixed, and worse, that new problems were not just created.

There is also the problem of label noise. A label might not be necessarily correct, especially when the dataset is large. Nobody got time to check them all, and even if you put two people they might disagree.

Re: Scientists Increasingly Can’t Explain How AI Works

#116

Post by hn_throwaway_99 »

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

> My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception.

I can't comment much on Siri or Alexa, but Google Assistant has gotten heaps better at understanding my voice and intent since it was first released about 6 years ago. It used to pretty reliably misunderstand at least 1 word per sentence (which made for an extremely frustrating experience originally); now, at least for me, it's extremely rare that it misunderstands me.

This would point to the problem being company-specific, and not a fundamental issue with the "black box" aspect of the technology.

Re: Scientists Increasingly Can’t Explain How AI Works

#117

Post by joxel »

aliqot wrote: post #106
joxel wrote: post #100

Earlier quoted context omitted.

That is not analogous

Tell me how a GAN works. it's 1000 pedantic monkeys trying to out-HN the other. At the root of all AI is a similar process.

Comparing brute force to backpropogation with stochastic gradient descent shows you don't understand the process, or are being purposefully obtuse.

Re: Scientists Increasingly Can’t Explain How AI Works

#120

Post by daniel-cussen »

belval wrote: post #95
mdp2021 wrote: post #87

Now for a small elephant which entered the room: the article recites that it would be > worth mentioning that the white-box / black-box terminology is in itself part of a long history of racially coded terms in science; researchers have pushed to change "blacklist" to "blocklist," for example Very plainly, the idea of "black-box" comes from the clear, basic and original, notion and experience that "in the dark, you c…

I guess it's the hill I'll die on, but these "white vs black" and "master slave" are inherently racist just make my head turn every time. blacklist came from BEFORE slavery in America. > According to the Henry Holt Encyclopedia of Word and Phrase Origins the word "blacklist" originated with a list England's King Charles II made of fifty-eight judges and court officers who sentenced his father, Charles I, to death in…

No post body was provided.
Post reply on HN