Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

Post reply on HN
Print view View original

101–110 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#101

Post by tromp »

karpierz wrote: post #88
tromp wrote: post #54

Earlier quoted context omitted.

> Obviously we know they were the result of a deep minimax-style parallel search with a certain evaluation function, and we could simulate a similar search by hand if we wanted to. You don't really need to though. The computer can justify its evaluation with a principal variation (best play by both sides) leading to a leaf position of the same value. And if a human were to wonder why at any point in this variation, s…

That's punting the problem of explaining down the line. All the PV tells you is the computer is making this move because it thinks that 20 moves down the line, you'll end up in some desirable position. It doesn't abstract or summarize any understanding. If you say, "well, what if they do Kf3 instead", all it can show is another PV. It can't tell you "knights on the rim are dim", or "this pawn needs to move now to pre…

It doesn't provide a clear explanation in those terms, and in some fraction of positions, there is no simple such explanation. But I believe that in a decent fraction of cases, a human chess player probing PVs will gain a reasonable understanding of the winning or drawing strategy involved, and that such interactive PV probing is often a better form of explanation than a concise non-interactive textual one could offer.

There is no equivalent to that in neural networks.

Note that PVs explain the connection between the root position of minimax search and a leaf positions. Evaluation at the latter is easy to explain in terms of features like material, mobility, pawn structure, king safety, etcetera.

Re: Scientists Increasingly Can’t Explain How AI Works

#103

Post by saladtoes »

mdp2021 wrote: post #92
saladtoes wrote: post #25

Explainability is not a given in many more traditional complex systems. Decisions are often an aggregation of a large number of signals, and one can often not conceive of a single intuitive explanation for the system's decisions. A lot is expected of AI systems today, from fairness (how do we even define that?) to universality. In my view we need to develop a practical understanding of what it means to build the syst…

> I would much rather have False dichotomy. In fact, well understood systems must be more reliable.

Is that really a universal fact? In any case, my statement goes in the opposite direction: is a reliable system necessarily "well understood", in the sense that it can explain its decisions? Most complex systems powering our lives cannot tell us anything about how they made those decisions.

AI systems add a layer of complexity. Even if you can explain a decision well on your training data, I seriously doubt you will be able to still provide reasonable explanations in completely out of distribution data.

Re: Scientists Increasingly Can’t Explain How AI Works

#105

Post by naasking »

impalallama wrote: post #78
fallingknife wrote: post #72

Earlier quoted context omitted.

Assuming that a human stereotype is untrue is just as biased as assuming it is true. If you judge a model by how well it confirms your previous bias, then the model tells you nothing you didn't already know.

Quoted post unavailable.

Setting aside the rudeness, you basically just agreed with the parent. They said people don't want an AI with no bias, we want an AI that mirrors own biases.

Your example of loans is perfectly in line with this: we have a social bias against racism and racial profiling, and we want our AIs to reflect that bias. Purely on the question of risk assessment, race must meaningfully impact a loan risk or the AI wouldn't develop that bias [1], but that isn't a correlation we want the AI to consider in its risk assessment, ie. we build in a different bias.

[1] race in this case would be a proxy for all sorts of complex historical socioeconomic factors

Re: Scientists Increasingly Can’t Explain How AI Works

#106

Post by aliqot »

joxel wrote: post #100
aliqot wrote: post #86

Earlier quoted context omitted.

"How can a thousand monkeys solve what humans can't solve"

That is not analogous

Tell me how a GAN works. it's 1000 pedantic monkeys trying to out-HN the other. At the root of all AI is a similar process.

Re: Scientists Increasingly Can’t Explain How AI Works

#107

Post by visarga »

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

> you can't explain what's going on inside that black box

Do you have a better solution for speech recognition? We all know how well speech recognition works in reality, and we know language models can accomplish more complex tasks than setting your music and lights.

These models are not state of the art, they are cheap versions for scaling up to millions of users. It's sad but we rarely get to see SOTA in a product.

Re: Scientists Increasingly Can’t Explain How AI Works

#108

Post by noobermin »

joenot443 wrote: post #76
recuter wrote: post #28

> Black box models can be extremely powerful, which is how many scientists and companies justify sacrificing explainability for accuracy. > AI systems have been used for autonomous cars, customer service chatbots, and diagnosing disease, and have the power to perform some tasks better than humans can. For example, a machine that is capable of remembering one trillion items, such as digits, letters, and words, versus…

Just a hack journalist, I think. Her other publications include: "Alaska Canceled Snow Crab Season for the First Time Ever Because All the Crabs Are Gone"

what's wrong with that?

Re: Scientists Increasingly Can’t Explain How AI Works

#110

Post by HPsquared »

FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

The explanations used to explain or justify human behaviour are just as speculative though, really. We can try and hypothesize, but that's it - there is no engineering design specification for the brain, nor even a debugger.
Post reply on HN