Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

21–30 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#21
post #13

It's quite easy to explain. You take a bunch of tensors and multiply them by a bunch of tensors. Humans can't wrap their heads around multiple tensors being multiplied together and never will. It's not a problem with AI. It's a problem with humans. It's not AIs fault that we can understand F=ma but can't understand 50 tensors being stacked.

>It's not a problem with AI. It's a problem with humans. It's not AIs fault

This is a really, really funny defense of AI.

Re: Scientists Increasingly Can’t Explain How AI Works

#22
post #4

To me the problem of AI is not how it works but how reliable it works. Do we have a way to test AI to prevent corner cases which could lead to catastrophic results?

What you're saying is (kinda) my response when people ask me about self-driving cars: How does the car deal with object X on the road? For X=piano, a stack of solar panels, a tank, an airplane, a pile of stones ... No one knows and since the size of the set of X is infinite, no one can appropriately train for it either. That's why we can't have self-driving cars without a general understanding of what objects are and…

Yes and no. Yes, because I agree with what you said. No, because it assumes self-driving has to rely purely on normal vision/cameras, which is just garbage². Think LiDAR and other such means to detect obstacles, but things like that are (currently) pretty expensive, so there is the attempt to just do it with (more or less) normal cameras, which leads to what you said (and I fully agree with that). Aside of the fact that safety assumptions+validations towards cameras are complicated in itself already, not only the technology in all its aspects, but also dirty lenses and things like that.

Re: Scientists Increasingly Can’t Explain How AI Works

#23

The history of scientific development is one of finding patterns in data (think Kepler's studies of Tycho Brahe's accurate astronomical observations) - yes, AI excels at that - but then expressing those patterns in terms of simple mathematical equations: orbits are elliptical to a first approximation (not counting other graviational bodies than the sun and a planet), and the sun->planet vector sweeps out equal areas…

> yes, AI excels at that

Sorry for being pedantic about the terminology, but AI has been "deterministic", using direct algorithms as opposed to oracular machines, since those times in which the perceptron was relatively weak.

What we are talking about here is very probably Artificial Neural Networks.

Re: Scientists Increasingly Can’t Explain How AI Works

#25
Explainability is not a given in many more traditional complex systems. Decisions are often an aggregation of a large number of signals, and one can often not conceive of a single intuitive explanation for the system's decisions.

A lot is expected of AI systems today, from fairness (how do we even define that?) to universality. In my view we need to develop a practical understanding of what it means to build the system we have in mind: do I understand where I want my system to perform, and do I have the tools to assess whether I am getting there? Interpretability is orthogonal to all of this.

I would much rather have a well tested system, accompanied by online monitoring to detect unusual inputs in an ever-changing data distribution and notify when updates are needed or a human needs to take control, than an unreliable system that is great at providing explanations.

Re: Scientists Increasingly Can’t Explain How AI Works

#26
post #17

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

> anything new It is nothing new. It is called (at least) "the problem of transparency". The chess context is probably not the best, because many systems allow a lengthy complex explanation of the response (I cannot remember now the exact workings of Deep Blue - it has been a while last time I met the full info). It is a real problem in general, because we may not just want responses but we may want " to learn someth…

In 1963 Marvin Minsky called it “The Credit Assignment Problem” — which, among a multitude of variables, were most important in solving an AI task?

Re: Scientists Increasingly Can’t Explain How AI Works

#28
> Black box models can be extremely powerful, which is how many scientists and companies justify sacrificing explainability for accuracy.

> AI systems have been used for autonomous cars, customer service chatbots, and diagnosing disease, and have the power to perform some tasks better than humans can. For example, a machine that is capable of remembering one trillion items, such as digits, letters, and words, versus humans, who on average remember seven in their short-term memory would be able to process and compute information at a much faster and improved rate than humans.

I suspect this article was written by AI. Or a hack journalist. And most of you haven't read it and are reacting to the title. Certainly won't find any explanations of how AI works on vice.

Re: Scientists Increasingly Can’t Explain How AI Works

#29
post #17

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

> anything new It is nothing new. It is called (at least) "the problem of transparency". The chess context is probably not the best, because many systems allow a lengthy complex explanation of the response (I cannot remember now the exact workings of Deep Blue - it has been a while last time I met the full info). It is a real problem in general, because we may not just want responses but we may want " to learn someth…

I suppose this is why it is called "machine learning" and not "human learning"!

Re: Scientists Increasingly Can’t Explain How AI Works

#30
post #24

Only a fourth paragraph before the lede is revealed: racial and gender biases!

I cannot imagine your reaction when you will arrive at the part where a pressure will be mentioned to «change "blacklist" to "blocklist"».

Well, Andrew Tanenbaum remembered when at IBM he received a full explanation of why they felt very important his shirt should not just be of some specific colour, but of the specific shade of some colour. I would not say it is not part of the job: I would say it ["we feel it very important"] is part of "what happens".

Post reply on HN