Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

Post reply on HN
Print view View original

51–60 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#51

Post by mdp2021 »

bobosha wrote: post #12

I think explainability is overrated (to use a Trumpian expression) & this fixation with coming up with explanations for inferences is a red herring. We cannot explain our own thoughts and actions and tend to ascribe logic & reason to many of our own actions, but it's almost always system-1 driven, for the most part. Correcting for edge-cases and unknown-unknowns is where we should focus our efforts methinks.

A very critical core human activity is that of reflection.

You should explain the point about «correcting for edge-cases and unknown-unknowns», which may not be clear.

Re: Scientists Increasingly Can’t Explain How AI Works

#52

Post by mdp2021 »

randcraw wrote: post #50
FartyMcFarter wrote: post #33

Earlier quoted context omitted.

> And of course, "understanding" the engine is required to advance it. I don't think so. People are clearly making huge advances in machine learning even though they can't explain the systems made 10 years ago, let alone more recent ones. There's a lot of trial and error / cargo-culting involved. Some researchers go as far as comparing machine learning research to Alchemy: https://www.science.org/content/article/ai-r…

But an AI must “understand its reasoning” in order to revise it efficiently to learn from its mistakes. If revision is impossible, the only way to correct its reasoning is to re-train the system from scratch. But in doing that, you can only hope the problem is fixed, and worse, that new problems were not just created.

It is very rare, but I must congratulate.

An aim is "non-deterministic virtual self-modifying code".

Re: Scientists Increasingly Can’t Explain How AI Works

#53

Post by omginternets »

kriro wrote: post #40

I don't see a problem with not being able to explain how something works as long as it's not failing empirically. Scientists also cannot explain how humans work in detail and yet humans are "allowed" to do many things and make many decisions that cannot be fully explained. The question for me is...if I have an AI system that outperforms humans empirically, why do I need to understand how it works to use it? In fact i…

There's the obvious problem of knowing if and when it might suddenly start failing empirically.

Re: Scientists Increasingly Can’t Explain How AI Works

#54

Post by tromp »

FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

> Obviously we know they were the result of a deep minimax-style parallel search with a certain evaluation function, and we could simulate a similar search by hand if we wanted to.

You don't really need to though. The computer can justify its evaluation with a principal variation (best play by both sides) leading to a leaf position of the same value. And if a human were to wonder why at any point in this variation, some other move is inferior, the computer can again easily show a principal variation branching off from that other move, leading to a worse (or equal) result for the deviating player. In this way, minimax results are FAR better justifiable than neural net results.

Re: Scientists Increasingly Can’t Explain How AI Works

#55

Post by fallingknife »

p0pcult wrote: post #39
fallingknife wrote: post #10

Earlier quoted context omitted.

What people are really saying, but never will admit, is that they want AI to mirror their own biases, not to have none. That's the reason they are "cleaning" the input data.

I think with this response, you reveal more about yourself and your worldview, than any general truth about others.

And with yours you reveal that you have no point at all

Re: Scientists Increasingly Can’t Explain How AI Works

#56

Post by [deleted] »

FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

[deleted]

Re: Scientists Increasingly Can’t Explain How AI Works

#57

Post by mchaver »

fallingknife wrote: post #10
Xcelerate wrote: post #5

There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works. I think that assumption is totally wrong, and trying to reconcile the two is probably a distraction and a waste of time. Evolution led to human intelligence just fine on i…

What people are really saying, but never will admit, is that they want AI to mirror their own biases, not to have none. That's the reason they are "cleaning" the input data.

I always thought it was the opposite. That we want AI to have no biases, but we have trouble avoiding it. Choosing data and cleaning is where we insert our bias.

Re: Scientists Increasingly Can’t Explain How AI Works

#58

Post by mdp2021 »

joxel wrote: post #13

It's quite easy to explain. You take a bunch of tensors and multiply them by a bunch of tensors. Humans can't wrap their heads around multiple tensors being multiplied together and never will. It's not a problem with AI. It's a problem with humans. It's not AIs fault that we can understand F=ma but can't understand 50 tensors being stacked.

> You take a bunch of tensors

No, radically not. That is not the kind of understanding we seek.

The advance in knowledge is given by, e.g., "simulated annealing returned a blueprint for very odd circuit schematic with No-Op loops: we were puzzled but then understood they were there to correct timing".

And this is not the level of explanation where tensors are.

Re: Scientists Increasingly Can’t Explain How AI Works

#59

Post by bombcar »

tromp wrote: post #54
FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

> Obviously we know they were the result of a deep minimax-style parallel search with a certain evaluation function, and we could simulate a similar search by hand if we wanted to. You don't really need to though. The computer can justify its evaluation with a principal variation (best play by both sides) leading to a leaf position of the same value. And if a human were to wonder why at any point in this variation, s…

Exactly, the Deep Blue's move explanations are boring - they all boil down to "based on the inputs and rules programmed, this line of moves has the best overall outcome to a depth of X" where X is however deep it goes.

You can try to translate that to human methods of understanding, but that's not how the computer "thinks", and attempting to do that translation leads to misunderstanding. Kasparov may make moves because he wants to get a better board position or knows his opponent is weak to certain positions (I have no idea how experts explain moves) but the computer isn't programmed that way.

Re: Scientists Increasingly Can’t Explain How AI Works

#60

Post by williamcotton »

kortex wrote: post #49
williamcotton wrote: post #31

Earlier quoted context omitted.

LiDAR can't tell you mass of an object as it is headed towards your vehicle. Imagine you're driving down the road and you see: 1.) A kid throw a rubber playground ball into the road and at your windshield. 2.) A kid heave a bowling ball into the road and at your windshield. How would you react in each case? How do you distinguish between the two? As a kid is heaving the bowling ball you can tell that it has a lot mor…

Same for both: brake. I don't want either hitting my windshield. The kickball may be bouncy, sure, but it may have gotten rocks stuck to it, which can create a point stress and scratch paint or chip the windshield. Humans can't intuit mass unless the object meets certain criteria, either. What if it's an opaque cardboard box in the road? Generally the strategy is the same: avoid it.

In certain circumstances with a playground ball you would have to choose between extensive damage to the body of your car by swerving out of the way or to just take the risk of a chipped windshield from a playground ball that somehow has rocks stuck to the outside.

And have you seen ever seen a rubber playground ball with rocks stuck to the outside? Do you really hesitate to barefoot kick an unexamined playground ball that is bouncing in your direction? I mean, I literally did this yesterday. I was out barefoot in the neighborhood with my 1 year old and the neighbor kids were kicking a ball around and it bounced in my direction and I kicked it back to them.

"Treat every object as the same" is not at all intelligent behavior for a man or a machine!

Post reply on HN