Scientists Increasingly Can’t Explain How AI Works
1–10 of 231 posts
Re: Scientists Increasingly Can’t Explain How AI Works
#2The overly simplified explanation is that the AI models see patterns in the training data that we do not.
Feed a system a series of pictures of stop signs, and humans might key on things like the red color, or the shape, or even the word "STOP". An AI model might also recognize that they commonly have 2 bolts, and are attached to a post, and have a contrasting border.
Typical machine vision systems can only classify what they have been trained on, they are ignorant to other objects. Train a system on a bunch of STOP signs, and then show it a SPEED LIMIT sign with morning sun giving the sign an amber glow, and the AI system might call it a STOP sign.
Again, this is an overly simplified explanation, but we can explain how AI works, we just can't always explain or know what parameters in the training data it keyed on.
Re: Scientists Increasingly Can’t Explain How AI Works
#3> AI experts and researchers are warning developers to take a step back and focus more on how and why a system produces certain results than the fact that the system can accurately and rapidly produce them.
We need to find a way to incentivize that otherwise it’s not going to happen. Perhaps looking how the security industry works with “buy bounties” is a place to borrow ideas from?
Re: Scientists Increasingly Can’t Explain How AI Works
#4To me the problem of AI is not how it works but how reliable it works.
Do we have a way to test AI to prevent corner cases which could lead to catastrophic results?
Re: Scientists Increasingly Can’t Explain How AI Works
#5There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works.
I think that assumption is totally wrong, and trying to reconcile the two is probably a distraction and a waste of time.
Evolution led to human intelligence just fine on its own; why are humans not simply the catalyst in the next stage of this natural process from which something more complex arises?
With regard to the moral questions around machine learning, I think people are overthinking things. There needs to be a sharp (societal) line between inferring causation and the capability for prediction. Maximize the latter any way that you can, but a lot of careful thought should be put into how the results of a prediction are utilized. Instead we are currently going about this backward by trying to haphazardly “clean” the input data so that the output gets a free pass on how it can be used. We can never decouple all of the biases and eliminate intra-predictivity in the input data though, so it’s fundamentally a bad approach.
Re: Scientists Increasingly Can’t Explain How AI Works
#6> AI experts and researchers are warning developers to take a step back and focus more on how and why a system produces certain results than the fact that the system can accurately and rapidly produce them. We need to find a way to incentivize that otherwise it’s not going to happen. Perhaps looking how the security industry works with “buy bounties” is a place to borrow ideas from?
I fairly familiar with bug bounties in the security industry, what do you propose specifically?
Re: Scientists Increasingly Can’t Explain How AI Works
#7> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has.
I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers.
In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-style parallel search with a certain evaluation function, and we could simulate a similar search by hand if we wanted to. But this is no better than explaining the output of a neural network as "matrix multiplication plus a few non-linearities".
Even back then a chess expert could try and rationalize certain moves in human-like terms: "oh, Deep Blue realised it needs to fight for the dark squares on the queenside". But this wasn't a real explanation, just like "this part of the picture looks like dog hair" isn't necessarily a correct explanation for why an AI labels an image as a dog.
Whenever you perform a massive amount of computation, you can get results that are nearly impossible to explain.
Re: Scientists Increasingly Can’t Explain How AI Works
#8To me the problem of AI is not how it works but how reliable it works. Do we have a way to test AI to prevent corner cases which could lead to catastrophic results?
To be fair, this is a problem with people too. We use "interpersonal weirdness" as a proxy for "may produce catastrophic results" to some degree.
Re: Scientists Increasingly Can’t Explain How AI Works
#9To me the problem of AI is not how it works but how reliable it works. Do we have a way to test AI to prevent corner cases which could lead to catastrophic results?
We can't even do that with humans.
Best we've got is qualifications, and punishments for breaking rules.
Re: Scientists Increasingly Can’t Explain How AI Works
#10There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works. I think that assumption is totally wrong, and trying to reconcile the two is probably a distraction and a waste of time. Evolution led to human intelligence just fine on i…
What people are really saying, but never will admit, is that they want AI to mirror their own biases, not to have none. That's the reason they are "cleaning" the input data.