61–70 of 72 posts
Earlier quoted context omitted.
We only know which ones are spurious if we run the trimming algorithms. I wrote a paper about 10 years ago showing how to use generic algorithms on the weight matrix to remove spurious interactions. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2538912/
Genetic algorithms?
I think most people agree in the deep learning community that ways to understand NN's are useful, if for nothing other than debugging, but I don't think the community agrees at all over whether explainability is necessary to use it. In many cases, it depends on the context. For self driving cars, most companies only use neural networks for components of the cars and an explainable algorithm on top of that that interp…
The poll is biased. There is no evidence that performance and explainability are at odds.
Earlier quoted context omitted.
Good luck explaining that to a judge who demands to know why the AI made the decisions that caused an accident, and what exactly the manufacturer is doing to prevent it from happening again.
There are tons of physical processes that aren't explainable. Most current petrochemical plants have inner workings that are only half known, at best. What are we doing about it? Modeling the failure, and engineering a solution that no longer fails in that situation. Unit testing our way to robustness!
When these processes kill people we very much do somethint about them.
Earlier quoted context omitted.
What if the AI drove the car into a pedestrian? The pedestrian didn't consent to letting the AI drive.
What if a drunk drives the car into a pedestrian? Not much content there, either.
This is easy to explain. The driver is at fault. Explainability is what we are debating.
I don't think AI needs to be able to explain itself in order to be trusted. Human beings, in general, cannot give arbitrarily deep and valid explanations for their actions, and yet, somehow, we manage to come to trust them.
That's true, but in general humans are concerned about the consequences of their actions and are metacognitive about them, or at the very least can be punished for making the wrong choices, its an iterative corrective process. How do you apply the same logic to a neural net? It has no comprehension of desirable outcomes, or what its choices actually entail and never can. There is no way to tell a neural net that a hu…
Earlier quoted context omitted.
Because we dont know which ones are spurious and the parent is pretty optimistic about the idea of being able to analyize which ones are/are not.
We only know which ones are spurious if we run the trimming algorithms. I wrote a paper about 10 years ago showing how to use generic algorithms on the weight matrix to remove spurious interactions. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2538912/
Like for this image of Grandma, these cells are recognizing it, so the others are spurious. Oops, for this other image of Grandma, a different subset are recognizing it.
If, say, the same 90% of the structure is not involved in recognizing any instances, then why wouldn't we have made the whole thing 10X smaller in the first place and trained that much smaller network? Or why don't we make it 10X smaller now? Trim away the dead weight and make it lighter and faster.
Earlier quoted context omitted.
What if a drunk drives the car into a pedestrian? Not much content there, either.
> What if a drunk drives the car into a pedestrian? This is easy to explain. The driver is at fault. Explainability is what we are debating.
Earlier quoted context omitted.
> What if a drunk drives the car into a pedestrian? This is easy to explain. The driver is at fault. Explainability is what we are debating.
So the AI is at fault then.
Earlier quoted context omitted.
If a self driving car decides to ram into a tree/another car at full speed regularly, im guessing there will be a huge demand for explainability, not just a collective shrug and "I guess we need to train it more dunno lol".
Very much not. Becauae there’s nothing to explain. The software is broken and needs to be fixed, and if the problem is severe and the manufacturer knowingly tries to cover it up, that manufacturer is culpable of negligence. The software has no more agency than a cruise control.
Earlier quoted context omitted.
Since alphago plays the game differently (optimizing on odds to win, even if by a little, instead of the human method which involves intuition about shapes), it may be that its explanation if it were programmed to provide one would not provide any human readable 'insight' which an expert could then take to improve his or her game. Imagine a simpler example, some optimization solution using multiple dimensions and lin…
At the higher level it isn’t just shapes. The human player uses the shapes the help estimate the future value of any group of stones. They are running a priority queue of likelihood’s for each part of the board. When they resign that probability has dropped to zero. If you’ve played against a dan player a few times, you might have had the experience of them ignoring a move you made and play elsewhere. That’s what hap…