Why are we using black box models in AI when we don’t need to? (2019)
hdsr.mitpress.mit.edu
Why are we using black box models in AI when we don’t need to? (2019)
1–10 of 200 posts
Re: Why are we using black box models in AI when we don’t need to? (2019)
#2I think the main reason is that creating a so called "Black Box" model is faster and easier to some degree.
Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?"
I tend to agree with the articles premise that any model being used in critical decision making should at the very least have a list of parameters and the weight given to each, no matter how long and complicated such a list might be.
I think eventually this will be the end result of most critical implementations of machine learning applications as they make their way through the courts, as I can't imagine a judge accepting the argument "The machine made a mistake, we are not sure why or the reason why is proprietary, but we are not responsible because the machine made the error".
Re: Why are we using black box models in AI when we don’t need to? (2019)
#3I think the main reason is that creating a so called "Black Box" model is faster and easier to some degree. Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?" I tend to agree with the articles premise that any model being used in critical decision making should at the very leas…
Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. As much as I hate regulations I think this may be a good place to legislate in order to protect those wrongly affected.
It makes me think about the mess the derivatives markets made in 2008
Re: Why are we using black box models in AI when we don’t need to? (2019)
#4I think the main reason is that creating a so called "Black Box" model is faster and easier to some degree. Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?" I tend to agree with the articles premise that any model being used in critical decision making should at the very leas…
The killer feature with "black box" models is that you can do illegal discrimination with regards to eg. race and gender and then just blame the computer since you can't see the weight as easily as in component analysis.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#5My understanding is that interpretable models, especially for neural networks, are very far and away from state of the art performance. Work that e.g. tries to approximate neural nets with decision trees have yet to be applied to very large models [1].
Even in computer vision, which is where I think they've been most successful, the visualization techniques used seem more suggestive, then explanatory [2].
[1] http://www.shallowmind.co/jekyll/pixyll/2017/12/30/tree-regu... [2] https://distill.pub/2019/activation-atlas/
Re: Why are we using black box models in AI when we don’t need to? (2019)
#6I've happen to be in that workshop room and had chosen the 2% robot, not the 15% surgeon. If I remember correctly, the point of the question was determining willingness of society to use black box models, not challenging the need for models interpretability.
Interpretability obviously doesn't hurt accuracy. But it is costly to engineer. And not always possible to make. Not always possible, because human capacity (and willingness to put in the effort) into understanding the explanation is limited.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#7I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability.
Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met.
This logic is perhaps most evident in google/facebooks content moderation dilemmas, where both companies refuse to define any sort of non-vague actionable standards about what they censor (or fail to promote) as 1) they can't provide them and 2) they don't want to be held responsible for what they actually are.
Then, as outrage has grown over facebook/google's terrible content moderation and censorship policies, and the need for accountability has grown, both companies have been forced to hire ever more moderators and censors, because the technologies as claimed just don't work well enough.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#8I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability. Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met. This logic is perhaps most evident in google/facebooks conte…
Sounds more like a useful side effect than anything someone would design on purpose. But that's an interesting point none-the-less.
Occam's razors the easier way to explain the actions of the big guys and their own internal responses to cultural pressures to not allow things they don't like from happening or being tolerated in the world.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#9I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability. Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met. This logic is perhaps most evident in google/facebooks conte…
Well maybe also because black box models can be applied to a wider range of systems and can help to understand general properties of a class of algorithms. I’m sure it’s mostly all a conspiracy, though.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#10I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability. Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met. This logic is perhaps most evident in google/facebooks conte…
>the purpose of black-box models is to be impossible to understand
nothing is impossible to understand