Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we using black box models in AI when we don’t need to? (2019)

hdsr.mitpress.mit.edu

Post reply on HN
Print view View original

11–20 of 200 posts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#11

Post by James-primitive »

taneq wrote: post #9
zevets wrote: post #7

I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability. Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met. This logic is perhaps most evident in google/facebooks conte…

Well maybe also because black box models can be applied to a wider range of systems and can help to understand general properties of a class of algorithms. I’m sure it’s mostly all a conspiracy, though.

> I’m sure it’s mostly all a conspiracy, though.

Yeah, mostly

Re: Why are we using black box models in AI when we don’t need to? (2019)

#12

Post by klyrs »

sigmaprimus wrote: post #2

I think the main reason is that creating a so called "Black Box" model is faster and easier to some degree. Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?" I tend to agree with the articles premise that any model being used in critical decision making should at the very leas…

> Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?"

Exactly. We included mathematicians, computer scientists and engineers in the design of those calculators. They use algorithms backed by rigorous proofs, run upon circuits with rigorous tolerances such that the answer is correct (to a reasonable amount of precision). Kids with their calculators are intellectually lazy -- we should expect that, but continue to teach otherwise. To see this from full-grown adults in academia is an embarrassment. But, god, would I enjoy a ML salary...

Re: Why are we using black box models in AI when we don’t need to? (2019)

#13

Post by mkl »

They never got to the results of the competition and how the interpretable model did!

From https://www.fico.com/en/newsroom/fico-announces-winners-of-i...: "The team representing Duke University, which included Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang and Tong Wang, received the FICO Recognition Award acknowledging their submission for going above and beyond expectations with a fully transparent global model and a user-friendly dashboard to allow users to explore the global model and its explanations. The Duke team took home $3,000."

Cynthia Rudin is one of the article's authors.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#14

Post by taneq »

sigmaprimus wrote: post #3
sigmaprimus wrote: post #2

I think the main reason is that creating a so called "Black Box" model is faster and easier to some degree. Its very similar to the argument kids make "Why do I have to learn how to do long division when I have a calculator?" or "Why do I need to show my work on the exam if I got the right answers?" I tend to agree with the articles premise that any model being used in critical decision making should at the very leas…

Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. As much as I hate regulations I think this may be a good place to legislate in order to protect those wrongly affected. It makes me think about the mess the derivatives ma…

It’s not like your bank manager used to give you a phone call to explain in depth why your loan was declined. Most of these situations were already impenetrable from a human standpoint.

Humans are far more ‘black box’ than neutral nets, we just happen to be able to construct plausible explanations in parallel.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#15

Post by anfilt »

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough."

For-instance when a model starts being over fitted, one can say it's pretty much converging to something akin to a LUT (Look up table). While over fitting is generally undesirable it still might be interesting. Principally, if someone could figure out how basically it is indexing the data. Perhaps you could create a simpler hash function or find some rules for creating useful hash functions for that type of data.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#16

Post by RSchaeffer »

mkl wrote: post #13

They never got to the results of the competition and how the interpretable model did! From https://www.fico.com/en/newsroom/fico-announces-winners-of-i... : "The team representing Duke University, which included Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang and Tong Wang, received the FICO Recognition Award acknowledging their submission for going above and beyond expectations with a fully…

But how did their model compare against others? The article only mentions how their interpretable model compared against their own ML attempts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#17

Post by throwawayjava »

owenshen24 wrote: post #5

My understanding is that interpretable models, especially for neural networks, are very far and away from state of the art performance. Work that e.g. tries to approximate neural nets with decision trees have yet to be applied to very large models [1]. Even in computer vision, which is where I think they've been most successful, the visualization techniques used seem more suggestive, then explanatory [2]. [1] http://…

I think it's a real open question whether the interpretable models are actually worse, or merely worse in competition/benchmark problem sets. The more deep models I build, the more I'm convinced that behind every inscrutable parameter hides a certain amount of overfitting, with maybe a few notable exceptions. E.g., can you build a decision tree that's not obviously overfit but is also susceptible to adversarial perturbations?

Re: Why are we using black box models in AI when we don’t need to? (2019)

#18

Post by owenshen24 »

throwawayjava wrote: post #17
owenshen24 wrote: post #5

My understanding is that interpretable models, especially for neural networks, are very far and away from state of the art performance. Work that e.g. tries to approximate neural nets with decision trees have yet to be applied to very large models [1]. Even in computer vision, which is where I think they've been most successful, the visualization techniques used seem more suggestive, then explanatory [2]. [1] http://…

I think it's a real open question whether the interpretable models are actually worse, or merely worse in competition/benchmark problem sets. The more deep models I build, the more I'm convinced that behind every inscrutable parameter hides a certain amount of overfitting, with maybe a few notable exceptions. E.g., can you build a decision tree that's not obviously overfit but is also susceptible to adversarial pertu…

I agree that there's probably a good question about to what extent looking at benchmark sets is biasing our judgment.

But it's also unclear to me how to get a decision tree to perform as well on image recognition tasks the same way that a CNN does. (Of course, as you mention, the CNN will likely face adversarial examples.)

Re: Why are we using black box models in AI when we don’t need to? (2019)

#19

Post by psv1 »

anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#20

Post by psv1 »

zevets wrote: post #7

I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability. Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met. This logic is perhaps most evident in google/facebooks conte…

> purpose of black-box models is to be impossible to understand

This is just outright wrong. And the argument about moderation is completely unrelated.

Post reply on HN