Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we using black box models in AI when we don’t need to? (2019)

hdsr.mitpress.mit.edu

Post reply on HN
Print view View original

151–160 of 200 posts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#151

Post by netcan »

dragonwriter wrote: post #149
netcan wrote: post #148

Earlier quoted context omitted.

Reading "subjective" to mean "nonexistent" is a potentially big mistake. Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal. In this area, we don't exactly…

All objective criteria are chosen, directly or indirectly, based on subjective criteria.

Also true. Subjectivity is unavoidable as long as we are relevant.. or so it seems circa 2020

Re: Why are we using black box models in AI when we don’t need to? (2019)

#153

Post by hesselink »

sytelus wrote: post #152

There are probably dozens of holes one can drill into this sensationally titled click-baity study. It's surprising what gets through Nature's review system these days.

And yet you provide none. What do you base this on?

Re: Why are we using black box models in AI when we don’t need to? (2019)

#154

Post by bitL »

lonelappde wrote: post #113
bitL wrote: post #99

I am worried about the recent trend of "ethical AI", "interpretable models" etc. IMO it attracts people that can't come with SOTA advances in real problems and its their "easier, vague target" to hit and finish their PhDs while getting published in top journals. Those same people will likely at some point call for a strict regulation of AI using their underwhelming models to keep their advantage, faking results of th…

In your view, it's better to answer the wrong questions optimally than the right questions suboptimally? That's not the kind of AI want putting people in jail or denying them housing or healthcare or shooting missiles at them.

There are two parts to my opinion:

1) I would prefer something that works most of the time (blackbox) than something interpretable that doesn't

2) if my DL converged to e.g. solving some complex partial differential equation, how does it help me that I could "interpret that" if 99.9999% human population has no clue what is going on regardless? "Your loan was rejected, because the solution to this obscure PDR with these starting conditions said so; these Nobel prize economy winners created this model with these simplifications and we deem it good enough to keep you where you are."

Re: Why are we using black box models in AI when we don’t need to? (2019)

#155

Post by MaxBarraclough »

perl4ever wrote: post #123
lonelappde wrote: post #112

Earlier quoted context omitted.

Parent wasn't saying that the weighting process was rational. The weights are also feelings.

As long as we don't know what the weighting process is, we shouldn't make unfounded statements about how it works. You're asserting that it exists, but nobody knows anything about what it is. It's a program that you haven't read the source code for.

At this point we seem to be reasoning by analogy. We'd do better to just study the fact of the matter. Psychologists do not tend to discuss this topic in terms of weightings.

https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r...

http://www.theunconsciousconsumer.com/consumer-psychology/20... (regrettably thin on citations)

https://academic.oup.com/scan/article/6/4/460/1648209

https://en.wikipedia.org/wiki/Rationalization_(psychology)

Re: Why are we using black box models in AI when we don’t need to? (2019)

#156

Post by nurettin »

We have all these tools to build neural networks and we use them to peform feats where a simple C4.5 would have sufficed.

To continue innovation in the algorithmic field, I think we need a branch of scientific research or a competition where researchers are constrained to 100MHz CPU clocks and no GPU cheats when actually running their algorithms (not when loading and parsing data)

Re: Why are we using black box models in AI when we don’t need to? (2019)

#157

Post by shrimpx »

crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

It's true that humans are black boxes but there are whitebox techniques such as the scientific method and algorithms that are justified step by step.

I think the article tries to compare black box learned algorithms to explicit algorithms. Humans being a blackbox is more of a philosophical issue from that perspective.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#158

Post by mkolodny »

xelxebar wrote: post #74
mkolodny wrote: post #39

Earlier quoted context omitted.

Say you're building a self-driving car. You need to take an image of what the car sees, and figure out whether/where there is a pedestrian/car/bicycle/etc in the image. So your input/variable is the image itself. Almost every image the car sees is going to be different. I'd guess that's why GP said that it doesn't make sense to use a hashing function - there's little value in mapping inputs to results, because your i…

It seems like you might be deep in the ML rabbit hole. Zoom out a bit. A hash function is "just a function." Every image labeling black box can be though of as a hash from images to label vectors. OP's comment effective asks whether there is another, more grokkable function that maps/hashes inputs to the same labels. Granted, that question boils down to "can we create human-understandable models?" which is the whole…

Thanks for explaining that perspective. I was seeing things a little too narrow-mindedly for sure.

Based on your phrasing of the issue, it seems like we could think of the problem as: can we reduce the number of parameters in an ML model to the point where humans can understand all of the parameters? That's related to an active research area - minimizing model size. ML models can have billions of parameters. It's unfeasible for a human to evaluate all of them. Research shows that (sometimes) you can reduce the number of parameters in a model 97% without hurting it's accuracy [0]. But, 3% of a billion parameters is still way too many for people to evaluate. So I think the answer so far is no, we can't create human-understandable models that perform as effectively as black boxes.

[0] https://openreview.net/forum?id=rJl-b3RcF7

Re: Why are we using black box models in AI when we don’t need to? (2019)

#159

Post by choeger »

tdhttt wrote: post #145
crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

One more benefit: an interpretable model could at least give some hint when the model is not working as intended (e.g. debugging).

Sometimes I have the feeling that this is exactly why so many people prefer black-box modeling. You are not going to be criticised and you don't have to do all the hard work of software development.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#160

Post by IshKebab »

sullyj3 wrote: post #139
crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

Black boxness isn't a property of the thing, it's a property of our understanding of the thing. Just because we don't understand the how pattern recognition in humans works, doesn't mean the process has no internal structure, or that we couldn't potentially understand it in the future.

Right, and currently we understand neither human vision, nor CNN vision - they are both black boxes. I don't think anyone said they must remain black boxes forever.

Edit: I guess you were referring to this:

> Robust pattern recognition based on 100+'s of factors is just inherently black box.

I think he meant " ... given our current knowledge."

Post reply on HN