Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we using black box models in AI when we don’t need to? (2019)

hdsr.mitpress.mit.edu

Post reply on HN
Print view View original

191–200 of 200 posts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#191

Post by feanaro »

drunkpotato wrote: post #182
feanaro wrote: post #164

Earlier quoted context omitted.

I agree with your thinking on learning, so I have nothing to add, but it doesn't sound like we were referring to the same thing. I'll try rephrasing, though I find this subject extremely hard to communicate effectively. When you're looking at a red sheet of paper, reading the input in the form of a signal is not the only thing happening. The signal is definitely read and passed along to other brain circuitry for proc…

I don’t know if you’re asking rhetorically or not, but the honest answer is we don’t know yet. Those sensory experiences are called “qualia” in philosophy and cognitive science and they are not well understood yet. You might enjoy the paper “What is it like to be a bat?” https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/hum...

Yes, it was rhetorical in an attempt to get the point across. I try to refrain from naming qualia when first explaining it. I've read Nagel's paper, but I haven't thought about it for some time so thanks for reminding me.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#192

Post by inimino »

8bitsrule wrote: post #190
inimino wrote: post #126

Earlier quoted context omitted.

Sure, human pattern recognition is poorly understood, but it's a system that evolved over millions of years. We should expect more from something we ourselves designed. The question of why we would choose a black box model over something interpretable that works equally or nearly as well still needs to be asked, especially if there is a misconception of black box models working better than interpretable models due to…

>human pattern recognition is ,,, a system that evolved over millions of years. We should expect more from something we ourselves designed. Why? Human intelligence evolved over millions of years. And we're struggling to keep our changes to the world from killing us.

Because we design it, so we have the ability to choose its characteristics. The current prioritization of 1% improvements in artificial benchmarks over all else is a disease that is actively retarding progress in the entire field.

Political systems are also designed. If human behavior is in danger of killing us, we must look to the political systems we have created and solve the problems there.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#193

Post by throwawayjava »

_y5hn wrote: post #42
throwawayjava wrote: post #17

Earlier quoted context omitted.

I think it's a real open question whether the interpretable models are actually worse, or merely worse in competition/benchmark problem sets. The more deep models I build, the more I'm convinced that behind every inscrutable parameter hides a certain amount of overfitting, with maybe a few notable exceptions. E.g., can you build a decision tree that's not obviously overfit but is also susceptible to adversarial pertu…

If it can be explained in terms of precise rules, that's called programming.

The distinction between ML and programming is mostly propaganda, in the sense that it's not flat out wrong but is mostly used to win money/power. It's not actually a helpful way of understanding... anything.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#194

Post by throwawayjava »

missosoup wrote: post #170
Frost1x wrote: post #127

Earlier quoted context omitted.

I think it's also important to remember a lot of human models were abandoned because of the inherent bias and black-boxed nature of them (feelings, subjectivity, etc.). This is one of the reasons the scientific method blossomed: objectivity, rigor, transparency, reproducibility, etc. Black boxes can lead to bad decisions because it's difficult to question the process leading to decisions or highlight flaws in conclus…

> I think it's also important to remember a lot of human models were abandoned because of the inherent bias The industry needs to stop misusing the term bias this way. Virtually every attempt to find this supposed human bias has failed. Latest public example was Amazon and hiring[1] Bias is the tendency to consistently mis-classify towards a certain class or tendency to consistently to over or under-estimate. Somehow…

Yes, mathematicians mean something different and specific when using the word bias. The average non-expert is not misusing the word. They are using the word to express a different -- and far more popular -- meaning.

Neither is wrong, but insisting that a naming clash carries any substantive significance on an underlying issue is just silly. Similarly, insisting that nonmathematicians should stop using a certain word unless they use it how mathematicians use it is a tad ridiculous.

Of anything, it's more reasonable for mathematicians to change their language. After all, their intended meaning is far less commonly understood.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#195

Post by autokad »

MiroF wrote: post #130
autokad wrote: post #100

Earlier quoted context omitted.

> The main advantage of a blackbox ML solution is shorter development time to a useful performance level. I think that's kinda true but kinda false. Its true that deep learning often makes feature engineering moot. however, a lot of deep learning projects takes machine learning engineers and applied scientists along with a host of other support engineers + hardware costs ( I seen people at work say: I only used a 16…

gradient boosted trees are black box

I would disagree with that statement. There are many ways at getting to what the model is doing, feature importances, SHAP, etc. It's not as clear as logistic regression or a single decision tree, but its not a black box either.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#196

Post by tripzilch »

> there are actually a lot of applications where people do not try to construct an interpretable model, because they might believe that for a complex data set, an interpretable model could not possibly be as accurate as a black box. Or perhaps they want to preserve the model as proprietary.

I think they missed an important reason here. It's why I decided to study AI and Machine Learning: The ability to "just throw some data at it and see if it figures it out" is just so exciting, mysterious and intriguing.

I remember actually being a bit disappointed when I learned how (classic, 3 layer, early 2000s) neural networks worked. That it was just something super simple with derivatives, and resulted in something that seemed a lot like a more complex form of statistical regression. Kind of took the magic out of it for me, a little (didn't stop me of course, it was and still is an exciting field of research).

I know it's not a good reason, but surely I'm not the only one who thinks it's extremely cool that one is even able to build a black box that does useful things with what you feed it, but you don't know how it works, and yet you built it.

It's just something mystifying, and I believe there is also a tinge of fear for that disappointment, to figure it out and find out it's not really doing as clever things as you hoped it might.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#197

Post by derangedHorse »

cmendel wrote: post #172
missosoup wrote: post #170

Earlier quoted context omitted.

> I think it's also important to remember a lot of human models were abandoned because of the inherent bias The industry needs to stop misusing the term bias this way. Virtually every attempt to find this supposed human bias has failed. Latest public example was Amazon and hiring[1] Bias is the tendency to consistently mis-classify towards a certain class or tendency to consistently to over or under-estimate. Somehow…

Hold up. Bias is being used in two different manners because it has two differed meanings. When you are using bias in industry you are talking about a minor mathematical factor added to a learning rate. When we talk about human bias, we aren't. The term was never hijacked, it just has multiple meanings.

I’m not sure if he was using any of those known definitions of bias though. I’m not sure if I would define bias as the consistent act of mis-classification

Re: Why are we using black box models in AI when we don’t need to? (2019)

#198

Post by nickpsecurity »

1_over_n wrote: post #65
deepnotderp wrote: post #62

Earlier quoted context omitted.

Do you mean like symbolic regression or algorithmic information theory based stuff? If so, I'd love to get in touch, shoot me an email

We are using biophysics based approaches. Will send an email over :)

Send me one, too, if you don't mind. I like collecting these techniques to give to researchers or practitioners wanting to try new things.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#199

Post by barrkel »

feanaro wrote: post #191
drunkpotato wrote: post #182

Earlier quoted context omitted.

I don’t know if you’re asking rhetorically or not, but the honest answer is we don’t know yet. Those sensory experiences are called “qualia” in philosophy and cognitive science and they are not well understood yet. You might enjoy the paper “What is it like to be a bat?” https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/hum...

Yes, it was rhetorical in an attempt to get the point across. I try to refrain from naming qualia when first explaining it. I've read Nagel's paper, but I haven't thought about it for some time so thanks for reminding me.

I usually see the Turing Test as a thought experiment for showing the irrelevance of qualia for AI, among other philosophical objections (e.g. Chinese Room).

Re: Why are we using black box models in AI when we don’t need to? (2019)

#200

Post by computerex »

erostrate wrote: post #67
computerex wrote: post #66

This is a pointless article. Obviously everyone would prefer fully interpretable models if it was possible to get comparable performance with the leading black box/deep learning approaches. No one chooses to make black box, non-interpretable models, it's simply the fact that the models that give the best performance tend to be black box. There is a lot of work happening on trying to interpret deep learning models, be…

The article explains that their fully interpretable model actually got within 1% accuracy of the best black box model. Which was within cross validation error. That was at a data science competition whose explicit goal was to encourage people to explain black box models. They also give other examples of cases where people have used black box models unnecessarily.

> The article explains that their fully interpretable model actually got within 1% accuracy of the best black box model. Which was within cross validation error. That was at a data science competition whose explicit goal was to encourage people to explain black box models.

That's awesome, but this was a single model for a single application being measured against presumably by a limited test criteria/benchmarks. The point still stands that the recent narrow AI "renaissance" is largely due to deep learning, which is inherently black box. There is a lot of work going on in making deep learning more interpretable precisely because it's so prominent today and because its lack of interpretability is a huge con.

Machine vision for example has come a long way due to deep learning. A lot of autonomous car companies are relying on it for perception. All of these companies hate the fact that there is no way to tell when the classifier will fail or why it'd fail. When using deep learning in finance setting, its lack of interpretability is a huge downside.

Despite this deep learning is being used because it provides an appreciable improvement in performance over the alternatives. Certainly there could be alternatives to deep learning that may perform just as well or even better. But finding those alternatives for the vast number of applications deep learning is being used for today is much easier said than done.

Post reply on HN