Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we using black box models in AI when we don’t need to? (2019)

hdsr.mitpress.mit.edu

Post reply on HN
Print view View original

181–190 of 200 posts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#181

Post by Ericson2314 »

The interest in black-box models is partly for the reasons given, that it is presumed that their inexplanability makes them more powerful, but also because business people are sick of dealing with programmers and analysts as a class, people with arcane knowledge limited replacability, and a penchant for blowing deadlines.

The dream of machine learning is to fire the programmer and break up the last trade (baring doctors and lawyers).

If it was just business people being alienated, I'd say fuck it they deserve it. But the fact is everyone, and many of the programmers too, are alienated too when much of the code makes sense to no one. I would rather fix programming to be comprehensible to more people even though it will take down the walls between programmer and non-programmer.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#182

Post by drunkpotato »

feanaro wrote: post #164
jotm wrote: post #163

Earlier quoted context omitted.

By sensing, I mean just that, reading input. You mean reasoning / decision , I guess. That would come from learned data, experience, imo. A child who's never touched a red hot electric stove, for example, would not have any bias towards it. No fear, no love, unless they previously interacted with one. They would try and get close, watch it, smell it, touch it, to learn more. The most interesting part of consciousness…

I agree with your thinking on learning, so I have nothing to add, but it doesn't sound like we were referring to the same thing. I'll try rephrasing, though I find this subject extremely hard to communicate effectively. When you're looking at a red sheet of paper, reading the input in the form of a signal is not the only thing happening. The signal is definitely read and passed along to other brain circuitry for proc…

I don’t know if you’re asking rhetorically or not, but the honest answer is we don’t know yet. Those sensory experiences are called “qualia” in philosophy and cognitive science and they are not well understood yet. You might enjoy the paper “What is it like to be a bat?” https://warwick.ac.uk/fac/cross_fac/iatl/study/ugmodules/hum...

Re: Why are we using black box models in AI when we don’t need to? (2019)

#183

Post by perl4ever »

MaxBarraclough wrote: post #155
perl4ever wrote: post #123

Earlier quoted context omitted.

As long as we don't know what the weighting process is, we shouldn't make unfounded statements about how it works. You're asserting that it exists, but nobody knows anything about what it is. It's a program that you haven't read the source code for.

At this point we seem to be reasoning by analogy. We'd do better to just study the fact of the matter. Psychologists do not tend to discuss this topic in terms of weightings. https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r... http://www.theunconsciousconsumer.com/consumer-psychology/20... (regrettably thin on citations) https://academic.oup.com/scan/article/6/4/460/1648209 https://en.wikipedia.org/wiki/R…

I'm not reasoning by analogy; I'm disputing that it's useful.

If you look at the physical world with objects that have relationships and characteristics, and describe a mental world that also has discrete entities that interact in designated ways, then I think you are reasoning by (unfounded) analogy.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#184

Post by YeGoblynQueenne »

rubyn00bie wrote: post #44

> [...] these black box models are created directly from data by an algorithm, meaning that humans, even those who design them, cannot understand how variables are being combined to make predictions. So uhhh, isn't this like not science? Like my biggest problem with "machine learning" is people assume the data they have can correctly answer the question they want to answer. If your data is off, your model might be in…

>> I guess I can sum this up with, I wish people looked at AI more as a tool to help guide our intuition helping us solve problems we already have well defined knowledge (and data) of, and not as an means to an end itself.

Problem is, most machine learning algorithms cannot incoprorate background knowledge except by hard-coding inductive biases (as, e.g. the convolutional filters in convolutional neural nets). Unfortunately, this is a very limited way to incorporate existing knowledge.

This is actually why most machine learning work tries to learn concepts end-to-end, i.e. without any attempt to make use of previously learned or known concepts: because it doesn't have a choice.

Imagine trying to learn all of physics from scratch- no recourse to knowledge about mechanics, electomagnetism, any kind of dynamics, anything. That's how a machine learning algorithm would try to solve a physics problem. Or any other problem for which "we already have well defined knowledge (and data) of". We might as well be starting at around before the stone age.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#185

Post by 6gvONxR4sf7o »

crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

Not all human thinking is the same. How you decide the amount to tip your waiter is not at all the same as how your doctor diagnoses you. And at we have a zillion years of evolution towards understanding and judging and conversing with each other's black boxes.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#186

Post by Balgair »

sandpaper26 wrote: post #124
Balgair wrote: post #114

Earlier quoted context omitted.

Object recognition is firmly in the realm of Neuroscience. We've had a good idea what the eye-V1 path is doing in mammals for a few decades now. To the point where we are implanting chips into the brain to (crudely) re-establish vision in blind patients. Though the research is very much on-going, it is being better understood with each passing year.

Respectfully, unless I'm not understanding what you mean by this, that seems very far from object recognition. You're replicating the data stream that would normally be coming from the optic nerve/retina, but that's not the same thing as understanding how the data of an object's image relates recognition of that object. If I've never seen the entire Mona Lisa before (but maybe have seen portions of it), your proposed…

I don't think you are understanding. But Bio isn't meant to be easy.

At each synapse a significant amount of data processing occurs, right from the rod and cone cells in the retina and at nearly each synapse along the way. 'Data' is more-ish conserved into V1 where all hell seems to break loose, path-wise. After V1 and in nearly the entire cortex, these data are distributed in unique and very complicated ways throughout the brain. Most of the mammalian eye-V1 pathway is well understood, but that process is too much for a comment on HN. But, I want to stress, we have a very very good idea of what is going on.

That said, the research seems to indicate (highly debated still) that there are specific synapses that encode 'the left side of grandma's nose in dim lighting from 10 feet away' and other many other things. Again, I cannot stress enough that such 'grandma' synapses are not firm science and there is a LOT of research still ongoing. But the evidence seems to be pointing that way at this time. Such cells are fed this information from V1 and, likely, a lot of other places. When such 'grandma' cells fire, they then send that signal out to other cells they may be connected to. Such a system is likely replicated many times all over the brain; there are many copies of 'grandma' cells and they are wired up in many different ways. The physical location of 'grandma' cells in highly unique, if they exist at all. Again, research is very much on-going.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#187

Post by Balgair »

username90 wrote: post #120
Balgair wrote: post #117

Earlier quoted context omitted.

> I think it's important to note that human pattern recognition is basically black-box as well. Mammalian pattern recognition is fairly well understood. We have a good understanding of the information path from the eye to V1 in most model animals. So sure are we of this pathway that we've successfully implanted ships into blind patients and restored sight (results vary a LOT, though). I know this may not fit pattern…

Knowing how to input data to the visual cortex doesn't mean that we know how it works. It is the same with neural networks, we know how to pass data to them and how they do the math, but we can't trace back its reasoning to know why it comes to the conclusions it does.

We have a very very good idea of the mammalian eye-V1 pathway. It's not a straightforward path and describing it in a HN comment is not going to be useful to anyone. Explaining the complex, but well known, pathway is a job best done by other people than myself. We are well aware of what 'data' processing the mammalian brain does at nearly every synapse, and it is a LOT. Right form the cones and rods, data is being highly processed (synapses are VERY noisy, as it turns out). So while more research always comes up with more questions, we are fairly confident about the eye-V1 pathway (contrast that to other organs, like the vestibules, where we're fairly in the dark still).

That said, wait, what? We can't do the back-propagation calculus in a NN? When has that been true and for how long? I thought it was fairly straightforward to know the weights of the connections of the nodes in your network. It's just a tensor you grep for, right?

Re: Why are we using black box models in AI when we don’t need to? (2019)

#188

Post by XuMiao »

missosoup wrote: post #177
TomMckenny wrote: post #176

Earlier quoted context omitted.

If an algorithm is clearly sorting on irrelevant criterion, especially a black box algorithm, we normally assume it's a bug. It's not reasonable to reverse that, assume the code is incapable of being mistaken and say that obviously irrelevant criterion are somehow correct in an unknown way. Amazon's problem is a bug, they even describe it's nature. And given how flawed their recommendation algorithms are, it's especi…

You say "obviously irrelevant criterion" Data says criterion is an eigenvalue and no matter how hard amazon tried to blind the solution to that eigenvalue, the ML system kept finding ways to infer it because it was that strongly correlated with the fitness function. This is the difference between political newspeak '''bias''' and actual bias. Amazon scrapped the model despite it performing just fine and being bias-fr…

Supervised learning algorithms assume that the input data are iid of the future. This is not valid in most of the real applications. The observation that we see men more than women in programming does not necessarily generalize to the future. That's why online learning provides an exploitation vs exploration mechanism to minimize the bias in the hindsight. In many applications, people just forgot about this simple strategy and blame the bias caused by supervised learning to the black box model.

Of course, black box AI itself is not the right solution. As more and more cross domain multitask settings emerge, open box AI will gradually take off. It is about compositional capability like functor and monad in functional language. Explanable or not is just a communication problem which is parallel to the ultimate intelligence problem. It is very possible that human intelligence is bounded.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#189

Post by agra »

wongarsu wrote: post #175
agra wrote: post #166

Earlier quoted context omitted.

Was the fact that objects fall down a "black box" before Newton came up with some equations? I don't think its fair to describe CNN's and human vision as being black boxes in the same way.

The fact that objects fall down was well understood, obviously. What wasn't understood was the mechanism that makes this happen. Why did that apple fall down instead of up? Where exactly is down a round earth? Where is down when I stand next to the Himalaya? Why don't planets fall down? In my eyes this is very similar to our lack of understanding about the inner workings of a CNN, or human vision. We understand the i…

I see it differently.

The distinction I would make is that even before Newton, people could make useful predictions about the trajectory of falling objects. My dog does it well, every time he catches a ball. He even seems to grok the effect of wind on a frisbee.

We have no way of predicting how a CNN will react to an input. GANs really illustrate this point. I think this is an important distinction.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#190

Post by 8bitsrule »

inimino wrote: post #126
crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

Sure, human pattern recognition is poorly understood, but it's a system that evolved over millions of years. We should expect more from something we ourselves designed. The question of why we would choose a black box model over something interpretable that works equally or nearly as well still needs to be asked, especially if there is a misconception of black box models working better than interpretable models due to…

>human pattern recognition is ,,, a system that evolved over millions of years. We should expect more from something we ourselves designed.

Why? Human intelligence evolved over millions of years. And we're struggling to keep our changes to the world from killing us.

Post reply on HN