Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we using black box models in AI when we don’t need to? (2019)

hdsr.mitpress.mit.edu

Post reply on HN
Print view View original

161–170 of 200 posts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#161

Post by feanaro »

galaxyLogic wrote: post #142
Frost1x wrote: post #127

Earlier quoted context omitted.

I think it's also important to remember a lot of human models were abandoned because of the inherent bias and black-boxed nature of them (feelings, subjectivity, etc.). This is one of the reasons the scientific method blossomed: objectivity, rigor, transparency, reproducibility, etc. Black boxes can lead to bad decisions because it's difficult to question the process leading to decisions or highlight flaws in conclus…

That is the fascinating thing about consciousness. How do we "know" we see red and not green or vice versa? How do we "know" we are feeling heat or coldness? We just do. We feel different things in different, yet recognizable ways. And these "feelings" are multi-dimensional, feeling heat and feeling color are not on the same axis. Red and Green seems to be on different points of the same axis and hot and cold are poi…

These feelings you're describing are called qualia (https://en.wikipedia.org/wiki/Qualia), in case you're not familiar with the term.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#162

Post by feanaro »

jotm wrote: post #144
galaxyLogic wrote: post #142

Earlier quoted context omitted.

That is the fascinating thing about consciousness. How do we "know" we see red and not green or vice versa? How do we "know" we are feeling heat or coldness? We just do. We feel different things in different, yet recognizable ways. And these "feelings" are multi-dimensional, feeling heat and feeling color are not on the same axis. Red and Green seems to be on different points of the same axis and hot and cold are poi…

Physical sensation seems simple tbf. Any robot can sense temperature or color gradient. It's the decision making that's complicated. To make a decision on whether it would like chocolate ice cream or not ( before trying it ) a brain will take all the data gathered over time, cross reference it with related data (does chocolate taste good alone? \ did I like it? \ It looked rather inedible. \ was it high cocoa content…

> Physical sensation seems simple tbf. Any robot can sense temperature or color gradient.

It can measure them and process it as a number, but when does sensing come into play? Does a red sheet of paper recorded by a simple camera evoke the same "feeling" of redness inside the camera that it evokes in a human being?

The flippant answer is "of course not, the camera has no consciousness/perception". The question is then how and when this feeling of "redness" gets created and what are the necessary conditions for it to happen.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#163

Post by jotm »

feanaro wrote: post #162
jotm wrote: post #144

Earlier quoted context omitted.

Physical sensation seems simple tbf. Any robot can sense temperature or color gradient. It's the decision making that's complicated. To make a decision on whether it would like chocolate ice cream or not ( before trying it ) a brain will take all the data gathered over time, cross reference it with related data (does chocolate taste good alone? \ did I like it? \ It looked rather inedible. \ was it high cocoa content…

> Physical sensation seems simple tbf. Any robot can sense temperature or color gradient. It can measure them and process it as a number, but when does sensing come into play? Does a red sheet of paper recorded by a simple camera evoke the same "feeling" of redness inside the camera that it evokes in a human being? The flippant answer is "of course not, the camera has no consciousness/perception". The question is the…

By sensing, I mean just that, reading input. You mean reasoning/decision, I guess.

That would come from learned data, experience, imo. A child who's never touched a red hot electric stove, for example, would not have any bias towards it. No fear, no love, unless they previously interacted with one. They would try and get close, watch it, smell it, touch it, to learn more.

The most interesting part of consciousness is how does one decide based on incomplete information? If you need to learn more, where do you find the raw information? It seems to be done subconsciously, some people have a higher affinity for learning/fact-finding than others. But all people are pre-wired to learn from others, distributed computing works best heh.

I guess you're asking the same question as me, where is this "programming" and how is it created? It's not just raw experience, it seems to be genetic/evolutionary. A sort of basic firmware to bootstrap further learning.

I find it fascinating, it would seem the brain never stops processing the data it acquires. During the day, and during the night, it always runs learning jobs.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#164

Post by feanaro »

jotm wrote: post #163
feanaro wrote: post #162

Earlier quoted context omitted.

> Physical sensation seems simple tbf. Any robot can sense temperature or color gradient. It can measure them and process it as a number, but when does sensing come into play? Does a red sheet of paper recorded by a simple camera evoke the same "feeling" of redness inside the camera that it evokes in a human being? The flippant answer is "of course not, the camera has no consciousness/perception". The question is the…

By sensing, I mean just that, reading input. You mean reasoning / decision , I guess. That would come from learned data, experience, imo. A child who's never touched a red hot electric stove, for example, would not have any bias towards it. No fear, no love, unless they previously interacted with one. They would try and get close, watch it, smell it, touch it, to learn more. The most interesting part of consciousness…

I agree with your thinking on learning, so I have nothing to add, but it doesn't sound like we were referring to the same thing. I'll try rephrasing, though I find this subject extremely hard to communicate effectively.

When you're looking at a red sheet of paper, reading the input in the form of a signal is not the only thing happening. The signal is definitely read and passed along to other brain circuitry for processing, but somehow another thing happens: the experience of red (which is what the other poster called "feeling"). This experience of red is fundamentally different from the experience of green or blue or from the complete absence of looking. This experience is what I was referring to.

This experience seems to happen without any prior knowledge or exposure to the color red. The first time you stumble upon red light, this experience of red arises. You can tell red from green by the marked difference in experiences, but you cannot explain this difference in words to someone who hasn't experienced red or green themselves.

How does this experience get created? Does any sensor (such as a camera) have such experiences? Why not?

Re: Why are we using black box models in AI when we don’t need to? (2019)

#165

Post by [deleted] »

crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

[deleted]

Re: Why are we using black box models in AI when we don’t need to? (2019)

#166

Post by agra »

crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

Was the fact that objects fall down a "black box" before Newton came up with some equations?

I don't think its fair to describe CNN's and human vision as being black boxes in the same way.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#167

Post by raverbashing »

Frost1x wrote: post #127
crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

I think it's also important to remember a lot of human models were abandoned because of the inherent bias and black-boxed nature of them (feelings, subjectivity, etc.). This is one of the reasons the scientific method blossomed: objectivity, rigor, transparency, reproducibility, etc. Black boxes can lead to bad decisions because it's difficult to question the process leading to decisions or highlight flaws in conclus…

> When a model is developed with more rigor, it can be openly critiqued.

Yes. And the models developed like this haven't solved the problems the current black box models have.

> we have models running across such massive datasets with so many degrees of freedom that we have no feasible way of isolating problems when we see or suspect certain conclusions are amiss. Instead, we throw more data at it or train the model around those edge cases.

There are ways being studied to check for sensitivity to parameters, biases, etc. But in the end, reality is difficult and there's no way of dealing with that, or "getting the right answer" every time

Re: Why are we using black box models in AI when we don’t need to? (2019)

#169

Post by z3rgl1ng »

bitL wrote: post #99

I am worried about the recent trend of "ethical AI", "interpretable models" etc. IMO it attracts people that can't come with SOTA advances in real problems and its their "easier, vague target" to hit and finish their PhDs while getting published in top journals. Those same people will likely at some point call for a strict regulation of AI using their underwhelming models to keep their advantage, faking results of th…

The paper specifically mentions this as an issue when interpreting black box models, using ProPublica’s reporting on COMPAS as an example.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#170

Post by missosoup »

Frost1x wrote: post #127
crazygringo wrote: post #49

I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…

I think it's also important to remember a lot of human models were abandoned because of the inherent bias and black-boxed nature of them (feelings, subjectivity, etc.). This is one of the reasons the scientific method blossomed: objectivity, rigor, transparency, reproducibility, etc. Black boxes can lead to bad decisions because it's difficult to question the process leading to decisions or highlight flaws in conclus…

> I think it's also important to remember a lot of human models were abandoned because of the inherent bias

The industry needs to stop misusing the term bias this way. Virtually every attempt to find this supposed human bias has failed. Latest public example was Amazon and hiring[1]

Bias is the tendency to consistently mis-classify towards a certain class or tendency to consistently to over or under-estimate.

Somehow the term has been hijacked to mean 'discriminate on factors that are politically incorrect'. You can have a super racist model that's bias free, and most models blinded to protected factors are in fact statistically biased.

It's not constructive to conflate actual bias with political incorrectness.

Operational decision making, whether AI or human or statistical, faces an inherent trilemma: it's impossible to simultaneously treat everyone the same way, to have a useful model, and to have uniformly distributed 'bias'-free outcomes. At best a model can strive to achieve two of these factors.

See: https://www.youtube.com/watch?v=Zn7oWIhFffs

[1] https://www.reuters.com/article/us-amazon-com-jobs-automatio...

Post reply on HN