I feel like we're ignoring the obvious answer: because most people in AI right now are overwhelmed by the necessary mathematical background and comfort necessary to produce first-principles generative innovative algorithms and models, so all they're left with is following the latest trend, which is locked on deep learning right now.
Why are we using black box models in AI when we don’t need to? (2019)
141–150 of 200 posts
Re: Why are we using black box models in AI when we don’t need to? (2019)
#142I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…
I think it's also important to remember a lot of human models were abandoned because of the inherent bias and black-boxed nature of them (feelings, subjectivity, etc.). This is one of the reasons the scientific method blossomed: objectivity, rigor, transparency, reproducibility, etc. Black boxes can lead to bad decisions because it's difficult to question the process leading to decisions or highlight flaws in conclus…
That is the fascinating thing about consciousness.
How do we "know" we see red and not green or vice versa? How do we "know" we are feeling heat or coldness? We just do. We feel different things in different, yet recognizable ways.
And these "feelings" are multi-dimensional, feeling heat and feeling color are not on the same axis. Red and Green seems to be on different points of the same axis and hot and cold are points on some different axis, other dimension.
How many different dimensions of "feelings" are there? Note they are not the same as different senses. There can be many different feelings associated with things we see and recognize that way: colors, and shapes and lightness vs. darkness etc.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#143Ultimately, a lot comes down to the accessibility of tools and know how. The more widespread and general a technique or tool, the more widespread and general it becomes... because people already have it and know how to apply it.
OTOH, these authors (and old school heavyweights like David Ferucci) are not wrong. Interpretability, explainability & interoperability with human intelligence is not something to just give up on.
I like "challenges" as ways of exploring these areas. Good luck to the authors. Female team, btw.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#144Earlier quoted context omitted.
I think it's also important to remember a lot of human models were abandoned because of the inherent bias and black-boxed nature of them (feelings, subjectivity, etc.). This is one of the reasons the scientific method blossomed: objectivity, rigor, transparency, reproducibility, etc. Black boxes can lead to bad decisions because it's difficult to question the process leading to decisions or highlight flaws in conclus…
That is the fascinating thing about consciousness. How do we "know" we see red and not green or vice versa? How do we "know" we are feeling heat or coldness? We just do. We feel different things in different, yet recognizable ways. And these "feelings" are multi-dimensional, feeling heat and feeling color are not on the same axis. Red and Green seems to be on different points of the same axis and hot and cold are poi…
Physical sensation seems simple tbf. Any robot can sense temperature or color gradient.
It's the decision making that's complicated.
To make a decision on whether it would like chocolate ice cream or not (before trying it) a brain will take all the data gathered over time, cross reference it with related data (does chocolate taste good alone? \ did I like it? \ It looked rather inedible. \ was it high cocoa content or not? \ Chocolate yoghurt seems similar, but I did not like it. \ I do like cold, but only when it's hot outside. \ My friend says chocolate ice cream tastes good. \ My other friend says it tastes bad. \ trust or discard opinions? and much more).
All within minutes, seconds even.
An AGI would need an absolutely massive database of knowledge similar to the one acquired by a human over the first 20-30 years of their life, if it's to be truly general.
It would also need bias correction, and maybe 2-5 other AGIs to form a consensus before making any decision properly.
And even that would not be enough to fit in with humans - if you want an AGI to make the objectively right decisions, it will have to ignore many human feelings/biases.
If you want it to fit in with people, it will have to make "mistakes", i.e. ineffective decisions.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#145I think it's important to note that human pattern recognition is basically black-box as well. We can't "explain" how we recognize a particular person's face, for example. Robust pattern recognition based on 100+'s of factors is just inherently black box. Even when people make decisions, they're generally a weighted combination of a bunch of "feelings". We attempt to explain a kind of simple, logical rationalization a…
One more benefit: an interpretable model could at least give some hint when the model is not working as intended (e.g. debugging).
Re: Why are we using black box models in AI when we don’t need to? (2019)
#146When using ML models to deal with natural science such as physics, working well+being interpretable actually implies new discovery. Working well only, on the other hand, cannot win you a Nobel Prize.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#147Earlier quoted context omitted.
I agree in spirit, but a lot of human behaviors, derived from decisions, don’t have consequences. Did Nestle face consequences for their decisions to promote formula, etc?
Let's not confuse accountability of decision making entity with consequences faced by the accountability. I was referring to the former, not the latter. The former is a case for law, the latter is a subject of law enforcement. In the case of Nestlé, you have a clear assigned accountable entity. We don't have that with algorithmic decision making (yet) where organizations wash off their hands saying "Oops! It was an a…
> We don't have that with algorithmic decision making (yet) where organizations wash off their hands saying "Oops! It was an algorithm, we didn't do anything!"
That's what individuals within companies do all the time.
You're onto something, form a limited liability company for your AIs and hire them as contractors. There, a clear assigned accountable entity :)
Re: Why are we using black box models in AI when we don’t need to? (2019)
#148Earlier quoted context omitted.
Their model didn't win. IBM's model won, based on actual metrics around useful insights.
The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good. IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are. And the evaluation was completely subjective, so there's n…
Reading "subjective" to mean "nonexistent" is a potentially big mistake.
Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal.
In this area, we don't exactly know how to define narrow, objective goals & subsequent criteria. We can definee goalposts, but not goals. These are guesses at useful markers of success, useful to the larger goal of useful/novel ai.
Subjective goals have their own (massive problems), but since we can't objectively define the goals of ai research... we need to fall back on human subjectivity to define our subgoals.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#149Earlier quoted context omitted.
The IBM team got $5,000 and the second place/honorable mention NYU got $2,000. So going by prize amounts, the Duke model was still pretty good. IBM turned the model/paper into a toolkit: https://www.ibm.com/blogs/research/2019/08/ai-explainability... Their model seems to be a variant of decision trees that has a knob controlling how complicated the trees are. And the evaluation was completely subjective, so there's n…
Reading "subjective" to mean "nonexistent" is a potentially big mistake. Objectivity is more accurate, sure. The winner of an objective contest is always objectively better against objective criteria. But, objective criteria are generally narrow. This works well if one is either (a) seeking fundamental principles like in physics or (b) the narrow objective criteria is the definite goal. In this area, we don't exactly…
All objective criteria are chosen, directly or indirectly, based on subjective criteria.
Re: Why are we using black box models in AI when we don’t need to? (2019)
#150The main advantage of a blackbox ML solution is shorter development time to a useful performance level. Creating a transparent, explainable solution typically takes more time, more work, and a higher level of expertise to get to the same performance level. If the problem is complicated and the cost of a mistake is low, then your best approach today is likely to be blackbox. If the cost of a mistake is high, you shoul…
This isn't how it works, at all. You're going to get better results with 1,000 parameters than with 100 explainable ones. There's a limit to how much humans can understand. We use machine learning to surpass that limit
> You're going to get better results with 1,000 parameters than with 100 explainable ones.
This is not always true.
Many problems are modeled very well with less than 100 parameters and adding more is of little-to-no benefit.
Many problems are naturally hierarchical such that simple models can be combined to yield a large number of explainable parameters. If done well, this can result in a high-performing solution. Admittedly, this is usually harder than just applying a blackbox.
In critical applications, an explainable model with benign failure modes (even if it has worse overall performance), can be far preferable to a blackbox with wildly unpredictable failure modes. From a utility standpoint, the explainable results are better.
> There's a limit to how much humans can understand. We use machine learning to surpass that limit
We can also work to improve our ability to discover and understand. I think that holds far more promise than improving our ability to do things we don't understand.