Why are we using black box models in AI when we don’t need to? (2019) • Hacker News Board

Re: Why are we using black box models in AI when we don’t need to? (2019)

#31

Post by psv1 » Sat, Jan 04, 2020, 10:58 PM UTC

shakna wrote: post #26

psv1 wrote: post #19

Earlier quoted context omitted.

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Shouldn't this fall into the "trust by verify" category of things? Whilst your model may not be able to explain to you today why it can operate on future data, if it can explain to you why it works on any data you've already given it, it makes it easier to explain behaviour. You have to trust it on new data. You can always verify why it made the choices it did on old data. That seems like something incredibly helpful…

> You can always verify why it made the choices it did on old data.

This makes no sense. Your model typically isn't asked to make predictions on "old data" because it was trained on it. And understanding why it made certain prediction isn't any different on "new" vs old data.

You can look into bias-variance tradeoff, train-test splitting and cross validation to get a better picture of this.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#32

Post by jayd16 » Sat, Jan 04, 2020, 11:00 PM UTC

Can you train a white box solution with only a tagged data set? It seems obvious to me why we have black box solutions even if we could have spent man hours engineering an algorithm.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#33

Post by sgt101 » Sat, Jan 04, 2020, 11:00 PM UTC

jayd16 wrote: post #23

anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

Indeed. It would be nice to read an article about NN introspection/debuggers. I don't see why you couldn't use an introspected ML model as the basis for a behavior tree in some cases.

It's an old idea (22 years at least), the black box method can be used to remove noise and find a none bayes/compression/simplest info theory model, then the model is used to generate clean training for a decision tree or association rules.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#34

Post by ummonk » Sat, Jan 04, 2020, 11:02 PM UTC

I think there is a lot of scope for interpretation in black box models. E.g. when doing transfer learning with a frozen pretrained model it should explain the new classifications in terms of pretrained classes.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#35

Post by bllguo » Sat, Jan 04, 2020, 11:05 PM UTC

zevets wrote: post #7

I think this argument fails to understand that purpose of black-box models is to be impossible to understand, as it shields the operator from pre-emptive accountability. Then, since they are supposedly the 'only way' to provide whatever function the market is 'demanding', then clearly we must abandon any notion of accountability, as this need 'must' be met. This logic is perhaps most evident in google/facebooks conte…

I don't think it's worthwhile assuming this level of malice.. Ultimately we choose models that are the most performant. Interpretability is just a nice bonus. I can assure you that I am held accountable for my model performance regardless of whether I can explain its decisions satisfactorily. All the business cares about is whether we're burning or making money.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#36

Post by psv1 » Sat, Jan 04, 2020, 11:06 PM UTC

rubyn00bie wrote: post #28

psv1 wrote: post #19

Earlier quoted context omitted.

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

I'm confused by what you're saying... by data that hasn't seen before, do you mean new variables which need to be hypothesis tested? Or just a larger sample? I also don't see why you can't create a "function" hash or otherwise, that works for 1000s of dimensions. The model itself, if it was perfect would be an equation... the number of dimensions really doesn't matter that much. Sorry, I'm just real confused by your…

By data it hasn't seen before I mean a sample which doesn't exist in the training set. Maybe I'm not understanding something in the parent's argument.

For example, you train an image recognition model to tell cats and dogs apart using images from the internet, then you take out your phone, snap an image of your dog and give it to your algorithm to determine the species. When preprocessed this picture is equivalent to one row in a table with thousands of columns and this specific combination of pixel values doesn't exist anywhere else. Where is the hash function looking and for what in that case?

Re: Why are we using black box models in AI when we don’t need to? (2019)

#37

Post by Gaelan » Sat, Jan 04, 2020, 11:07 PM UTC

OrgNet wrote: post #22

why include 2019 in the title of this post? was it automatically added (is that a bug)?

There's a (human-enforced) HN rule to do that for old articles, so they're not mistaken for news. I'm not sure what the standard for "old" is, but it isn't 2019 anymore, so I guess it's technically correct.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#38

Post by 1e-9 » Sat, Jan 04, 2020, 11:08 PM UTC

The main advantage of a blackbox ML solution is shorter development time to a useful performance level. Creating a transparent, explainable solution typically takes more time, more work, and a higher level of expertise to get to the same performance level. If the problem is complicated and the cost of a mistake is low, then your best approach today is likely to be blackbox. If the cost of a mistake is high, you should not even be considering a blackbox approach (at least not without transparent safeguards). There is a large gray area between these two extremes that requires good engineering judgement to choose well.

I think we definitely need more R&D dedicated to creating easier-to-use, lower-cost approaches to transparent, explainable ML. There is way too much effort devoted to blackbox R&D today. Ultimately, transparent, explainable ML should almost always beat blackbox ML due to a better ability to find and resolve hidden problems, such as dataset biases, that may be holding back performance.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#39

Post by mkolodny » Sat, Jan 04, 2020, 11:12 PM UTC

rubyn00bie wrote: post #28

psv1 wrote: post #19

Earlier quoted context omitted.

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

I'm confused by what you're saying... by data that hasn't seen before, do you mean new variables which need to be hypothesis tested? Or just a larger sample? I also don't see why you can't create a "function" hash or otherwise, that works for 1000s of dimensions. The model itself, if it was perfect would be an equation... the number of dimensions really doesn't matter that much. Sorry, I'm just real confused by your…

Say you're building a self-driving car. You need to take an image of what the car sees, and figure out whether/where there is a pedestrian/car/bicycle/etc in the image. So your input/variable is the image itself.

Almost every image the car sees is going to be different. I'd guess that's why GP said that it doesn't make sense to use a hashing function - there's little value in mapping inputs to results, because your input images are pretty much always going to be different. So pretty much every time you look up an image in your hashmap, you wouldn't find a match.

That's the point of ML models - find patterns in data so that when you see a new example, you can predict what you're seeing based on what you've seen in the past.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#40

Post by m0zg » Sat, Jan 04, 2020, 11:13 PM UTC

Human cognition itself is a black box model. We routinely make snap decisions first and then justify them after the fact. Anything that requires serious consideration is darn near insurmountable for a human mind in the presence of incomplete information, emotions, and conflicting motivations.

So while I'd like neural nets to be more interpretable, to me it'd take a distant second place. The first would be to get models that actually work better than humans for practical tasks, even under limited circumstances.