Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are we using black box models in AI when we don’t need to? (2019)

hdsr.mitpress.mit.edu

Post reply on HN
Print view View original

21–30 of 200 posts

Re: Why are we using black box models in AI when we don’t need to? (2019)

#21

Post by BoiledCabbage »

taneq wrote: post #14
sigmaprimus wrote: post #3

Earlier quoted context omitted.

Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. As much as I hate regulations I think this may be a good place to legislate in order to protect those wrongly affected. It makes me think about the mess the derivatives ma…

It’s not like your bank manager used to give you a phone call to explain in depth why your loan was declined. Most of these situations were already impenetrable from a human standpoint. Humans are far more ‘black box’ than neutral nets, we just happen to be able to construct plausible explanations in parallel.

No, they were impenetrable from the customer perspective, but either the human making the decision or the software being used can answer. With a black box model, nobody can answer.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#23

Post by jayd16 »

anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

Indeed. It would be nice to read an article about NN introspection/debuggers. I don't see why you couldn't use an introspected ML model as the basis for a behavior tree in some cases.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#24

Post by otnobot »

psv1 wrote: post #19
anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Yes, the model should work with any "new or old" data and also any "type".

Data from any products, services, commodities or measurable force / energy.

The model will need "adjustments" from time to time but the core will stay intact indefinitly.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#25

Post by fallous »

psv1 wrote: post #19
anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Unless there is a self-modifying feedback loop you can indeed replace any ML model with a hash function. The problem is one of size rather than any fundamental limitation of a look-up table.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#26

Post by shakna »

psv1 wrote: post #19
anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Shouldn't this fall into the "trust by verify" category of things?

Whilst your model may not be able to explain to you today why it can operate on future data, if it can explain to you why it works on any data you've already given it, it makes it easier to explain behaviour.

You have to trust it on new data.

You can always verify why it made the choices it did on old data.

That seems like something incredibly helpful, to me.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#27

Post by jayd16 »

psv1 wrote: post #19
anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Uh, yeah you can. Hash functions handle data its never seen before.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#28

Post by rubyn00bie »

psv1 wrote: post #19
anfilt wrote: post #15

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough." For-instance when a model starts being over fitted, o…

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

I'm confused by what you're saying... by data that hasn't seen before, do you mean new variables which need to be hypothesis tested? Or just a larger sample?

I also don't see why you can't create a "function" hash or otherwise, that works for 1000s of dimensions. The model itself, if it was perfect would be an equation... the number of dimensions really doesn't matter that much.

Sorry, I'm just real confused by your statement.

Re: Why are we using black box models in AI when we don’t need to? (2019)

#29

Post by _jal »

taneq wrote: post #14
sigmaprimus wrote: post #3

Earlier quoted context omitted.

Ofcourse the scary thing is when decisions are being made that will affect a persons life but will never make it to the courts. Like the example of predicting loan defaults or parole releases, those decisions are made with little to no explination. As much as I hate regulations I think this may be a good place to legislate in order to protect those wrongly affected. It makes me think about the mess the derivatives ma…

It’s not like your bank manager used to give you a phone call to explain in depth why your loan was declined. Most of these situations were already impenetrable from a human standpoint. Humans are far more ‘black box’ than neutral nets, we just happen to be able to construct plausible explanations in parallel.

Bank managers, however, can at least theoretically be hauled in to court and questioned.
Post reply on HN