Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

Post reply on HN
Print view View original

121–130 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#121

Post by dinosaurdynasty »

kriro wrote: post #40

I don't see a problem with not being able to explain how something works as long as it's not failing empirically. Scientists also cannot explain how humans work in detail and yet humans are "allowed" to do many things and make many decisions that cannot be fully explained. The question for me is...if I have an AI system that outperforms humans empirically, why do I need to understand how it works to use it? In fact i…

One of the problems of highly capable AI ("outperforms humans empirically" in enough tasks) versus most physical systems is that it can plan and it can lie. One situation AI safety researches worry about is "AI wants X; human wants Y; AI knows human wants Y; AI does Y in training/experiments; once AI is in place to do X instead without human being able to stop AI AI does X"

Re: Scientists Increasingly Can’t Explain How AI Works

#122

Post by P5fRxh5kUvp2th »

kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

This is why I don't even like the "AI assisted autocomplete".

I want consistency, not fanciness. consistency breeds productivity. autocompleting different things from 1 file to the next is not helpful.

Re: Scientists Increasingly Can’t Explain How AI Works

#123

Post by diputsmonro »

zmgsabst wrote: post #112
diputsmonro wrote: post #75

Earlier quoted context omitted.

So what you're really saying is, it's perfectly okay for AI to have negative preconceptions and concrete impacts to certain groups of people, as long as it agrees with you on which groups are bad. How would you feel about AI models that classify all modern white people as racists based on overwhelming historical data about slavery, the KKK, etc.? Hey man, data doesn't lie.

Isn’t the media doing exactly that? They fixate on white racists while ignoring (eg) the Christmas parade attack — leading to society developing a bias in their beliefs about racism. I always take these criticism of AI as confession-through-projection of the misdeeds people have engaged in already.

Your deflection about the media is entirely unrelated to AI, but it only backs up my argument. If giant media corporations are biased, why can't AI researchers be?

Every institution has biases, and blindly trusting a dataset from e.g. a police department is just as bad as blindly trusting data from CNN or Fox News. The collection, aggregation, and sharing of data is a biased operation. If a political or for-profit entity gives you data, it's going to be data that supports their agenda.

It takes active, conscious effort to combat that bias, in the same way that it takes active, conscious effort to not just get all your news from one source. The concern (which has been demonstrated) is that that isn't being done

Re: Scientists Increasingly Can’t Explain How AI Works

#124

Post by visarga »

diputsmonro wrote: post #69
fallingknife wrote: post #10

Earlier quoted context omitted.

What people are really saying, but never will admit, is that they want AI to mirror their own biases, not to have none. That's the reason they are "cleaning" the input data.

I completely disagree. You're right that everyone has biases - including the authors of the AI who will therefore choose a biased training dataset. When the tool they are constructing has unparalleled power, and is very likely to negatively impact the lives of those who the authors are biased against (consciously or not), it's entirely fair for those impacted groups to be concerned. Every dataset has a skew to it, wh…

These imbalances in the training data are there because the world generates data in an imbalanced way. Sometimes you can correct for it, other times it's impossible. You simply can't find good data for all your cases. For example in medical diagnosing or in self driving.

Re: Scientists Increasingly Can’t Explain How AI Works

#125

Post by saiya-jin »

visarga wrote: post #107
kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

> you can't explain what's going on inside that black box Do you have a better solution for speech recognition? We all know how well speech recognition works in reality, and we know language models can accomplish more complex tasks than setting your music and lights. These models are not state of the art, they are cheap versions for scaling up to millions of users. It's sad but we rarely get to see SOTA in a product.

OP just described ridiculously bad behavior that consistently changes over time, not randomly random results. That's more a case of bad engineering (ie light sensor interfering with microphone data feed) or some stupid management decision then 'AI is complicated but we don't have anything better'

Re: Scientists Increasingly Can’t Explain How AI Works

#126

Post by jhbadger »

alexpetralia wrote: post #29
mdp2021 wrote: post #17

Earlier quoted context omitted.

> anything new It is nothing new. It is called (at least) "the problem of transparency". The chess context is probably not the best, because many systems allow a lengthy complex explanation of the response (I cannot remember now the exact workings of Deep Blue - it has been a while last time I met the full info). It is a real problem in general, because we may not just want responses but we may want " to learn someth…

I suppose this is why it is called "machine learning" and not "human learning"!

But some forms of machine learning actually are interpretable, which makes them useful in scientific investigation, which doesn't just want a black box. A good example of interpretation machine learning is decision trees -- given a series of variables and outcomes you can find out which variables are important to the outcomes and in which combinations.

Re: Scientists Increasingly Can’t Explain How AI Works

#127

Post by jmagoon »

mdp2021 wrote: post #87

Now for a small elephant which entered the room: the article recites that it would be > worth mentioning that the white-box / black-box terminology is in itself part of a long history of racially coded terms in science; researchers have pushed to change "blacklist" to "blocklist," for example Very plainly, the idea of "black-box" comes from the clear, basic and original, notion and experience that "in the dark, you c…

The real issues around data bias get completely crippled by the article writers' need to make everything about culture wars. Many computer vision systems have literally nothing to do with humans, e.g. one of the most common use cases in CV at this point is defect detection in manufacturing.

The data bias issue is about having way more samples of class X than of class Y, or class Y sharing an unknown but correlated feature (medical images with labels have this problem) that the developer doesn't identify, or any other number of "biases", like all the images being too bright, or taken with a camera that isn't identical to the one that's going to get deployed in production, etc., etc.

There are real issues that can be fixed / engineered / understood in terms of producing reliable output, and xAI absolutely helps with that! But for whatever reason journalists don't seem to understand that these systems need normal engineering safeguards, like any automated system, and bring it back to one poorly engineered model to talk about Big Bad Racist AI always denying loans based on race.

Re: Scientists Increasingly Can’t Explain How AI Works

#128

Post by thethirdone »

tromp wrote: post #101
karpierz wrote: post #88

Earlier quoted context omitted.

That's punting the problem of explaining down the line. All the PV tells you is the computer is making this move because it thinks that 20 moves down the line, you'll end up in some desirable position. It doesn't abstract or summarize any understanding. If you say, "well, what if they do Kf3 instead", all it can show is another PV. It can't tell you "knights on the rim are dim", or "this pawn needs to move now to pre…

It doesn't provide a clear explanation in those terms, and in some fraction of positions, there is no simple such explanation. But I believe that in a decent fraction of cases, a human chess player probing PVs will gain a reasonable understanding of the winning or drawing strategy involved, and that such interactive PV probing is often a better form of explanation than a concise non-interactive textual one could offe…

Interactive PV probing is just playing chess against the AI to better understand the position. That is literally the interface for which people built chess AIs.

Unfortunately that misses a lot of what makes grandmasters worth listening to. GMs will say things like "Kh6 almost works, but fails to ..." which is not something you can get out of a computer currently.

Computers currently fail to explain any moves which are good ideas, but are not the best move now. Humans normally describe these moves as "resources" which are good to keep in mind for the future or to play later when the board cools down.

Re: Scientists Increasingly Can’t Explain How AI Works

#130

Post by shadowgovt »

fallingknife wrote: post #61
mchaver wrote: post #57

Earlier quoted context omitted.

I always thought it was the opposite. That we want AI to have no biases, but we have trouble avoiding it. Choosing data and cleaning is where we insert our bias.

What people want is for the models to show no differences between certain groups of people. That is the standard they use for "no bias." But that itself is bias, so really they just want the model output to conform to their bias that there isn't any difference.

Correct. As a tool serving humans, we want the tool to reflect the decisions humans should make given the same broad swath of data.

"Unbiased machine learning models" is basically a nonsense idea; a machine learning engine is a discrimination / classification tool, and its entire point is to become biased based on inputs so that on future inputs, its outputs tilt over in a desired fashion instead of just being uncorrelated noise. Making those biases have the desired shape is the art and science of the process.

Post reply on HN