Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scientists Increasingly Can’t Explain How AI Works

vice.com

Post reply on HN
Print view View original

171–180 of 231 posts

Re: Scientists Increasingly Can’t Explain How AI Works

#171

Post by outworlder »

hn_throwaway_99 wrote: post #116
kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

> My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I can't comment much on Siri or Alexa, but Google Assistant has gotten heaps better at understanding my voice and intent since it was first released about 6 years ago. It used to pretty reliably misunderstand at least 1 word per sentence (which made for an extremely frustrating ex…

Interesting. Alexa misunderstands me _a lot_. There are a few commands I use very often, but somehow they don't rank high enough.

I have one Google Assistant I got for free. I'll try it.

Most of what I want Alexa to do is interfacing with Home Assistant anyway.

Re: Scientists Increasingly Can’t Explain How AI Works

#172

Post by notahacker »

criddell wrote: post #91
dmichulke wrote: post #65

Earlier quoted context omitted.

I agree with you. However, we can't really meaningfully quantify it a-priori. So we need a sufficiently big set of accidents and accident free rides to see when it's better. In addition we can't distinguish what driver would have been better or worse than the AI in a given situation. This opens the door for all sorts of lawsuits and we will end up with legal self-driving systems in almost ideal situations only. To no…

> To note, this is as much an engineering problem as it is societal and legislative. Yeah, it definitely is. I think there are two things that should happen. In the short term, producers of self-driving cars (or systems) should be given immunity from lawsuits related to crashes provided that the NHTSA (or some other authority) can verify that the deaths per passenger mile in their cars does not exceed the rate for tr…

I'm not sure that the political 'solution' of giving special privileges to avoid immediate consequences from their actions to AV manufacturers and special responsibility for signoff in the longer term solves the engineering problem that they need each minor version update of their software to have driven tens of billions of real world miles before they remotely approach statistically significant evidence that it is less deadly than the last version, or indeed an average human.

And this is where lack of general AI or tractability is an issue; knowing that the software typically makes fewer driving errors than the last version over a typical route gives us no confidence whatsoever it doesn't handle rare edge cases marginally worse leading to a couple of extra fatalities per billion miles (making it less safe than the average driver, many of whom suffer actual legal consequences for their erratic driving killing a person even if all their other driving is 'above average'...). Humans aren't bug-free or particularly tractable either, but at least we have enough of a mental model of how they understand driving to be confident that training them on a certain road sign found in urban areas won't make them more likely to stop on a freeway.

Re: Scientists Increasingly Can’t Explain How AI Works

#173

Post by xg15 »

Xcelerate wrote: post #5

There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works. I think that assumption is totally wrong, and trying to reconcile the two is probably a distraction and a waste of time. Evolution led to human intelligence just fine on i…

> There is a somewhat widespread implicit assumption that having sufficient intelligence to develop increasingly sophisticated machine learning implies that we also have sufficient intelligence to develop an “intuitive” understanding of how it works.

I agree with you on that point, but not with the conclusion that this is in any way a good thing.

> Evolution led to human intelligence just fine on its own; why are humans not simply the catalyst in the next stage of this natural process from which something more complex arises?

I think I have the complete opposite view here. Why would "being the catalyst in the next stage of evolution" be anything desirable? Evolution has no "stages", it's just chaos and life trying to deal in the best way with the circumstances at hand. If we blew everything up in nuclear armageddon tomorrow and the only remaining life were archaeae, those would be a perfectly fine "next stage of evolution".

Similarly, we can build a would in which only robots can survive, but why would we want to?

I think, gaining more understanding how the world works is the one thing AI can really bring to the table. If it solves practical problems on the way to that, that's great, but we evidently survived so far pretty well by doing that stuff ourselves.

Re: Scientists Increasingly Can’t Explain How AI Works

#174

Post by notahacker »

sebzim4500 wrote: post #38
dmichulke wrote: post #11

Earlier quoted context omitted.

What you're saying is (kinda) my response when people ask me about self-driving cars: How does the car deal with object X on the road? For X=piano, a stack of solar panels, a tank, an airplane, a pile of stones ... No one knows and since the size of the set of X is infinite, no one can appropriately train for it either. That's why we can't have self-driving cars without a general understanding of what objects are and…

I don't see why self-driving cars should have to be perfect though. I would argue that rationally I should start using self-driving cars as soon as they are better at driving than I am. The fact there are edge cases that I handle better than the AI is not necessarily disqualifying, so long as the AI is so much better at the rest of the cases that on average the AI is safer. For example, the fact that I handle a piano…

> I would argue that rationally I should start using self-driving cars as soon as they are better at driving than I am.

How would you know?

Of course the edge cases matter, since human drivers make minor driving mistakes on a too-regular basis but fatal ones very rarely. 100ms faster reaction time is very impressive up until the point the system interprets the 10,000th bridge support it passes as an offramp and attempts to exit via it, which makes it several orders of magnitude more deadly than humans as they start using it on that road...

Re: Scientists Increasingly Can’t Explain How AI Works

#175

Post by xg15 »

FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

How would you debug such a system?

E.g. imagine somewhere in the code for your chess supercomputer is a genuine bug - some dumb typo, I don't know - which leads it to make the wrong decisions in some situations.

If you have no idea how the program arrives at its results or even if the results are correct or not, how would you be able to find that bug?

Similarly, how do you make sure the program doesn't learn garbage features or overfits on your testing environment? There are enough war stories of image classifiers that just learned some subtle lighting differences in the trainset photos of something that actually distinguished the object in question.

Or how do you prevent models learning certain features that you specifically don't want to learn, such as skin colour?

I think developing those models without trying to understand what they do has a high risk of leading to magical thinking.

Re: Scientists Increasingly Can’t Explain How AI Works

#176

Post by [deleted] »

karpierz wrote: post #88
tromp wrote: post #54

Earlier quoted context omitted.

> Obviously we know they were the result of a deep minimax-style parallel search with a certain evaluation function, and we could simulate a similar search by hand if we wanted to. You don't really need to though. The computer can justify its evaluation with a principal variation (best play by both sides) leading to a leaf position of the same value. And if a human were to wonder why at any point in this variation, s…

That's punting the problem of explaining down the line. All the PV tells you is the computer is making this move because it thinks that 20 moves down the line, you'll end up in some desirable position. It doesn't abstract or summarize any understanding. If you say, "well, what if they do Kf3 instead", all it can show is another PV. It can't tell you "knights on the rim are dim", or "this pawn needs to move now to pre…

[deleted]

Re: Scientists Increasingly Can’t Explain How AI Works

#177

Post by FartyMcFarter »

xg15 wrote: post #175
FartyMcFarter wrote: post #7

> The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. I don't think this is anything new. This was already the case 20+ years ago with chess-playing computers. In the mid-90s, Deep Blue was evaluating 200 million chess positions per second. How do you explain the resulting moves? Obviously we know they were the result of a deep minimax-styl…

How would you debug such a system? E.g. imagine somewhere in the code for your chess supercomputer is a genuine bug - some dumb typo, I don't know - which leads it to make the wrong decisions in some situations. If you have no idea how the program arrives at its results or even if the results are correct or not, how would you be able to find that bug? Similarly, how do you make sure the program doesn't learn garbage…

> E.g. imagine somewhere in the code for your chess supercomputer is a genuine bug - some dumb typo, I don't know - which leads it to make the wrong decisions in some situations.

If the evaluation function executed at the leaf nodes of the search tree contains a bug that massively over-evaluates a position, you might be able to find this by having the chess computer play against a different computer that exploits this bug. Then you'd observe the positions at the end of the principal variation as some other comments mentioned.

Even if you can do all that, that still leaves the main problem unanswered which is to explain the correct moves (or rather, moves "assumed to be correct").

Re: Scientists Increasingly Can’t Explain How AI Works

#178

Post by dinosaurdynasty »

loyalcinnamon wrote: post #152
dinosaurdynasty wrote: post #121

Earlier quoted context omitted.

One of the problems of highly capable AI ("outperforms humans empirically" in enough tasks) versus most physical systems is that it can plan and it can lie. One situation AI safety researches worry about is "AI wants X; human wants Y; AI knows human wants Y; AI does Y in training/experiments; once AI is in place to do X instead without human being able to stop AI AI does X"

Hmm, well, instead of an ai we put a human, they could do that exact thing too. So what's stoping a human? Punishment? Does this mean we need to make the ai understand punishment and/or consequences?

How many kids, from punishment, learn the lesson "don't do X while parents are watching" instead of "don't do X for moral reasons/etc"? How many children cut strict parents from their lives once they are self-sufficient? If you could always guarantee punishment and/or consequences this wouldn't be such a big deal in the first place.

Re: Scientists Increasingly Can’t Explain How AI Works

#179

Post by kybernetyk »

alcover wrote: post #162
kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

> "Siri turn off lights in the living room" No snark here : why don't you make a few steps and switch them off manually ? Saving these few steps doesn't make sense to me. You need to walk to be healthy.

Because the Hue lights aren't connected to a switch :) It's just accent lighting that's all over the room.

Re: Scientists Increasingly Can’t Explain How AI Works

#180

Post by kybernetyk »

JohnJamesRambo wrote: post #142
kybernetyk wrote: post #83

Who would have guessed that having a highly complex black box means you can't explain what's going on inside that black box... My pet theory is that this is the reason why Siri, Alexa and co. still are shit and haven't moved an inch forward since their inception. I like to play "sleep music" through Alexa when I bring my kid to bed. For his mid day nap it all works perfectly. But in the evening when I say the same ph…

Are you sure you are speaking as clearly in the evening? I don’t understand how voice command recognition could fluctuate based on AI and time of day.

Don't know ... I just have had a bottle of Scotch by that time ;)

Nope, I speak clearly. I even tried _very_ clear pronunciation (like talking to a person with bad hearing, etc). My wife tried, too. It comes back to playing German Rap and Dance music.

Maybe I have been classified by Amazon and people who are similar to me tend to listen to dance/rap in the evening. No idea. It's just a curious case of a bug. (I switched to piano music since - which works pretty well).

Post reply on HN