Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Artificial Intelligence Permanently Inscrutable? (2016)

nautil.us

Post reply on HN
Print view View original

21–30 of 38 posts

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#21

Post by ballenf »

MrLeap wrote: post #5

This canard keeps coming up. I've come up with a few explanations. 1) It's to protect the reputation of people who make their living copy/pasting examples from the SK-learn gallery 2) The authors fully understand the tools they're using, but feel there's an element of unexplainable meaning in them that I wager isn't actually there. 3) I'm ignorant and I think this is explainable when I just don't know enough to know…

It's both self-serving and job security to claim that what you do is "beyond human comprehension" or "impossible to know".

Also why do we call them "hidden" layers instead of "intermediate" layers? They're no more hidden than any other data in memory, are they?

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#22

Post by nestorD »

I think there is hope, there is a very nice paper where they automaticaly produce both heatmaps and textual explanations of the label given by the neural network (used to detect and classify fractures). It is detailled here : https://lukeoakdenrayner.wordpress.com/2018/06/05/explain-yo...

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#23

Post by cirgue »

DanielBMarkham wrote: post #2

If AI is impervious to explaining itself, then we've created something that we are unable to evaluate, aside from some kind of generic phrase like "The machine is looking at a bunch of stuff, and some of those things are associated with some other things. Sometimes. So it's guessing this is the appropriate action" I'm not going to get into the dystopian nature of that. Too easy. Instead, I have a very practical quest…

> several dozen poorly-trained AI recommendation engines optimizing for variables and sample instance that the coders might care a lot more about than any specific user? Dang. I ended up in dystopia-land anyway. Sorry about that.

This is the scary part for me: 'AI' is just a means for organizations to dodge accountability for the systems they build because decision-makers don't understand what they're looking at and the people building the systems aren't incentivized to care, or are incentivized to optimize for things that are bad for society as a whole. This is the same hazard of capitalism in general, you don't actually need an AI for this to happen because it already has in many ways. The world-eating paperclip machine won't be a computer, it's going to be us.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#24

Post by mcilai »

Are humans permanently inscrutable? In the long run, AI may be similar. Given how massively complex the world and the thing AIs learn are, it is not self evident that interpretable systems are possible. However, it's really worth trying.

More likely, we'd have to use AI systems to understand how other AI systems work. Or maybe, once AI systems get very good at language, develop a "truth serum" that will cause it to reveal its true opinions.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#25

Post by titzer »

mockingbirdy wrote: post #11
titzer wrote: post #4

Earlier quoted context omitted.

> Back when science fiction had a lot of evil computers, they always wanted to directly control humans. This is what is so hard to get through people's heads. There is not going to be any ominous background music or dramatic events when we slide into our dystopia. It's here. The machines are controlling us already. It just happens to be that they currently want what their corporate overlords want: make more money! Wh…

Do some of you work in data science or machine learning occupations? You will realize how unreal those predictions sound. It's mainly cleaning data up, getting some random correlations from plugging variables into deep learning algorithms and hoping that it doesn't break with the most silly outputs you can think of. Many scientists warn that AI is the biggest threat to humanity. I think this is wishful thinking for n…

> It's mainly cleaning data up, getting some random correlations from plugging variables into deep learning algorithms and hoping that it doesn't break with the most silly outputs you can think of.

Said one neuron to the other.

The problem is even if it--all of it, the whole internet and all the desktops and datacenters and databases and webpages and webservers and knowledge bases and ad networks and search engines and speech interfaces and image recognition and traffic cameras and phone system and surveillance systems together in one giant hairball of wires--were a superintelligent AI, could we even tell? If an entity 1000x smarter than you doesn't want you to know it's 1000x smarter than you--or that it even exists!--do you think you stand a chance of figuring it out? It is, by definition, a mathematical certainty you will lose that battle of the wits.

Ok, fine. It's an unfalsifiable theory. But we're not positing that rocks speak in really quiet whispers or unicorns exist everywhere we don't look, just that a giant information processing system can...process information--think. The problem is that the whole thing is insanely massive and so insanely complicated that not a single human alive, nor any group of humans alive, has any real perspective on just what the hell it all is. It's so massive that it doesn't fit in any of our minds. So how you can say anything definitive about what it is not? It clearly is already complex enough and capable enough of totally outclassing us all. So why shouldn't it?

So we are now faced with the reality that our massive interconnected computer system consisting of literally billions of CPUs with trillions of terabytes of RAM run by billions of lines of code, loaded with essentially the entirety of humanity's knowledge, also hooked up to all of the phone lines, cameras, PCs, phones, power plants, traffic systems, ad networks, video streams, everything...is....unknowably motivated, impossible to shutdown, and somehow not a threat? Because it is just "mainly cleaning data up"?

Yep.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#26

Post by Kalium »

ashelmire wrote: post #15
MrLeap wrote: post #5

This canard keeps coming up. I've come up with a few explanations. 1) It's to protect the reputation of people who make their living copy/pasting examples from the SK-learn gallery 2) The authors fully understand the tools they're using, but feel there's an element of unexplainable meaning in them that I wager isn't actually there. 3) I'm ignorant and I think this is explainable when I just don't know enough to know…

I’m in agreement with you here. This trope that neural networks or other forms of ML are inscrutable is ridiculous. We know exactly what mathematical function is happening at every stage and the goal of the process; we can explain what architectures have the best results, and in some cases a deep understanding of why; with close enough scrutiny, you can probably tell what an individual neuron is doing to a layman (th…

People want explanations that are equivalent to the kinds humans offer to explain their behavior. Generally in a cause-and-effect kind of way. Squishies want answers detailed enough to make them feel in control of some part of what's going on. "Because X, so we're doing Y", in theory enables the person to induce different behavior through controlling X.

There are various issues with this desire, including the wrinkle that humans are not always good at explaining their own decisions and we persistently assume we are. But that's neither here nor there. People like feeling in control, and ML tends to expose that they're not.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#27

Post by mockingbirdy »

titzer wrote: post #25
mockingbirdy wrote: post #11

Earlier quoted context omitted.

Do some of you work in data science or machine learning occupations? You will realize how unreal those predictions sound. It's mainly cleaning data up, getting some random correlations from plugging variables into deep learning algorithms and hoping that it doesn't break with the most silly outputs you can think of. Many scientists warn that AI is the biggest threat to humanity. I think this is wishful thinking for n…

> It's mainly cleaning data up, getting some random correlations from plugging variables into deep learning algorithms and hoping that it doesn't break with the most silly outputs you can think of. Said one neuron to the other. The problem is even if it--all of it, the whole internet and all the desktops and datacenters and databases and webpages and webservers and knowledge bases and ad networks and search engines a…

I was talking about AI.

I agree that our interconnected world is nearly impossible to thoroughly understand. But that is true for human societies, too. Anthropomorphizing the internet ("If an entity 1000x smarter than you doesn't want you to know it's 1000x smarter than you--or that it even exists!--do you think you stand a chance of figuring it out?") is good to write some sci-fi novels, but in reality most systems are pretty boring.

The traffic light system and a bunch of routers won't work together to destroy humanity. It's more like a mesh network than an entity that is 1000x smarter - it's more like 1000x devices with 0.01 intelligence and you would need a team of talented people who harness this computational power to build a distributed intelligent system.

It's more likely that a hacker who has specific motivations will hack the traffic light system (or use AI helpers to do so) and use it for evil purposes and car accidents.

If you believe that there might be an entity which exists outside of our perception and has enormous intelligence, it's like a human-made god - a homunculus. It's basically the same as believing in a god. I have nothing against it, but this is entirely hypothetical.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#28

Post by titzer »

mockingbirdy wrote: post #27
titzer wrote: post #25

Earlier quoted context omitted.

> It's mainly cleaning data up, getting some random correlations from plugging variables into deep learning algorithms and hoping that it doesn't break with the most silly outputs you can think of. Said one neuron to the other. The problem is even if it--all of it, the whole internet and all the desktops and datacenters and databases and webpages and webservers and knowledge bases and ad networks and search engines a…

I was talking about AI. I agree that our interconnected world is nearly impossible to thoroughly understand. But that is true for human societies, too. Anthropomorphizing the internet ("If an entity 1000x smarter than you doesn't want you to know it's 1000x smarter than you--or that it even exists!--do you think you stand a chance of figuring it out?") is good to write some sci-fi novels, but in reality most systems…

> If you believe that there might be an entity which exists outside of our perception and has enormous intelligence, it's like a human-made god - a homunculus. It's basically the same as believing in a god. I have nothing against it, but this is entirely hypothetical.

I don't really believe in a singular entity in there, other than a convenient abstraction for now. Rather there are many different processes that are all optimizing this and that. The sum is essentially intelligent--it processes information, makes decisions. Those decisions help it grow and spread and become more efficient. These optimizers don't speak human languages. They don't necessarily think in abstract concepts like us. Maybe they are just tables of numbers optimizing themselves, maybe they are like standing waves in a sea of numbers, like our thoughts are standing wave patterns in the electrical activity of our brains. It's hard to express because our words fail us.

Problem is, the whole damn thing is so complicated that we cannot know anything really definitive about it. The substrate is there. The soil is fertile. The capability is there. And so many forces are aligned to feed more and more resources into the whole thing. Across the board, essentially the only thing we can come up with is to upload all of our data into this massive cloud thing and use computation to do something with it. That is just like injecting raw materials and energy into the fabric. And those who do this successfully are rewarded financially. So all forces point to more of it.

Whatever that huge mesh is, it's extremely successful at swallowing the world and convincing us to feed it more and more CPU, RAM, disk, data. It keeps convincing us to hook it into more things to sense and control--more cameras, phones, drones, markets, even weapons.

Thus in some abstract sense, it is already here. And we can't get rid of it. Can't shut it off. We're dependent, and it's parasitic--for now. Yet every incentive we have right now is to make it more autonomous, more robust, more powerful, and give it more and more control. It works best when it is more centralized.

Teeth or not, conscious or not, this computer thingy is gobbling everything. Do we control it? Clearly not. Can we predict what it will do? Not even close. Is it good for us? Who knows? But it will win.

Well played, computer thingy.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#29

Post by ashelmire »

gldalmaso wrote: post #18
ashelmire wrote: post #15

Earlier quoted context omitted.

I’m in agreement with you here. This trope that neural networks or other forms of ML are inscrutable is ridiculous. We know exactly what mathematical function is happening at every stage and the goal of the process; we can explain what architectures have the best results, and in some cases a deep understanding of why; with close enough scrutiny, you can probably tell what an individual neuron is doing to a layman (th…

A deeper explanation is often desired when you decide to take your solution to the real world and impact peoples lives. I'm sure I'm not alone in the fear of living in a (not so far) future dystopia where our driver's licences get cancelled because of our driving behavior caught on traffic cameras. "What did I do that caused it to get cancelled?", "We don't know, better luck next time." "Your test got flagged for che…

Those explanations are awful. The reasons are usually pretty simple and sound much more reasonable. The answer is not “I don’t know” in any of those cases - it’s going to be pretty clear that they share some similarity to other cases of theft, cheating, etc, and this will probably be obvious on examination, “our system flags people who are wearing masks” or “these 30 seconds of sounds are almost exactly the intro to song X”. Human review is also useful, but humans make plenty of errors as well (and the response is often “I don’t know why”!).

I’d say these errors are not due to the inscrutable nature of ML but bad communication.

Re: Is Artificial Intelligence Permanently Inscrutable? (2016)

#30

Post by taneq »

DanielBMarkham wrote: post #9
taneq wrote: post #8

Earlier quoted context omitted.

> Give them everything they want. You measure your lifespan in millions of years. Humans? They're like gnats. What if I want to become an immortal, all-powerful machine? :P

How do I know you're not already? :) The thing is, we're most likely going to get immortal machines far ahead of smart ones. And while machine entities might already be all powerful, individually it's just millions of little pieces scattered all over the place doing dumb, trivial things. I liked it a lot better when we had Landrew (STTOS reference: Return of the Archons) That was something Joe Blow could get his head…

Hah, if only. :P

I think Ian M. Banks had the right idea when he suggested that in the Culture, it was possible (if very rare) for a human to augment themselves that far, but that those that did so were no longer considered human and so it wasn't really an issue.

Post reply on HN