Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dawn of AI: How to ensure the promise outweighs the perils

economist.com

Post reply on HN
Print view View original

31–40 of 75 posts

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#31

Post by DanAndersen »

return0 wrote: post #24
coldtea wrote: post #10

Earlier quoted context omitted.

> Now we are scared of something we haven't even created I don't see the contradiction or strangeness in that. That's the whole idea behind "caution" -- to be wary of the potential outcome BEFORE it happens. > How could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it For one we are quite good consumers it. How about they get rid of us, so they…

> Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc. I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the…

I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess.

But I feel like this particular argument is (please correct me if I'm wrong) saying something like "Well of course we wouldn't let them go crazy! That'd be ridiculous!" I agree -- a sensible solution would be to have good design and introduce safeguards. That's the point of people like Bostrom when they're advocating for this, because before they started writing about this, it seems like a lot of people weren't even thinking it could be a problem. How many systems will be built without good design, because they were made to solve a short-term problem. Will the finance bots doing transactions at the microsecond scale in order to maximize the assets of some particular company have been built to not let them go haywire?

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#32

Post by arethuza »

loup-vaillant wrote: post #12
return0 wrote: post #8

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

The machine does not hate you, nor does it loves you. But you are made of atoms, that it could use for something else. --- If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally…

Or we might end up with something a bit like the Culture - which is pretty much the best case scenario for humans co-existing with god-like AIs:

http://en.wikipedia.org/wiki/The_Culture

Vinge and Stross both hint at the worst cases - and they would be pretty bad, simply having using atoms re-purposed might not look that bad by comparison:

"There is life eternal within the eater of souls. Nobody is ever forgotten or allowed to rest in peace. They populate the simulation spaces of its mind, exploring all the possible alternative endings to their life."

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#33

Post by return0 »

SCHiM wrote: post #29
return0 wrote: post #14

Earlier quoted context omitted.

i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?

It 'wants' what we programmed it to want. It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do. The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the…

if (nearby() == human) {dontTouch(); abort() ; }

There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#34

Post by SCHiM »

coldtea wrote: post #13
rndn wrote: post #7

Earlier quoted context omitted.

I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.

> It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. That's at best a peripheral detail to the parallel the parent tries to draw. Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.

Except they're not really superhumanly intelligent in the sense that AGI is superhumanly intelligent.

Sure a lawyers office is probably able to to tackle more cases that a single lawyer. A big multinational with momentum probably a lot more power than a single human.

But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter?

I think not, the point here being that while corporations are super-human, they are not in the same league as true AGI. And that containment of corporations/governments is not a good place to learn about the containment of true AGI.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#35

Post by DanAndersen »

justinpaulson wrote: post #26

Posted this on HN a couple days ago but I don't think anyone saw it: http://justinpaulson.com/posts/0bb97847af7e2123f6173361 tl;dr: Robots and AI will not be built as separate entities, but will instead be incorporated into our own bodies. They will not be the superhumans, we will be.

That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario.

My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal.

The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#37

Post by coldtea »

TeMPOraL wrote: post #18
coldtea wrote: post #15

Earlier quoted context omitted.

> their intelligence is usually not the identifying characteristic Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).

I didn't mean intelligence as moral characteristic - the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things. As alien minds, they're not very smart - just smart enough to be dangerous.

>the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things

Isn't that true for people too, that can otherwise be extremely intelligent? From being terrible at social life, to sabotaging their careers and so on?

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#38

Post by TeMPOraL »

return0 wrote: post #33
SCHiM wrote: post #29

Earlier quoted context omitted.

It 'wants' what we programmed it to want. It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do. The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the…

if (nearby() == human) {dontTouch(); abort() ; } There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?

Define nearby(). Because if the AI starts eating chunks of our planet we will get hurt as well.

INB4 Asimov's Three Laws of Robotics don't work. Actually, a lot of Asimov's writing is concerned with exploring their corner cases.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#39

Post by coldtea »

SCHiM wrote: post #34
coldtea wrote: post #13

Earlier quoted context omitted.

> It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. That's at best a peripheral detail to the parallel the parent tries to draw. Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.

Except they're not really superhumanly intelligent in the sense that AGI is superhumanly intelligent. Sure a lawyers office is probably able to to tackle more cases that a single lawyer. A big multinational with momentum probably a lot more power than a single human. But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstei…

>But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter?

Maybe not, but it sure as hell be more than "1 Einstein unit" smarter. Whether that's 1.2 or 5 depends on various elements, but it surely would beat a single Einstein, if not by anything else, by mere ability to break down a problem to study in parts and assign it to each Einstein.

I mean, that's the reason we have research teams with multiple researchers on the same team.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#40

Post by SCHiM »

return0 wrote: post #33
SCHiM wrote: post #29

Earlier quoted context omitted.

It 'wants' what we programmed it to want. It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do. The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the…

if (nearby() == human) {dontTouch(); abort() ; } There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?

What's a human then?

Is a mentally retarded person a human? An embryo? A person who's lost his/her legs? A person who has education?

auto HumanObject = nearby();

if(Humanobject.IsPregnantWoman()) { PoisionEmbryoButNotHuman(); } else { abort(); }

Post reply on HN