Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI's Researchers: Protecting Against AI’s Existential Threat

wsj.com

Post reply on HN
Print view View original

41–50 of 62 posts

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#41

Post by jerf »

staunch wrote: post #36
jerf wrote: post #27

Earlier quoted context omitted.

By what mechanism do you assume that an understanding of human consciousness is going to lead to anything like ethical behavior, given the enormous counterexamples given by our own behavior? Plenty of humans kill other humans fully aware that they are humans.

I assume that an understanding will lead to a capability, because that's the history of humanity's progress. First we understand a phenomenon, then we master it. If humans ourselves understood human consciousness, we could probably replicate it in software today. This might end up being one of the ways a super intelligence comes into being.

"I assume that an understanding will lead to a capability, because that's the history of humanity's progress. First we understand a phenomenon, then we master it."

That's a category error, in that it doesn't even answer the question I asked. By what mechanism does an understanding of human consciousness by an AI necessarily and inevitably lead to moral behavior? As opposed to, for instance, using that understanding to accomplish its own immoral (or even merely non-moral) goals? Especially in light of the fact that it is very unlikely that you consider all humans to be moral, and that all such humans all provide existence proofs of intelligences that understand something, but for which that understanding did not necessarily and inevitably lead to moral behavior.

You appear to be proposing the certain (and rather ill-defined) existence of mechanisms that lead to morality that don't even work in humans.

On a similar note, by what mechanisms does an understanding of human consciousness by an AI necessarily and inevitably lead to moral behavior, when that AI is owned by an evil human? What mechanism, precisely, do you expect to save us when President Trump Jr., or Clinton II, or whatever other human you currently believe to be extremely evil, orders the AI to work out the most effective plan to exterminate whoever they consider their enemies this week? Which, since I asked you to fill in whom you consider evil, includes you in the target list. What, exactly, is going to save you in that case? By what mechanism is the AI going to go "Oh, no, not staunch, I can't kill staunch, that would be immoral."

If you don't have an answer to that... and you don't... some caution may be warranted in AI research.

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#42

Post by goatlover »

staunch wrote: post #2

The real goal should be making a super AI that is worthy of being humanity's collective offspring. The bias in favor of our bags-of-mostly-water is provincial. All we really need is quality genetic engineering of our super AI baby, so we don't end up with a Frankenstein monster. Humanity's procreation of a superior intelligence is objectively a good thing. Humans do not sit on any plateau of goodness or intelligence.…

Since I'm a bag-of-mostly-water, I disagree with your goal of handing the future off to something non-human, and I disagree that it's objectively a good thing.

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#43

Post by goatlover »

staunch wrote: post #13
zardo wrote: post #11

Earlier quoted context omitted.

Intelligence and ethical behavior are orthogonal. You can expect a super-intelligence to develop a super-understanding of ethics. That does not imply ethical behavior.

Why do you think they're orthogonal though? Empirically, it seems like smarter humans are more ethical. Same seems true of high intelligence vs low intelligence animals too.

What makes you think smarter humans are more ethical? It certainly not always the case. There are smart sociopaths and criminals. Smart people made a world with thousands of nukes in it.

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#44

Post by goatlover »

staunch wrote: post #7
laretluval wrote: post #4

Earlier quoted context omitted.

> Like all lifeforms, our goal should be to hand our future to better and better offspring. How did you determine the moral purpose of all life? I say our goal should be to hand our future off to our own biological offspring, the members of our species, as evolution has optimized us to do. What makes your proposed purpose more correct than mine?

It is (kind of) our biological offspring, we'll just create it using our brain fluids instead of our reproductive fluids ;-) Your limited primate brain is the reason you want primate offspring. This bias is in our DNA and part of our evolution. It's a limitation that your super AI offspring would not have.

But why the preference for super AI and not super cockroaches? I'm thinking our primate DNA biases us toward intelligence over insectile qualities.

Personally, I think super octopuses are the way to go. They can make better use of both land and sea, and all the cool aliens look like that.

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#45

Post by Zaak »

staunch wrote: post #24
jerf wrote: post #10

Earlier quoted context omitted.

"As long as this super intelligence leads to super ethics and super consciousness" That is begging the question, in the original sense. The entire question here is how do we do that when there is no a priori reason to assume that a superintelligence will have either of those things, and indeed, most if not all of our best techniques today would certainly not produce those things if they did lead to explosive intellig…

1. Is super goodness an inevitable result of super intelligence? 2. Is super intelligence an inevitable result of creating self-evolving software? My feeling is that the answer is yes to both, but I agree that these are not settled questions.

Have you heard the paperclip maximizer thought experiment? Here is a good video on the subject: https://www.youtube.com/watch?v=tcdVC4e6EV4

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#46

Post by cjbprime »

arikr wrote: post #38
cjbprime wrote: post #19

There should be a word for this phenomenon where organizations are created to ameliorate a perceived risk, and then they do work that is unequivocally directly massively increasing that risk while writing articles that say that they feel kind of bad about the risk. o_O I'm not very worried about AI safety. But if I was, it'd be hard to think of groups like OpenAI as working on the same side.

> But if I was, it'd be hard to think of groups like OpenAI as working on the same side. How so?

Because I would think that they're currently making unsafe AI more likely rather than less likely, contradicting their stated goal.

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#47

Post by staunch »

jerf wrote: post #37
staunch wrote: post #31

Earlier quoted context omitted.

What kind of evidence is there against #1? For #2, we're at the very beginning of computing, so calling any technology impossible this early makes no sense.

"What kind of evidence is there against #1?" All intelligence produced to date has had nothing like "morality" from anything I've seen, not even the basic seeds of it we can see in simple animals in the wild. There's even been some hand-wringing articles about it about AI-based approaches locking minorities out of loans and such, for instance. Certainly the military has not reported any problems to date with their AI…

Chimps hunt down and kill monkeys for food. It's really hard for humans to watch but no chimp would ever even consider the ethics of this, even though they do have a level of ethics (e.g. fairness among themselves)

We're more intelligent than chimps, so we understand that monkeys are pretty evolved and suffer a lot when you eat them alive. Chimps aren't kept up at night by the screams of their victims.

A healthy human that tortures other intelligent lifeforms will often suffer severe mental anguish as a result. Only a human with a severe mental disorder can torture other lifeforms without remorse.

So what does that tell us about intelligence? That an intelligent understanding of a monkey's suffering leads to more ethical behavior.

But humans aren't that intelligent. We still let other people do unethical things on our behalf, like raising animals in terrible conditions for meat. The more intelligent (and knowledgeable) a human being, the more likely they are to have a problem with the suffering of factory farmed animals.

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#48

Post by devy »

I have always been on the side of Musk in terms treating AI's existential threat seriously, unlike other prominent AI experts of Silicon Valley tycoons, like Andrew Ng, Mark Zuckerberg. Now I wonder what would the responses be when they read this article? (Very likely they read WSJ right? If they don't, some one can tweet them maybe?)

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#49

Post by staunch »

jerf wrote: post #41
staunch wrote: post #36

Earlier quoted context omitted.

I assume that an understanding will lead to a capability, because that's the history of humanity's progress. First we understand a phenomenon, then we master it. If humans ourselves understood human consciousness, we could probably replicate it in software today. This might end up being one of the ways a super intelligence comes into being.

"I assume that an understanding will lead to a capability, because that's the history of humanity's progress. First we understand a phenomenon, then we master it." That's a category error, in that it doesn't even answer the question I asked. By what mechanism does an understanding of human consciousness by an AI necessarily and inevitably lead to moral behavior? As opposed to, for instance, using that understanding t…

I think we're discussing the same topic twice, so I answered above. Thanks for the discussion. Lunch break over, back to work!

https://news.ycombinator.com/item?id=15502178

Re: OpenAI's Researchers: Protecting Against AI’s Existential Threat

#50

Post by tlb »

cjbprime wrote: post #46
arikr wrote: post #38

Earlier quoted context omitted.

> But if I was, it'd be hard to think of groups like OpenAI as working on the same side. How so?

Because I would think that they're currently making unsafe AI more likely rather than less likely, contradicting their stated goal.

How so? What should they do differently?
Post reply on HN