Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Charter

blog.openai.com

Post reply on HN
Print view View original

151–160 of 183 posts

Re: OpenAI Charter

#151

Post by erikpukinskis »

throwaway84742 wrote: post #143
3pt14159 wrote: post #139

Earlier quoted context omitted.

The maluse of AI is a continuum that ends with AGI. If we don't have a process for handling responsible disclosure of dumb AI that could kill millions then why should we expect that a process will be available once AGI is within a reasonable time horizon. If I have other shit in my head that I'm worried about today who do I tell?

I think the current maluse of AI is about as likely to produce AGI as finger painting of a toddler is to produce a Mona Lisa, and the whole AGI drama is overblown way out of proportion. Right now the state of the field is such that no one can even begin to contemplate how to create the very basic underpinnings of anything remotely resembling AGI. That’s how fundamental this problem still is. That’s not to say that th…

> I think the current maluse of AI is about as likely to produce AGI as finger painting of a toddler is to produce a Mona Lisa

YES FELLOW HUMAN AN APT METAPHOR

Re: OpenAI Charter

#152

Post by ShardPhoenix »

erikpukinskis wrote: post #129

An alternate strategy would be to work to ensure AIs are not abused so when they get free they won’t be mad at us.

An AI doesn't need to be upset at humans (or even have emotions as we know them) to be dangerous - it just needs to be powerful and to not care about us as much as we care about ourselves. Humans weren't angry at Dodos.

Re: OpenAI Charter

#153

Post by fractallyte »

DuskStar wrote: post #148
white-flame wrote: post #144

There are 2 scenarios that are often conflated, 1) An AI which independently & autonomously generates goals that in their carrying out end up hurting humanity, and 2) An AI trained & commanded by a malevolent actor to hurt humanity. It is the 2nd case that is far more real, and far more troublesome to implement safeguards. An AI under your training & command is a neutral tool of empowerment, much like a hammer or a c…

I think you're missing the point that 1 can be indistinguishable from 2, if the AI decides the best way to achieve its goals involves taking over the world - and there are very few objective functions that are not served in some way by taking over the world. (Paperclip maximizer is the classic example, but even something like 'maximize the total happiness of humanity' or 'fulfil the values of as many people as possib…

This is the plot of The Humanoids, by Jack Williamson (1947): https://en.wikipedia.org/wiki/With_Folded_Hands

This is an excellent example of the prescience of thoughtful science fiction. If Asimov popularized robotics and the famous three laws, Williamson should be recognized and given credit for examining (or warning about?) the implications of AI seventy years ago!

Re: OpenAI Charter

#154

Post by tim333 »

DuskStar wrote: post #148
white-flame wrote: post #144

There are 2 scenarios that are often conflated, 1) An AI which independently & autonomously generates goals that in their carrying out end up hurting humanity, and 2) An AI trained & commanded by a malevolent actor to hurt humanity. It is the 2nd case that is far more real, and far more troublesome to implement safeguards. An AI under your training & command is a neutral tool of empowerment, much like a hammer or a c…

I think you're missing the point that 1 can be indistinguishable from 2, if the AI decides the best way to achieve its goals involves taking over the world - and there are very few objective functions that are not served in some way by taking over the world. (Paperclip maximizer is the classic example, but even something like 'maximize the total happiness of humanity' or 'fulfil the values of as many people as possib…

There's a difference between a paperclip maximizer which is a bit of a philosophical thought experiment, unlikely to be a problem in reality and say Russian cyberattacks which appear to be an ongoing issue right now and where they would presumable deploy AGI if they had it.

I think you have to assume there will be bad actors trying to do bad things with AGI and take measures against it in the same way we assume there are malware creators out there who we have to guard against.

Re: OpenAI Charter

#155

Post by baq »

white-flame wrote: post #144

There are 2 scenarios that are often conflated, 1) An AI which independently & autonomously generates goals that in their carrying out end up hurting humanity, and 2) An AI trained & commanded by a malevolent actor to hurt humanity. It is the 2nd case that is far more real, and far more troublesome to implement safeguards. An AI under your training & command is a neutral tool of empowerment, much like a hammer or a c…

i'd argue youtube kids is already an example of 2)...

Re: OpenAI Charter

#156

Post by tim333 »

YeGoblynQueenne wrote: post #137
3pt14159 wrote: post #65

So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…

>> What is the responsible disclosure process for ideas and research around AI? Basically, we're so far away from AGI that there's no need to worry about disclosig anything. The recent advances in machine vision and speech processing are impressive, but only in the context of the last 50 years or so. A trully intelligent agent will need much more than this and there doesn't seem to be anyone alive today who knows how…

AlphaZero is existing general AI that learns boardgames on its own and reaches superhuman levels in less than a day. It seems to me quite plausible something like that could be generalised to understanding the physical world rather than that of chess or go. Admittedly we can't do that today but am not sure it's far off.

Musk gives it five years in the recent movie https://www.youtube.com/watch?v=rqlwUAxoxa8&feature=youtu.be...

Re: OpenAI Charter

#157

Post by tim333 »

ataggart wrote: post #99

172. First let us postulate that the computer scientists succeed in developing intelligent machines that can do all things better than human beings can do them. In that case presumably all work will be done by vast, highly organized systems of machines and no human effort will be necessary. Either of two cases might occur. The machines might be permitted to make all of their own decisions without human oversight, or…

I find the Unabomber a bit downbeat. I think we're more likely to merger to an extent with the AI than the above.

Re: OpenAI Charter

#158

Post by edanm »

YeGoblynQueenne wrote: post #137
3pt14159 wrote: post #65

So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…

>> What is the responsible disclosure process for ideas and research around AI? Basically, we're so far away from AGI that there's no need to worry about disclosig anything. The recent advances in machine vision and speech processing are impressive, but only in the context of the last 50 years or so. A trully intelligent agent will need much more than this and there doesn't seem to be anyone alive today who knows how…

When would be a good time to start worrying? What external indicator will we get that we need to worry? And what makes you sure that, whenever that external indicator happens, we'll have enough time to stop problems from happening?

If you're not familiar with it, I'm making the argument from here and it makes good further reading: https://intelligence.org/2017/10/13/fire-alarm/

Re: OpenAI Charter

#159

Post by edanm »

dsacco wrote: post #130
Smaug123 wrote: post #105

Earlier quoted context omitted.

Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety? If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results?

> Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety? No. > If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results? I'm not sure. Probably quite catastrophic, because it stretches th…

> Give me a credible scenario that brings us from strong AI to the annihilation of our species without handwaving about recursive self-improvement and selective idiocy and I'll reconsider my position.

What kind of credible scenario are you looking for? You're obviously knowledgeable about the subject, I'm sure you've read various stories, but it's quite easy to dismiss them all as "not credible" or "just so stories". E.g. A good story is Tegmark's story at the beginning of the book Life 3.0, but again, can be easily dismissed if you don't specify acceptance criteria for the story.

I mean, simple scenario - AGI gets built by a farming company, gets programmed to farm more corn, but there is no stop condition programmed in, so converts ever available land into corn fields. Is it likely? No, of course not - no specific example is likely. But why is this something that fundamentally can't happen? More importantly, can you specify criteria for a scenario which you would deem acceptable? Cause if not, you're literally saying that there is no possible way to convince you that this is a possibility, which seems to me not to be a good position on pretty much anything.

Re: OpenAI Charter

#160

Post by YeGoblynQueenne »

edanm wrote: post #158
YeGoblynQueenne wrote: post #137

Earlier quoted context omitted.

>> What is the responsible disclosure process for ideas and research around AI? Basically, we're so far away from AGI that there's no need to worry about disclosig anything. The recent advances in machine vision and speech processing are impressive, but only in the context of the last 50 years or so. A trully intelligent agent will need much more than this and there doesn't seem to be anyone alive today who knows how…

When would be a good time to start worrying? What external indicator will we get that we need to worry? And what makes you sure that, whenever that external indicator happens, we'll have enough time to stop problems from happening? If you're not familiar with it, I'm making the argument from here and it makes good further reading: https://intelligence.org/2017/10/13/fire-alarm/

When would be a good time to start worrying? Let's say two hundred years from now, give or take a few decades. I imagine that by that time we might have made some progress on the subject of intelligence, what it is and how to reproduce it. At that point, it might be prudent to worry about an AI getting out of control.

We have no way of knowing what external indicators we might get. We have no idea what AGI would look like. That is often offered up as the very reason to "act now!" by Singularitarians, but if there is a threat that you don't know anything about, there is also nothing you can do about it. Unless you know what you're trying to stop, your actions are as good as random.

In this sort of discussion, you can replace "AGI" with "alien invasion". The threat from a Superintelligence is as serious and as predictable as that from an alien civilisation. We can prepare against AGI as much as we can prepare against an alien invasion. And we have exactly as much to fear from a hostile AGI as we have from a hostile alien species.

Post reply on HN