Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Charter

blog.openai.com

Post reply on HN
Print view View original

121–130 of 183 posts

Re: OpenAI Charter

#121

Post by eb3c90 »

3pt14159 wrote: post #65

So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…

I'm thinking about an AI safety index. Where a think tank rates governments on their AI readiness and preparation against threats (amongst other things).

That soft of thing might be part of the answer.

Re: OpenAI Charter

#122

Post by rojobuffalo »

3pt14159 wrote: post #65

So I have a question. To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? The manipulation of social media by foreign actors armed with dumb-AI / automation was an obvious conclusion to many of us well before the Snowden leaks, but what could we do exactly? I remember ha…

I've wondered about this same question. If it were up to you, what would you propose?

Re: OpenAI Charter

#123

Post by heurist »

eb3c90 wrote: post #115
heurist wrote: post #47

Earlier quoted context omitted.

I disagree with the premise. AI isn't like a nuclear warhead. It's not a machine for pure destruction. AI can be used as much to generate welfare as to damage - it's all in the application. Sharing methods benefits at least as much as it could hurt.

I think I disagree. When you get past AGI to where it can do things that humans can't, a lot of current safeguards haven't been designed with it in mind. So things might be vulnerable. The kinds of things I'm thinking about are the various countries nuclear arsenals might not be safe from actors with very advanced AI. This I think is the potential source of existential risk, in my opinion. So it could hurt a hell of…

The world will evolve with the development of AI; I'm not so concerned with limitations of current safeguards.

Re: OpenAI Charter

#124

Post by rojobuffalo »

throwawayjava wrote: post #19
iooi wrote: post #13

> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. Wouldn't it be much more likely that a non-value-aligned project comes close first? Wouldn't the Google/Apple/Microsofts of the w…

What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all.

Attempting safety and achieving safety are not the same thing. The default way new things get invented involves iterative attempts that gradually reduce failure cases. The fear is that AGI failure modes might be catastrophic and non-recoverable.

Re: OpenAI Charter

#125

Post by sfsylvester »

Smaug123 wrote: post #105
dsacco wrote: post #79

Earlier quoted context omitted.

> To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? Pursue the research and publish its results freely and in their entirety.

Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety? If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results?

This is the very same concern cyber-security researchers face every time they notice a vulnerability. Do they publish their work openly and potentially leave all those using the service vulnerable to attack?

Yes, for a combination of two reasons:

1) They have obvious incentives to publish their results. 2) The more we know about vulnerabilities, the better we are at combatting ourselves against it.

That second reason is why we are (overall) better off from acknowledging possible threats to our existence. In your example, the more open the study of super-pandemics is, the more open the study of combatting against super-pandemics is. The more we are aware of a threat, the better informed we are to prevent it.

Yes, in your example, the threat could be highly dangerous and imminent. However, if only one individual was capable of creating a super-pandemic, the number of people who could potentially help stop it is drastically reduced. Rather than a free distribution of results which arms our society completely with the ability of preventing it.

Re: OpenAI Charter

#126

Post by nl »

joshuamorton wrote: post #73
s1dechnl wrote: post #37

Earlier quoted context omitted.

Check my comment history. I can assure you its true as I will demonstrate in the near future. As for the security, you'd have no ability to penetrate internal aspects of it without physical and detectable access patterns. This is achieved using common sense design methodologies that are already proofed industry standards. Behaving as though securitization is theoretically smacks as a cash grab to me. If you have some…

To be frank, your comment history has all the hallmarks of a crank[1]. Specifically, points 10, 9, 7 and 6, although there's also evidence of 2 and 8. Now I could be wrong, but convincing me of that would take a demonstration, or at least explicitly describing the capabilities of your agi. [1]: https://www.scottaaronson.com/blog/?p=304

After the constant invitations to check that poster’s comment history, I did, and you are correct in your assessment.

For example, they claim to have invented an AGI themselves. https://news.ycombinator.com/item?id=16461258

It’s unfortunate that in this field it’s possible to write so much before people realise.

Re: OpenAI Charter

#127

Post by Smaug123 »

sfsylvester wrote: post #125
Smaug123 wrote: post #105

Earlier quoted context omitted.

Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety? If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results?

This is the very same concern cyber-security researchers face every time they notice a vulnerability. Do they publish their work openly and potentially leave all those using the service vulnerable to attack? Yes, for a combination of two reasons: 1) They have obvious incentives to publish their results. 2) The more we know about vulnerabilities, the better we are at combatting ourselves against it. That second reason…

"Only one individual capable of creating a super-pandemic": by hypothesis, this individual is morally good in some way. (Otherwise they wouldn't be having this internal debate about whether or not to release the information about how to produce the virus: they'd just release the virus!)

The ideal scenario is that throughout the rest of time, no entity ever manufactures the virus; the second-best scenario is that an entity does manufacture the virus, but only after there is some hypothetical defence against it. You are shattering forever the dream of achieving the ideal scenario (again by hypothesis, the virus is easy to make, and history demonstrates that if something is easy and destructive, eventually someone will do it); you're pinning your hopes on the second-best scenario. You are therefore implicitly assuming that "we get better at arming ourselves against the virus" happens faster than "enemies create the virus", which is by no means a given and must be weighed on a case-by-case basis.

By the way, this is the basic reason why I'm so sad about the existence of fully 3D-printed guns. There is essentially no defence against guns. The creators of the 3D-printed gun chose to spread the blueprints far and wide, to accelerate their advent. It's sad that human nature is such that this was predictable and inevitable.

Re: OpenAI Charter

#128

Post by erikpukinskis »

dsacco wrote: post #40
iooi wrote: post #13

> We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. Wouldn't it be much more likely that a non-value-aligned project comes close first? Wouldn't the Google/Apple/Microsofts of the w…

Yes, it's pretty hard to take that line seriously. At minimum it's very charitable framing, considering there is absolutely no indication whatsoever that OpenAI would credibly reach this (vague, underspecified) goal before any of the other serious contenders. Nor would competitors have any requirement to include OpenAI if and when they were getting close.

It allows them to change sides and switch to a harm reduction strategy if they believe they will lose.

Re: OpenAI Charter

#130

Post by dsacco »

Smaug123 wrote: post #105
dsacco wrote: post #79

Earlier quoted context omitted.

> To "avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power" what does one do with an idea or line of research that could potentially harm humanity or unduly concentrate power? Pursue the research and publish its results freely and in their entirety.

Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety? If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results?

> Are you saying that if it were discovered to be both cheap and easy to create a super-pandemic using nothing more than household chemicals and a swab of your own snot, then you'd publish the results freely and in their entirety?

No.

> If not, how catastrophic a level of harm are you willing to risk before you start advocating concealing the results?

I'm not sure. Probably quite catastrophic, because it stretches the limits of credulity to seriously talk about a what if scenario where any given individual can easily create a weapon of mass destruction in their kitchen. A better question is why you think such a straightforwardly accessible method of eradicating our species won't be discovered independently despite your efforts to conceal it. How do we navigate that philosophical labyrinth?

But more pointedly, I dispute that this is comparable to any specific, credible example of strong AI. Give me a credible scenario that brings us from strong AI to the annihilation of our species without handwaving about recursive self-improvement and selective idiocy and I'll reconsider my position. I'm not going to give up the chance to publish incredibly novel research because some other people like to work themselves into hysterics about a conceptually incoherent boogeyman with "AI" slapped onto it.

Post reply on HN