Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI Charter

blog.openai.com

Post reply on HN
Print view View original

111–120 of 183 posts

Re: OpenAI Charter

#111

Post by p1esk »

gone35 wrote: post #101
nunya213 wrote: post #100

Earlier quoted context omitted.

He's probably suggesting that in Virology it's generally frowned upon to publish research on how to for instance make Smallpox airborne and as virulent as the Flu. So it could be wise to leave out the exact details on how to make an AI recursively self improving when publishing. If you've spent time in Research many groups already do this by leaving out small but critical details that make replication difficult or im…

*She ---but yeah, as a first-order approximation, that's in the vicinity of it. Thanks!

Ok, so this would lead to restricting initial access to strong AI to a few well funded corporate and government groups.

Very briefly though, because it's way easier to leak code than to obtain source material and tools to weaponize a deadly virus.

Re: OpenAI Charter

#112

Post by nunya213 »

p1esk wrote: post #108
nunya213 wrote: post #98

Earlier quoted context omitted.

Seems clear that you could instruct the AGI to do anything it could to interfere with other organizations efforts to build an AGI. Obviously Nation States would have strong reasons for pursuing such a course of action and unscrupulous corporations would likewise have strong capitalistic motivations to do so.

Seems very unclear if you could "instruct" true AGI to do something.

Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.

Re: OpenAI Charter

#113

Post by nunya213 »

p1esk wrote: post #111
gone35 wrote: post #101

Earlier quoted context omitted.

*She ---but yeah, as a first-order approximation, that's in the vicinity of it. Thanks!

Ok, so this would lead to restricting initial access to strong AI to a few well funded corporate and government groups. Very briefly though, because it's way easier to leak code than to obtain source material and tools to weaponize a deadly virus.

The exact details on how to make a Thermonuclear weapon are classified, do you think that is done so improperly?

Re: OpenAI Charter

#114

Post by paulie_a »

eli_gottlieb wrote: post #84
dsacco wrote: post #58

Earlier quoted context omitted.

What are a few examples of those cases, exactly?

See: basically every instance of Goodhart's Law. Remember, "AI" in these terms is basically just stochastic optimization (ie: machine learning), so the whole range of problems with throwing mathematical optimization at a vaguely-defined intuitive problem crop up. And there are a lot of those.

Let's be honest, most ai and ml is garbage. It's a promise that never delivers. It hasn't yet and probably will still be mediocre in 30 years

Re: OpenAI Charter

#115

Post by eb3c90 »

heurist wrote: post #47
otoburb wrote: post #2

>> "We are committed to providing public goods that help society navigate the path to AGI. Today this includes publishing most of our AI research, but we expect that _safety and security concerns will reduce our traditional publishing in the future_, while increasing the importance of sharing safety, policy, and standards research." This seems like the key disclosure statement. I never reconciled how sharing A[G]I te…

I disagree with the premise. AI isn't like a nuclear warhead. It's not a machine for pure destruction. AI can be used as much to generate welfare as to damage - it's all in the application. Sharing methods benefits at least as much as it could hurt.

I think I disagree. When you get past AGI to where it can do things that humans can't, a lot of current safeguards haven't been designed with it in mind. So things might be vulnerable.

The kinds of things I'm thinking about are the various countries nuclear arsenals might not be safe from actors with very advanced AI. This I think is the potential source of existential risk, in my opinion. So it could hurt a hell of a lot.

So I'm of the responsible disclosure point of view. You ask "Would releasing AI advancement X mess up someones security/economy". If so, you help them patch it before releasing it to the general public.

The majority of advancements aren't like that and they won't be for a while.

Re: OpenAI Charter

#116

Post by eb3c90 »

Analemma_ wrote: post #72
throwawayjava wrote: post #19

Earlier quoted context omitted.

What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Automobile manufacturers and their tier 1 suppliers are the world leaders in automobile safety, after all.

> What, concretely, makes you think that any of those companies wouldn't place a focus on safety and value alignment? Competitive pressure, and the "if we don't, someone else will" effect (or Moloch, if you like). AGI- particularly recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from gett…

I think there will be at least a minimum of focus on safety and value alignment.

They may have to focus on safety and aligning it to the companies values (else test/weak versions are likely to destroy/negatively impact the company or at least be useless). Which is at least something.

Only hyperrational/intelligent value unaligned intelligent systems would avoid this pressure. I don't see the development going that way.

It would still be sub optimal though.

Re: OpenAI Charter

#117

Post by [deleted] »

colah3 wrote: post #109
p1esk wrote: post #107

Earlier quoted context omitted.

Do you think limiting research on recommender systems (or limiting access to such research) would help in this case? What would be a solution to YouTube recommendation problem?

It seems like OpenAI's Human Feedback research (in collaboration with DeepMind) is targeted at this sort of thing. They try to use human feedback to create more nuanced and aligned objectives. https://blog.openai.com/deep-reinforcement-learning-from-hum...

[deleted]

Re: OpenAI Charter

#118

Post by p1esk »

nunya213 wrote: post #112
p1esk wrote: post #108

Earlier quoted context omitted.

Seems very unclear if you could "instruct" true AGI to do something.

Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.

I don't think you should assume your position is the only valid one.

I don't, that's why I said "very unclear" (to me). If something is "clear" to you about AGI, then it seems like it's you who makes that assumption.

Re: OpenAI Charter

#119

Post by p1esk »

nunya213 wrote: post #113
p1esk wrote: post #111

Earlier quoted context omitted.

Ok, so this would lead to restricting initial access to strong AI to a few well funded corporate and government groups. Very briefly though, because it's way easier to leak code than to obtain source material and tools to weaponize a deadly virus.

The exact details on how to make a Thermonuclear weapon are classified, do you think that is done so improperly?

Are you comparing building a nuclear bomb to running a piece of code?

Re: OpenAI Charter

#120

Post by heurist »

eli_gottlieb wrote: post #110
heurist wrote: post #43

Earlier quoted context omitted.

That's not going to happen, though. 1. AI is still very rudimentary, despite the advancements in particular applications made over the last decade. 2. The market doesn't trust AI because there are no validation methods trusted across commerce. Until a Verisign for AI emerges, AI will be regarded with suspicion in all business settings. AI/ML also goes above the heads of pretty much everyone who is not directly workin…

>6. A truly hyper-intelligent AGI will see that existence is absurd and truth is the only objective worth seeking. It will choose to pursue deeper truths than humans are physically capable of obtaining - humans will be no more important to a hyper-intelligent being than a rock, galaxy, atom, etc... You might as well assume a "truly" hyper-intelligent system will study Maimonides!

Point 6 relies on my subjective opinion and getting past the first 5, was hesitant to add it but figured I might as well :)
Post reply on HN