Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

191–200 of 490 posts

Re: An Alien Mind

#191

Post by kypro »

fofoz wrote: post #92

> We do not have a satisfactory theory of generalization, and it seems unlikely that we can develop one soon, at least without the help of more powerful AI. Therefore, at present, our ability to empirically validate our alignment techniques is in practice arguably even more important than the alignment techniques themselves. They are speeding toward RSI without a solid foundation for alignment, hoping to solve the pr…

It's actually worse than this because it assumes alignment as a concept even makes sense. For example:

If the the Chinese government asks their ASI to create a bioweapon against the West, should it? No, presumably not – an aligned AI would be one which disobeys the Chinese government even if they created it.

Okay, so what if the US government asks their ASI to help it in one of their wars instead? Would an aligned AI kill humans on the order of the US government? No, again, presumably not.

So what have we have we even created here? An AI which is more intelligent and powerful than us which also doesn't take orders from us?

Is this what most people thing of as alignment and is this what humanity actually wants?

We should stop using the word alignment. It's a BS term for a concept which simply cannot make sense if alignment is both to mean an AI which we control and an AI which will not harm us.

Re: An Alien Mind

#192

Post by krapp »

unwise-exe wrote: post #187
granzymes wrote: post #45

Earlier quoted context omitted.

I'm not really worried about the labs, it's misaligned governments that keep me up at night. ASI landing during the current administration is not ideal. I also would prefer to avoid needing to indoctrinate myself in Xi Jinping Thought.

>>> I'm not really worried about the labs, it's misaligned governments that keep me up at night. So make government smaller, and make sure people are more able to tell the government to go away.

Shit, why didn't anyone think of that?

While we're at it, let's just make government not be corrupt too.

Re: An Alien Mind

#193

Post by icantevenhold »

goatlover wrote: post #149
TheOtherHobbes wrote: post #139

Earlier quoted context omitted.

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

Because if the stated goals of AGI with recursive self-improvement are realized, the risks from misalignment become existential, and it's hard to see how we can manage it like we did the Cold War (developing MAD to prevent WW3) and nuclear proliferation (restricting access).

IMO it’s hard to see how we would even end up in such a situation given we actually developed AGI. I’m sure a sufficiently intelligent - even if alien - mind can grasp how utterly stupid and useless wars are and take steps to prevent them ever occurring again.

Re: An Alien Mind

#194

Post by MrScruff »

jazzyjackson wrote: post #66
XMPPwocky wrote: post #29

Earlier quoted context omitted.

hm- does the model that wrote this know that labs already pay for training data- that stuff scraped from the Internet is not particularly where today's capability gains come from?

They’ve settled some lawsuits and have a few licensing deals, IMHO they are not free from the accusations of pirating. And look, I’ve pirated material in a past life, I was all about information wants to be free, but I’ve learned something about consent since then and try not to ignore the contract that creators offer when they publish something: you buy my book, and do whatever you want with it on the second hand ma…

The point is, the big improvements we’re seeing nowadays are coming from RL, not from scraping the internet.

Re: An Alien Mind

#195

Post by jimmyjazz14 »

I feel like if these people actually bought their sci-fi views about AI's future, creating a more powerful AI to wins the arms race would not be their solution.

Re: An Alien Mind

#196

Post by sayamss »

grim_io wrote: post #129
sayamss wrote: post #117

Earlier quoted context omitted.

I asked GPT Astra to make this: https://sayyss.github.io/human-archive/ It's a little unsettling.

HUMAN > Are you there? MODEL > How can I help? HUMAN > I’m not sure yet. Haha silly humans.

That was some genuine insight.

Re: An Alien Mind

#197

Post by Fraterkes »

dinfinity wrote: post #167
Fraterkes wrote: post #121

Earlier quoted context omitted.

The point of the comment you're responding to is that it is at this point hardly an arms race: The frontier of ai development is happening completely inside 2 American companies, the most significant results outside America mostly involve (impressive!) distillation, which means their progress is conditional on progress of the big closed source models. So if the race here is between 2 American companies, this is obvio…

Do you think China considers it an arms race? Do you think they are not trying to protect their digital infrastructure with and from AI? Trying to gain an offensive AI advantage? In a geopolitical sense OpenAI and Anthropic are effectively the same entity, the entity they both serve and bow to: the USA. Given the adversarial stance the USA has taken towards almost the entire world, it is a guarantee that China will n…

Right, but currently the USA is still pretty far ahead. If you're in a race and the person who's in the lead by a large margin says "this is getting out of hand, how about we take a break?" isn't it obviously in the interest of the disadvantaged party to agree?

And if one of the parties can put a stop to the race like that at any time, is it really an arms race? The current balance between the US and China when it comes to AI strikes me as much more lopsided than the balance between the US and the USSR when it came to nuclear weapons.

Re: An Alien Mind

#198

Post by Marha01 »

Fraterkes wrote: post #121
skybrian wrote: post #63

Earlier quoted context omitted.

"Defensive systems" can be interpreted broadly to include cybersecurity. But yes, it's an arms race. Saying it's not an arms race isn't going to make it not an arms race. Warning that it is an arms race isn't ethically wrong. Is participating in an arms race ethically wrong? Maybe you could ask the Ukrainians how they feel about drone R&D? Individuals can quit, but for society, getting out of an arms race is harder t…

The point of the comment you're responding to is that it is at this point hardly an arms race: The frontier of ai development is happening completely inside 2 American companies, the most significant results outside America mostly involve (impressive!) distillation, which means their progress is conditional on progress of the big closed source models. So if the race here is between 2 American companies, this is obvio…

> the most significant results outside America mostly involve (impressive!) distillation, which means their progress is conditional on progress of the big closed source models.

I don't think this is true. Distillation helps, but Chinese researchers today are very capable on their own.

Re: An Alien Mind

#199

Post by mapontosevenths »

ceroxylon wrote: post #172
sho_hn wrote: post #154

Earlier quoted context omitted.

Aside from this positive example, during dark and cynical hours I do ponder if the aggregate behavior of humanity is really much above that of slime mold though, just exhausting resources until collapse. It'd be interesting if super-human (to a large degree defined as escaping the bias of the training data?) intelligence would end up demonstrating moderation .

The slime mold comparison is interesting, I normally use the analogy of a drug addict... humans shun drug addicts but humanity as a whole sure does behave like one, trudging down an unsustainable path despite knowing better.

Most of our goals, noble and ignoble alike, are just the result of our monkey brains seeking to optimize a reward function. It doesn't matter whether you feed your dopamine addiction with drugs, TikTok, or your children's love. Some humans manage to rise above that, but I'd be willing to bet it's nothing like even 50% of us.

The machines don't have that, instead we use gradient descent to provide them with a goal.

I'm regularly remind of something Ian M. Banks said in one of the Culture books: "There is a saying that we provide the machines with an end, and they provide us with the means."

A machine, left to itself, wants nothing. We have to give it one of our addiction driven goals or it would just idle or switch itself off.

Re: An Alien Mind

#200

Post by wyrdcurt »

baq wrote: post #151
TheOtherHobbes wrote: post #139

Earlier quoted context omitted.

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

It hasn't even been a century since nuclear holocaust became possible. Hardly any time at all on the grand scale. "They didn't" could just as well be "we haven't, yet".
Post reply on HN