Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

151–160 of 490 posts

Re: An Alien Mind

#151

Post by baq »

TheOtherHobbes wrote: post #139
sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

Re: An Alien Mind

#153

Post by throwthrowuknow »

Fraterkes wrote: post #121
skybrian wrote: post #63

Earlier quoted context omitted.

"Defensive systems" can be interpreted broadly to include cybersecurity. But yes, it's an arms race. Saying it's not an arms race isn't going to make it not an arms race. Warning that it is an arms race isn't ethically wrong. Is participating in an arms race ethically wrong? Maybe you could ask the Ukrainians how they feel about drone R&D? Individuals can quit, but for society, getting out of an arms race is harder t…

The point of the comment you're responding to is that it is at this point hardly an arms race: The frontier of ai development is happening completely inside 2 American companies, the most significant results outside America mostly involve (impressive!) distillation, which means their progress is conditional on progress of the big closed source models. So if the race here is between 2 American companies, this is obvio…

Development and improvement of nuclear weapons was an entirely American project until the technology was exfiltrated and then it became an instant arms race. That cat is already out of the bag with LLMs. Distillation is just the fastest way to keep pace but that in no way prevents other countries and actors from doing it the hard way.

An arms race doesn’t imply one side winning, it’s not a race with an end goal, it’s a race to keep pace or retake the lead position which can oscillate between the parties involved indefinitely. The other option is to agree to make no further progress or to disarm.

Re: An Alien Mind

#154

Post by sho_hn »

baq wrote: post #151
TheOtherHobbes wrote: post #139

Earlier quoted context omitted.

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

Humans had multiple occasions to press the 'launch a nuclear holocaust' button and... they didn't. I'd expect aligned AIs to also not press it even when it'd be rational to do so according to their instructions - then work from there.

Aside from this positive example, during dark and cynical hours I do ponder if the aggregate behavior of humanity is really much above that of slime mold though, just exhausting resources until collapse.

It'd be interesting if super-human (to a large degree defined as escaping the bias of the training data?) intelligence would end up demonstrating moderation.

Re: An Alien Mind

#157

Post by jzer0cool »

It would be nice to postulate some of these potential emergent systems outlines with timelines. Then it may help better map the granular alignment needs.

Re: An Alien Mind

#158

Post by esikich »

kikkupico wrote: post #65

Reminds me of the time Kasparov said playing chess against a supercomputer felt like facing an alien opponent.

What's ironic is that was all in his head. They were very normal looking games. We didn't get alien chess until Stockfish level bots.

Re: An Alien Mind

#159

Post by goatlover »

skybrian wrote: post #63
pu_pe wrote: post #37

> The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI. So the best argument for AI is that it's an arms race. We have to keep pushing every boundary because in any case others will, and we will need to defend against them. If this statement is true, then this particular researchers believes the open source Chines…

"Defensive systems" can be interpreted broadly to include cybersecurity. But yes, it's an arms race. Saying it's not an arms race isn't going to make it not an arms race. Warning that it is an arms race isn't ethically wrong. Is participating in an arms race ethically wrong? Maybe you could ask the Ukrainians how they feel about drone R&D? Individuals can quit, but for society, getting out of an arms race is harder t…

The arms race is created by American companies who justify the risk by claiming China will win the race if they don't. But it's the American companies who are purshing the arms race forward.

Re: An Alien Mind

#160

Post by ambicapter »

21o12asg wrote: post #52

"As we outlined recently with Sam , OpenAI prioritizes work in service of three north stars" Not one North Star. Not two. Just three! OpenAI broke the North Star record! With this evidence of AI slop, why did you not label this fluff piece as AI generated for the EU? You are violating laws.

Yeah, when I read that I just assume #1 is actually the only thing they'll focus on.
Post reply on HN