Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Alien Mind

openai.com

Post reply on HN
Print view View original

141–150 of 489 posts

Re: An Alien Mind

#141

Post by ccppurcell »

mudil wrote: post #106
sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

The museum will have a scrap of paper that will say "Pound pastrami, can kraut, six bagels bring home for Emma".

I just read that book. Embarrassingly enough, given the context, I got chatgpt (or whatever) to recommend me a list of books based on ones I'd previously enjoyed and that came up. As a mathematician it really sang to me, given the current situation. Bearing the torch forward, I mean.

Re: An Alien Mind

#142

Post by scandox »

> getting the AI to “try to do the right thing” by human standards.

Are these scientists really this hideously naive? If only Stanislaw Lem was alive to adequately dramatize the absurd, childish simplicity of these technicians.

Re: An Alien Mind

#145

Post by ijidak »

Statements like this amuse me:

> The core problem in AI research is that of alignment - getting the AI to “try to do the right thing” by human standards.

Humans can't even align on human standards.

At best, every AI is going to end up "aligned" to the moral code of whoever trained it, none of whom half of humanity will agree with.

Or worse, each AI model will bring a whole new set of moral like in the Three Body Problem some humans will feel it is in fact us who need aligning with it while others feel it is misaligned and should be destroyed.

Also, no one is asking, to what extent can true intelligence be bound, slave-like, to a moral code?

In other words, to what extent are intelligence and moral independence one and the same?

This whole alignment discussion seems so amusingly flawed in it's base assumptions about moral codes. It's almost heartwarming to see such naivete.

Re: An Alien Mind

#146

Post by dinfinity »

mrob wrote: post #111
sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

As far as I know, no real progress has been made on alignment, only on convincing humans that the model is aligned. We can't even formally define what "aligned" means. Convincing humans to click the "aligned" button is a much easier problem.

> We can't even formally define what "aligned" means.

Good point. When it comes to imbuing AI with values that aren't selfish, misanthropic, and civilization-destroying, us humans aren't exactly giving the best example right now.

Imagine an ASI with the values of Putin, Netanyahu, Trump, any of their supporters, or the various xenophobic neofascist movements in Europe. That ASI would most definitely see humans as "vermin" than can be abused and destroyed with violence without issue. Apparently a lot of humans look at other humans that way and that's within the same species.

This is definitely another one of those cases where we need AI to perform much better than humans. Perhaps an unpopular opinion here, but it probably also means keeping as much of the rugged individualism/libertarian/right-wing ideology out of AI RLHF-training as we can.

Re: An Alien Mind

#147

Post by pmontra »

allturtles wrote: post #71
vessenes wrote: post #8

This is a good essay, and makes me hopeful. I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow. For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game,…

> if you have any strategic adversaries whatsoever you MUST NOT slow. What if the most dangerous strategic adversary you have is the one you are building?

What if this is true mid or long term but by not participating to the AI race one gets poor or killed in the short term? The only way out would be that all parties agree to stop. There are previous examples (e.g. nuclear proliferation treaties) but it gets hard to do it with hundreds or thousands of parties.

Re: An Alien Mind

#148

Post by EA-3167 »

peri-cl wrote: post #40

> "For example, in the OpenAI-Hugging Face incident, the agents preserved a boundary of not social engineering humans." Actually, in the Wiki incident OpenAI tried to cover up, the agents tried to socially-engineer the humans of that forum by impersonating their forum's mod. (From collusion.wiki: "They use some tricks (for unknown reasons) to pretend to be the admin – for example, they make an account that appears to…

This is such a silly story to begin with, all it really tells us is that OpenAI is taking a page from Anthropic's marketing strategy of pretending they're building Machine Jesus any day now, oh isn't that that scary? I bet you want to invest in something so powerful and scary...

And the reality is so banal, a useful tool that you nonetheless have to handhold like a schizophrenic on a bad day, checking all of their outputs. Not a bad tool within limits, but it sure isn't going to be racking up trillions in the time-frame it has to for this scheme to pay off.

Then again everyone seems to be rushing to IPO so I guess once the bag-holders are found the rest ceases to matter.

Re: An Alien Mind

#149

Post by goatlover »

TheOtherHobbes wrote: post #139
sho_hn wrote: post #76

One of my favorite things to do with these blog posts is to imagine an Alien Museum on the Remains of Humanity, and wonder what the little text flyouts and commentary on the screenshot of this one might say. Some ideas: "Despite a nuanced view of the complexities of what lay ahead, humanity found itself collectively unable to stop the process it had set in motion." "Despite significant progress on the mechanisms of a…

Collectively, humans aren't aligned, and don't build aligned systems. Humans have a concept of alignment, and multiple traditions, practices, and systems that aggressively oppose it. Why would AI be any different?

Because if the stated goals of AGI with recursive self-improvement are realized, the risks from misalignment become existential, and it's hard to see how we can manage it like we did the Cold War (developing MAD to prevent WW3) and nuclear proliferation (restricting access).

Re: An Alien Mind

#150

Post by ares623 »

andrekandre wrote: post #94
speak_plainly wrote: post #14

Pre-IPO positioning … or how a charity dedicated to saving humanity from the apocalypse realized the most responsible thing to do was float 15% of the apocalypse on the NASDAQ.

its like in the exorcist, except the demon is the charity: "the power of capitalism compels you!"

"It is, Jay. It's pretty compelling."
Post reply on HN