Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

171–180 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#171

Post by corobo »

AgentME wrote: post #169
corobo wrote: post #141

Earlier quoted context omitted.

I'm sure a lot of people are basing their belief on AI on sci-fi interpretation. If you're one of the folks doing this at least acknowledge the story would have been quite boring if it was 800 pages of "everything went well, the AI turned out really great, very helpful. Would use again" If you're not blindly afraid of zombies you shouldn't be blindly afraid of AI. Give it thought. If you're still scared after dismiss…

The AI safety community has many people who were previously extremely optimistic about the sheer world-changing potential of AI, but then realized that getting AI right might be harder than getting it wrong, and that figuring out how to get it right must be a priority before a superintelligent AI is made. It's not people who just saw Terminator and made up their mind.

Can you back up those claims? I don't believe all of the nuance is contained in your comment.

I'm looking into MIRI, if that's not "the AI safety community" please do correct me.

I've become very aware ever since ChatGPT started saying inaccurate things confidently that humans have a much worse hit rate.

Seriously, keep an eye out for it, you'll see it everywhere. At least ChatGPT will double check if you ask if it's sure. People tend to just get annoyed when you don't blindly trust their "research" haha.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#172

Post by [deleted] »

renewiltord wrote: post #28

What is one or two rapes or sexual assaults against trillions of future lives? With sufficient future sight and the power of the exponent > 1, one can justify many present actions. Sadly, the EAs are small thinkers. True grand scale actions require truly grand visions https://cosmicbraintrust.org/

[deleted]

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#174

Post by PartiallyTyped »

AgentME wrote: post #165
PartiallyTyped wrote: post #155

Earlier quoted context omitted.

> It's about the risk that an AI smarter than us It's also about the risk that AI is dumber than what we expect it to be and fails to properly solve the issue. A sufficiently smart AI should be able to understand our intentions and - assuming it is aligned with us - attempt to fulfill that purpose. Dumb "AI" will fail in ways that don't actually solve the task, or solve it only on the surface / do reward hacking.

>assuming it is aligned with us If we don't figure out alignment, we can't make this assumption. This is the part that AI safety people are concerned with making reality.

I think I was misunderstood. My point is that not only do we need to consider the cases where the AI is smarter than us, but also the cases where it is dumber than we expect it.

An aligned agent will not fall into stupid errors, thus solving the lower bound of performance is a necessity for complete alignment.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#175

Post by AgentME »

PartiallyTyped wrote: post #174
AgentME wrote: post #165

Earlier quoted context omitted.

>assuming it is aligned with us If we don't figure out alignment, we can't make this assumption. This is the part that AI safety people are concerned with making reality.

I think I was misunderstood. My point is that not only do we need to consider the cases where the AI is smarter than us, but also the cases where it is dumber than we expect it. An aligned agent will not fall into stupid errors, thus solving the lower bound of performance is a necessity for complete alignment.

Ohh, right on. Definitely agree now that I've re-read it right. I was a little confused by the first sentence of your other post and thought you were putting forward a disagreement in the second sentence.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#176

Post by [deleted] »

arisAlexis wrote: post #119
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

There are top level scientistis warning of this apocalypse. Sam Altman which was the CEO of the very site you are posting and the CEO of OpenAI has warned starkly that there is a probability that humanity will go extinct from AI in his latest interview. Have you seen it? You think he is somehow bluffing or is he part of the same cult?

[deleted]

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#177

Post by rafabulsing »

simple-thoughts wrote: post #145
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

No goals are intrinsically stupid. The only reason that you might think some particular goal is stupid is that it goes against your goals.

You could conceive of a super intelligent AI that came into existence with the goal of terminating itself. That would be an "stupid goal" from our perspective, since we have the goal of self-preservation really ingrained in our brains.

But for a being that self-termination is the absolute best thing ever, it's not stupid. It makes perfect sense, since, well, that it's goal. It doesn't care about self-preservation, it doesn't care about becoming more intelligent/rich/powerful, other than as an instrumental goal to help achieve self-termination, if it's not able to do so in its current state.

And most importantly, no amount of getting more intelligent would change this fundamental goal, just as humans getting more intelligent has not overridden our fundamental goals of "breathe, feed, have sex". It may have given us other goals as well, but those are very much still there.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#178

Post by nyfern »

khazhoux wrote: post #125
arisAlexis wrote: post #118

but EA is at least doing something about a huge problem we will face.

What are they actually doing? Honest question, because I don't know.

Purchasing large mansions[0], chateaus[1], and spending lavishly on administrative kickbacks[2] because longtermism can justify any present action if you extend your horizon far enough. EA is effectively two disconnected ideas at this point.

[0] https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/...

[1]https://www.forbes.com/sites/sarahemerson/2023/02/28/effecti...

[2]https://intelligence.org/wp-content/uploads/2022/12/Independ...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#179

Post by AgentME »

corobo wrote: post #171
AgentME wrote: post #169

Earlier quoted context omitted.

The AI safety community has many people who were previously extremely optimistic about the sheer world-changing potential of AI, but then realized that getting AI right might be harder than getting it wrong, and that figuring out how to get it right must be a priority before a superintelligent AI is made. It's not people who just saw Terminator and made up their mind.

Can you back up those claims? I don't believe all of the nuance is contained in your comment. I'm looking into MIRI, if that's not "the AI safety community" please do correct me. I've become very aware ever since ChatGPT started saying inaccurate things confidently that humans have a much worse hit rate. Seriously, keep an eye out for it, you'll see it everywhere. At least ChatGPT will double check if you ask if it's…

Eliezer Yudkowsky and the LessWrong forum popularized AI safety/alignment ideas. (The Effective Altruism community was originally mostly populated by people from LessWrong.) I think this article is a little awkwardly infatuated with LessWrong, I say even as a fan, but it does fit this discussion conveniently well about where they're coming at the subject from: https://unherd.com/2020/12/how-rational-have-you-been-this-y...

While MIRI is prominently connected to Yudkowsky, I wouldn't treat them as defining the AI alignment community. There are many people not involved with it who make substantive posts and discussions on LessWrong and the Alignment Foundations forum. There are other organizations too. OpenAI considers alignment important and has researchers concerned with it, though Yudkowsky argues the company doesn't do enough to prioritize it relative to the AI progress they make. Anthropic is an AI company prioritizing AI safety through interpretability research.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#180

Post by gwd »

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

> AI safety is an apocalypse cult

Just because the parts of the community in the Effective Altruism / AI Safety have developed aspects of a cult doesn't mean that their arguments are wrong. It just means that they're humans doing human things. It sounds like their main mistake was to not realize that the cult-like behavior of other cults was sociological, due to the fact that they contained humans, as opposed to from their belief systems; and thus didn't take any steps to try to prevent their own movement from developing cult-like aspects.

That said, if you have a movement of thousands (?) of people all trying to figure out how to make AI safe for over a decade, and you still don't feel like you're any closer to that goal at the end of it, then you should probably step back and ask yourself what you're doing wrong.

Post reply on HN