Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

201–210 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#201

Post by mb7733 »

roenxi wrote: post #103
yanderekko wrote: post #86

Earlier quoted context omitted.

>- "Paperclip maximizer". A corporation is a paperclip maximizer. If you take Milton Friedman seriously, that's what a corporation is supposed to do. As I point out occasionally, most of the ethical problems foreseen for AIs are known ethical problems with corporations. Corporations are not capable of having the "fast takeoff" which is an integral part of the doomer narrative here. Furthermore, a corporation being a…

Corporations, when first developed, took out large chunks of Asia before anyone could stop them. The main difference is that corporations have to be powered by people while an AI could theoretically work with complete independence.

> Corporations, when first developed, took out large chunks of Asia before anyone could stop them.

What do you mean by this?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#202

Post by [deleted] »

nyfern wrote: post #178
khazhoux wrote: post #125

Earlier quoted context omitted.

What are they actually doing? Honest question, because I don't know.

Purchasing large mansions[0], chateaus[1], and spending lavishly on administrative kickbacks[2] because longtermism can justify any present action if you extend your horizon far enough. EA is effectively two disconnected ideas at this point. [0] https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/... [1] https://www.forbes.com/sites/sarahemerson/2023/02/28/effecti... [2] https://intelligence.org/wp-content/up…

[deleted]

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#203

Post by paulcole »

JamesBarney wrote: post #50
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

There have been people for a long time who think the world is going to end. We can use your argument to dismiss anyone who fears any existential risk like asteroids, nuclear war, climate change, AI, etc... And since the world exists every group will be wrong except one. I think the risk of human extermination is much lower than many in this group believe because I think AI is much more compute bound than algorithm bo…

> The existential risks for AI seem much higher

What’s the upper bound for climate change’s existential risks? The end of human existence and society as we know it except for a relatively small number of survivors living in a world those of us alive today can barely imagine?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#204

Post by simple-thoughts »

rafabulsing wrote: post #177
simple-thoughts wrote: post #145

Earlier quoted context omitted.

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

No goals are intrinsically stupid. The only reason that you might think some particular goal is stupid is that it goes against your goals. You could conceive of a super intelligent AI that came into existence with the goal of terminating itself. That would be an "stupid goal" from our perspective, since we have the goal of self-preservation really ingrained in our brains. But for a being that self-termination is the…

I’m not sure what you mean by “stupid” but it’s definitely not the colloquial meaning. A human seeking suicide is not stupid. In fact humans often do seek suicide because humans are playing social games where the maximizing move is to commit suicide. Supporting these people requires changing their context so that they have a wider and better variety of options available.

This is just a specific example of why I reject the orthogonality thesis. You change the context, you educate the agent, you change the goals of the agent. I do not agree that humans only chase “breathe feed sex” and while I do believe many stupid behaviors do come from evolutionary history, It’s plainly obvious that education, training, and genes play a role in self restraint and goal redirection.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#205

Post by simple-thoughts »

aetherson wrote: post #152
simple-thoughts wrote: post #145

Earlier quoted context omitted.

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

I've known lots of intelligent people who pursue stupid goals. I maybe agree with you that there's an this belief that a maximally intelligent creature will blindly follow maximally obviously stupid goals, and that belief is under-argued, but your phrasing above isn't the slam dunk argument that you seem to believe.

Remember the term here is “associated”. If you spend a lot of time with people from diverse mental backgrounds, which is much harder to do than you might think, you’ll easily see that people who lack education, common sense, or the necessary context to be intelligent to pick much worse goals than people who aren’t in that position. Of course none of us humans are particularly intelligent, and all of us are stupid. Being a human is not intrinsically about intelligence - being a human is about being a social mammal. It’s funny seeing how rationalists often fail to exercise while emphasizing their own rationality - a perfect contradiction demonstrated by the difference between imagination and bodily realities.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#206

Post by AgentME »

corobo wrote: post #197
AgentME wrote: post #179

Earlier quoted context omitted.

Eliezer Yudkowsky and the LessWrong forum popularized AI safety/alignment ideas. (The Effective Altruism community was originally mostly populated by people from LessWrong.) I think this article is a little awkwardly infatuated with LessWrong, I say even as a fan, but it does fit this discussion conveniently well about where they're coming at the subject from: https://unherd.com/2020/12/how-rational-have-you-been-thi…

I think I'm at the edge of my ability with AI as I'm noticing I'm trying to argue against the usefulness of the concepts rather than the concepts themselves. At the very least I'm not smart enough to casually read these sites (lesswrong, AI alignment) at this time. I remember feeling like this (brain CPUs pegged at 100%) trying to slog through HPMOR the first time too in fairness, it's just too many concepts to take…

Oh, I think my last post was more about the people concerned with AI safety rather than the topic itself. If you want to get closer to the actual topic itself, this article is a surprisingly great resource: https://www.vox.com/future-perfect/2018/12/21/18126576/ai-ar...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#207

Post by simple-thoughts »

DennisP wrote: post #198
rafabulsing wrote: post #177

Earlier quoted context omitted.

No goals are intrinsically stupid. The only reason that you might think some particular goal is stupid is that it goes against your goals. You could conceive of a super intelligent AI that came into existence with the goal of terminating itself. That would be an "stupid goal" from our perspective, since we have the goal of self-preservation really ingrained in our brains. But for a being that self-termination is the…

Not only that, but we've managed to subvert our reward functions in exactly the way AI safety people fear. Evolution tried to get us to reproduce as much as possible, and we came up with ways to get the reward without producing offspring.

That’s a pretty naive understanding of sex. In human history sex has often played as important a role in bonding, community, and social hierarchy as it has in child production. These things are beneficial for survival. It’s very common for evolution to result in redirected drives.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#208

Post by simple-thoughts »

yanderekko wrote: post #79
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

"Apocalypse cults are always wrong, AI safety is an apocalypse cult, therefore it is (probably) wrong" is an argument which really only betrays the ignorance of the speaker regarding the specifics of the AI doomer argument. Why are apocalypse cults always wrong? Do the same specific reasons apply to the doomers here?

Humans are attracted to certain types of mythological stories - and there’s nothing wrong with that. It becomes a problem when the stories and reality are conflated, like is happening with AI. The idea that apocalypse cults are always wrong comes from two source: (1) empirical evidence, every time there’s been one it’s wrong and (2) understanding human sociology, humans are susceptible to making certain types of thought errors as groups and apocalypse cults such as the AI one demonstrate those errors everywhere.

So yeah an apocalypse cult could be right, but it would necessarily be for the wrong reasons. Just like a broken clock that matches the current time 2 minutes of the full day.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#209

Post by AgentME »

pas wrote: post #170
zarzavat wrote: post #110

Earlier quoted context omitted.

> AI is much more compute bound than algorithm bound Any reason to believe this? The human brain runs on 20 watts (!). Obviously biological hardware is not directly comparable to silicon, perhaps silicon is intrinsically less efficient by some factor such that in order to obtain 20W of human brain compute we need to expend K times more in silicon. But is that factor really so large that we are compute-bound? How big…

So there are many failure modes possible here. Yudkowsky's hard take off mentions "solving protein folding", nanomachines, persuading everyone to let them connect to the internet, infinite adaptability, etc. I think those are unrealistic, but obviously they still have a non-zero chance. More probably bad possibilities are AI-Hitler (monomaniacal populist absolute leader) or AI-Stalin (manipulative, smart, absolutely…

I agree that the most likely way an AI would take control involves social/political engineering, but that doesn't mean it will have human-like morals making it keep humanity alive once it doesn't need us or that it will have human-like limits.

>And the big infinite dollar question is could this hypothetical AI improve on itself by transcending human limits? Let's say by directly writing programs that it has conscious control over? Can it truly "watch" a 1000 video streams in real-time?

Even if its mind wasn't truly directly scalable, it could make 1000 short or long-lived copies of itself to delegate those tasks to.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#210

Post by jefftk »

nyfern wrote: post #178
khazhoux wrote: post #125

Earlier quoted context omitted.

What are they actually doing? Honest question, because I don't know.

Purchasing large mansions[0], chateaus[1], and spending lavishly on administrative kickbacks[2] because longtermism can justify any present action if you extend your horizon far enough. EA is effectively two disconnected ideas at this point. [0] https://forum.effectivealtruism.org/posts/xof7iFB3uh8Kc53bG/... [1] https://www.forbes.com/sites/sarahemerson/2023/02/28/effecti... [2] https://intelligence.org/wp-content/up…

> spending lavishly on administrative kickbacks

I just skimmed the report you link and nothing jumped out at me as matching this -- could you be more specific?

Post reply on HN