Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

151–160 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#151

Post by aetherson »

Natsu wrote: post #83
esperent wrote: post #70

Earlier quoted context omitted.

The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. If they talked about all possible threats to humanity and how to solve them, from AI, to climate change, to nuclear bombs, to asteroids, to disease, they would be an interesting and important group. They don't. They hardly talk about anything else except AI. From the outside it looks a lot like a…

> The issue is that the effective altruism crowd are focused exclusively on AI safety. How many EA people do you know? That seems to get the most clicks, but I usually associate them with things like mosquito bed nets to help prevent malaria, because that's a far more common topic in my experience. I'm sure you can find particular people who are all about the AI risk stuff that seems pretty wild to me, too. But when…

As someone who is not an EA, but has been peripherally aware of them for years, the bed nets stuff was their big thing until maybe three years ago, but it genuinely is the case that a lot of their energy had recently redirected to AI safety and other X-risks.

That isn't to say that the malaria stuff isn't still a big topic, but there has been a shift.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#152

Post by aetherson »

simple-thoughts wrote: post #145
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

I've known lots of intelligent people who pursue stupid goals.

I maybe agree with you that there's an this belief that a maximally intelligent creature will blindly follow maximally obviously stupid goals, and that belief is under-argued, but your phrasing above isn't the slam dunk argument that you seem to believe.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#153

Post by humanrebar »

roenxi wrote: post #103
yanderekko wrote: post #86

Earlier quoted context omitted.

>- "Paperclip maximizer". A corporation is a paperclip maximizer. If you take Milton Friedman seriously, that's what a corporation is supposed to do. As I point out occasionally, most of the ethical problems foreseen for AIs are known ethical problems with corporations. Corporations are not capable of having the "fast takeoff" which is an integral part of the doomer narrative here. Furthermore, a corporation being a…

Corporations, when first developed, took out large chunks of Asia before anyone could stop them. The main difference is that corporations have to be powered by people while an AI could theoretically work with complete independence.

I think the argument is that due to emergent behavior and possibly perverse incentives corporations are indistinguishable from slow-moving AI -- they can be treated like independent agents.

Elsewhere in the thread, it's brought up that AI could move faster than corporations generally move. That might be true, but there are still physical limits: the speed of light, moving mountains of steel takes energy, etc.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#154

Post by aetherson »

precompute wrote: post #77
SamReidHughes wrote: post #24

Earlier quoted context omitted.

It went above your head.

Nah, HPMOR just sucks, plain and simple. There was no need for a HP fanfiction in the first place. Also, if you ever ask anyone that reads fanfiction, no one will ever have a positive opinion of HPMOR. There are millions of HP fanfics, and there are many that are much better than HPMOR. My guess is that it was set in the HP universe just so it could be popular.

My strong belief is that HPMOR is a Harry Potter fanfic not for any calculated reason, but just because the author was in a demographic maximally interested in making Harry Potter fanfic.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#155

Post by PartiallyTyped »

AgentME wrote: post #149
thom wrote: post #113

EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the outcomes because of the long timelines. And so you rope in more and more acolytes who both increase and accrue these same benefits. It seems unlikely that the fall from grace of prominent figures will affect the movement, given that it’s partly based on cogni…

People in EA donate lots of money to causes that help people today. AI is just one of the more interesting subjects they often talk about. AI safety isn't about Roko's basilisk, or the idea that AI will take revenge on us for our moral choices. It's about the risk that an AI smarter than us will be made and built to pursue some goal without caring about our well-being.

> It's about the risk that an AI smarter than us

It's also about the risk that AI is dumber than what we expect it to be and fails to properly solve the issue.

A sufficiently smart AI should be able to understand our intentions and - assuming it is aligned with us - attempt to fulfill that purpose.

Dumb "AI" will fail in ways that don't actually solve the task, or solve it only on the surface / do reward hacking.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#156

Post by pksebben »

karaterobot wrote: post #15
flangola7 wrote: post #12

Earlier quoted context omitted.

What's the name of it?

Harry Potter and the Methods of Rationality . I'm glad such a thing exists, but I could never work up the interest to read it myself. https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_o...

wait, really? I scrolled right past someone posting about "Harry Potter rationality fanfiction" and thought, "huh, oddly specific burn but it's chuckle worthy".

it's an actual thing?

This is a joke, right? tongue in cheek or something. Or should I go get some Brawndo to water my basil...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#157

Post by PartiallyTyped »

dbspin wrote: post #136
thom wrote: post #113

EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the outcomes because of the long timelines. And so you rope in more and more acolytes who both increase and accrue these same benefits. It seems unlikely that the fall from grace of prominent figures will affect the movement, given that it’s partly based on cogni…

Roko's basilisk makes no sense. Sans magic physics time travel - there's no motivation. One thing all A.I. is, no matter how nefarious, is goal directed. It might have poorly aligned goals, absurd goals, dangerous goals, goals it misunderstands, goals that contradict etc. But ultimately it proceeds toward some form of goal directed reward function. There's zero reward in punishing someone for past behaviour - except…

> There's zero reward in punishing someone for past behaviour

A similar argument regarding Roko's Basilisk is that it is effectively a prisoner's dilemma, where the Basilisk torturing us and us not helping it is the Defect-Defect scenario. It makes no sense for either party to defect.

> except in the context of altering their future behaviour.

It really seems like a very contrived version of Pascal's mugging.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#158

Post by dotsam »

pksebben wrote: post #143
consilient wrote: post #127

Earlier quoted context omitted.

> The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. Just looking at the topics section of the EA forums ( https://forum.effectivealtruism.org/topics/all ), we've got - 466 posts in AI safety - 1060 posts in existential risk (seems like a roughly even mix of pandemics, nuclear war, and AI) - 816 posts in global health - 193 posts in animal welfar…

didn't really have much of a dog in this fight, but looking at that breakdown I wonder which oil company is bankrolling them. I know I'm jumping the gun just a little, but c'mon. An apocalypse list without climate?

Climate change is unlikely to kill everyone on earth. Other risks, such as an engineered pandemic, asteroid impact, or an AI apocalypse have the possibility of killing ~everyone. This is not saying that climate change is not a real issue.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#159

Post by kubanczyk »

akvadrako wrote: post #90
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

My argument against is that we can't do much about it. We can't predict how machines will interpret our rules if they are smarter than us. Nor do we have a shared set of values we can articulate. Nor can we control rogue users making rogue AI. So unless one is proposing a global crackdown on AI research, AI safety is a lost cause.

> Nor can we control rogue users making rogue AI.

Any uncontrolled group that creates an AI can either find a body of safety research accessible to them, or not. Preparing the former is hardly a lost cause.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#160

Post by AgentME »

SiempreViernes wrote: post #93
SamReidHughes wrote: post #31

Earlier quoted context omitted.

It was supposed to be a flawed protagonist.

At the end Harry is basically a Messiah that can order the magical world to provide emergency medical aid to everyone in the whole world and turns Hermione into basically an angel; not really seeing the story itself saying he did anything wrong...

You're describing one of the unofficial fan-written fanfic sequels.
Post reply on HN