Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

141–150 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#141

Post by corobo »

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

I'm sure a lot of people are basing their belief on AI on sci-fi interpretation.

If you're one of the folks doing this at least acknowledge the story would have been quite boring if it was 800 pages of "everything went well, the AI turned out really great, very helpful. Would use again"

If you're not blindly afraid of zombies you shouldn't be blindly afraid of AI. Give it thought.

If you're still scared after dismissing sci-fi bias, cool, I'm all ears.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#142

Post by js8 »

Vecr wrote: post #104
js8 wrote: post #67

Earlier quoted context omitted.

Human nature. If you're a part of the common tribe, people are actually pretty good. There doesn't seem to be an upper limit on the size of the tribe (although some people just wanna hate somebody). You can also look at it this way. The only real objection against socialist societies was that people are lazy, and don't care enough about producing stuff or resolving economic inefficiencies. But with AI, this objection…

I think that's dubious, most people's motivations are only loosely coupled with the goal (if there really is one) of making the world a more efficient place. I guess that's what's good about capitalism, at least in the End of History sense[0]. Gets more people more aligned with general efficiency. Anyway, guess what's not efficient, at least in the standard economic sense? Stockpiling for a pandemic. Got to have a bi…

> most people's motivations are only loosely coupled with the goal (if there really is one) of making the world a more efficient place

That's my point! Your personal AI (in collaboration with other AIs) can take care of figuring that out as needed. You don't need capitalism anymore.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#143

Post by pksebben »

consilient wrote: post #127
esperent wrote: post #70

Earlier quoted context omitted.

The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. If they talked about all possible threats to humanity and how to solve them, from AI, to climate change, to nuclear bombs, to asteroids, to disease, they would be an interesting and important group. They don't. They hardly talk about anything else except AI. From the outside it looks a lot like a…

> The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. Just looking at the topics section of the EA forums ( https://forum.effectivealtruism.org/topics/all ), we've got - 466 posts in AI safety - 1060 posts in existential risk (seems like a roughly even mix of pandemics, nuclear war, and AI) - 816 posts in global health - 193 posts in animal welfar…

didn't really have much of a dog in this fight, but looking at that breakdown I wonder which oil company is bankrolling them.

I know I'm jumping the gun just a little, but c'mon. An apocalypse list without climate?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#144

Post by n4r9 »

consilient wrote: post #127
esperent wrote: post #70

Earlier quoted context omitted.

The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. If they talked about all possible threats to humanity and how to solve them, from AI, to climate change, to nuclear bombs, to asteroids, to disease, they would be an interesting and important group. They don't. They hardly talk about anything else except AI. From the outside it looks a lot like a…

> The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. Just looking at the topics section of the EA forums ( https://forum.effectivealtruism.org/topics/all ), we've got - 466 posts in AI safety - 1060 posts in existential risk (seems like a roughly even mix of pandemics, nuclear war, and AI) - 816 posts in global health - 193 posts in animal welfar…

I've never been on this website before, but am I correct in understanding that the "biosecurity" categoery is about a month old?

It would be interesting to understand how much of the "existential risk" category is devoted to biosecurity or pandemics, and how recent those posts are.

Pending the above, my tentative takeaway from this is that COVID - arguably the greatest threat to human health and wellbeing in the past several decades - seems to have been a minor/non-existent blip on their radar until it actually happened. This raises serious questions in my mind about the community's predictive abilities and/or bias.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#145

Post by simple-thoughts »

fshbbdssbbgdd wrote: post #69
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence.

The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent being can pursue stupid goals. States like that, it’s obviously wrong and laughable. But by using complex language, EA cultists hide the ridiculous assumptions their system has so that they can maintain their feelings of superiority while gaining real power that enables them to abuse others.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#146

Post by AgentME »

TheDong wrote: post #112
puzzledobserver wrote: post #106

I had previously heard of EA, and knew about paperclip maximizers, trolley problems, unintended consequences, and other risks associated with unregulated AI. But I wasn't aware of the connections between the two communities / families of ideas until recently. Because I don't consider myself sufficiently educated in these matters, I default to using GiveWell and their mosquito net recommendations as an effective low-e…

EAs premise includes basically "all human lives are equal. Saving one stranger's life is just as valuable as saving your father's life (more valuable actually if the stranger has more years left than your dad), so you should donate your kidney to a stranger, not your father" "All human lives are equal" then expands to "and there are more unborn lives than there are currently living lives, so it's more important to sa…

Many EA people are into forms of longtermism but AI safety arguments don't rely on longtermism at all. Many people worried about AI safety think it will be relevant in well less than 200 years. (Though uh, would you really not care at all if the world ended in 200 years? This sounds much more alien to me than any form of longtermism, and I think most people would agree that society should make decisions to sustain itself.)

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#147

Post by vcxy »

Vecr wrote: post #74
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

Yeah, I can't really argue against the whole idea. My "P(doom)"(language from the article), at least as related to agentic AI, is pretty low. About 7 or 8 percent before I die, and I'm not really old. There's probably not a lot I could do about it anyway, given that most AI jobs are alleged to just make it worse. That said, I can't really go around telling people with different numbers they should be working on antiv…

"Roll a D20 and if you get 1 humanity as you know it will be destroyed in your lifetime" seems bad. Why do you consider 7 or 8 percent low?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#148

Post by simple-thoughts »

nathan_young wrote: post #140
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

Ad what % is it crazy shit? I'm at about 7% of human disempowerment/exctinction, so it probably won't happen. Am I crazy?

You’re vastly overestimating the risks but you’re also not crazy - humans naturally respect authority figures and follow their beliefs which is in fact good for a healthy life.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#149

Post by AgentME »

thom wrote: post #113

EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the outcomes because of the long timelines. And so you rope in more and more acolytes who both increase and accrue these same benefits. It seems unlikely that the fall from grace of prominent figures will affect the movement, given that it’s partly based on cogni…

People in EA donate lots of money to causes that help people today. AI is just one of the more interesting subjects they often talk about.

AI safety isn't about Roko's basilisk, or the idea that AI will take revenge on us for our moral choices. It's about the risk that an AI smarter than us will be made and built to pursue some goal without caring about our well-being.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#150

Post by AgentME »

n4r9 wrote: post #144
consilient wrote: post #127

Earlier quoted context omitted.

> The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. Just looking at the topics section of the EA forums ( https://forum.effectivealtruism.org/topics/all ), we've got - 466 posts in AI safety - 1060 posts in existential risk (seems like a roughly even mix of pandemics, nuclear war, and AI) - 816 posts in global health - 193 posts in animal welfar…

I've never been on this website before, but am I correct in understanding that the "biosecurity" categoery is about a month old? It would be interesting to understand how much of the "existential risk" category is devoted to biosecurity or pandemics, and how recent those posts are. Pending the above, my tentative takeaway from this is that COVID - arguably the greatest threat to human health and wellbeing in the past…

The rationalist and EA communities are often credited for taking Covid much more seriously sooner than most and the media did. https://unherd.com/2020/12/how-rational-have-you-been-this-y...
Post reply on HN