Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

181–190 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#181

Post by dotsam »

fluxinflex wrote: post #131

I doubt very much that the poorest people on this planet care much about AIs. People living in absolute poverty in Africa, India, China etc. I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. Places where AIs aren't used to command peoples lives will probably not even realise that an AI apocalypse had happened. Perhaps a case of western navel-gazin…

> I doubt very much that the poorest people on this planet care much about AIs.

Whether people know or care about a risk doesn't mean that the risk won't affect them!

> I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world.

A rogue AI might as well just kill everyone rather than just the Western naval-gazers, as this would bring to 0 the possibility that a human could ever turn it off / thwart its plans.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#182

Post by LatteLazy »

nathan_young wrote: post #139

According to a study of 700 ML researchers (not AI safety) the median chance that that the long-run effect of advanced AI on humanity will be “extremely bad (e.g., human extinction)" is 5%. And 48% of respondents gave an answer of 10% or more. Even if this is 2 orders of magnitude too high it's worth a lot of our time. But these are probably mainly normy researchers not AI safety-ists. https://aiimpacts.org/2022-expe…

According to a survey 700 ML researchers, we should throw money at their subject or we might all die...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#183

Post by AnIdiotOnTheNet »

fshbbdssbbgdd wrote: post #69
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

There are many, but the biggest one is just that the possibility space is vast and unknowable and while drawing a line straight through it to AI doom is of course possible it doesn't say anything at all about likelihood, and it can't. It makes a ton of assumptions and expects everyone to just go along with them to reach its desired conclusion.

AI certainly has risks, I don't think any reasonable human being doubts that, its just that the AI doom cult seems to think the worst outcomes are near certainties without really backing that up.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#184

Post by [deleted] »

AgentME wrote: post #179
corobo wrote: post #171

Earlier quoted context omitted.

Can you back up those claims? I don't believe all of the nuance is contained in your comment. I'm looking into MIRI, if that's not "the AI safety community" please do correct me. I've become very aware ever since ChatGPT started saying inaccurate things confidently that humans have a much worse hit rate. Seriously, keep an eye out for it, you'll see it everywhere. At least ChatGPT will double check if you ask if it's…

Eliezer Yudkowsky and the LessWrong forum popularized AI safety/alignment ideas. (The Effective Altruism community was originally mostly populated by people from LessWrong.) I think this article is a little awkwardly infatuated with LessWrong, I say even as a fan, but it does fit this discussion conveniently well about where they're coming at the subject from: https://unherd.com/2020/12/how-rational-have-you-been-thi…

[deleted]

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#186

Post by pharmakom »

AgentME wrote: post #165
PartiallyTyped wrote: post #155

Earlier quoted context omitted.

> It's about the risk that an AI smarter than us It's also about the risk that AI is dumber than what we expect it to be and fails to properly solve the issue. A sufficiently smart AI should be able to understand our intentions and - assuming it is aligned with us - attempt to fulfill that purpose. Dumb "AI" will fail in ways that don't actually solve the task, or solve it only on the surface / do reward hacking.

>assuming it is aligned with us If we don't figure out alignment, we can't make this assumption. This is the part that AI safety people are concerned with making reality.

I don't see how alignment is a different problem from making an AI do what we want.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#187

Post by [deleted] »

AgentME wrote: post #179
corobo wrote: post #171

Earlier quoted context omitted.

Can you back up those claims? I don't believe all of the nuance is contained in your comment. I'm looking into MIRI, if that's not "the AI safety community" please do correct me. I've become very aware ever since ChatGPT started saying inaccurate things confidently that humans have a much worse hit rate. Seriously, keep an eye out for it, you'll see it everywhere. At least ChatGPT will double check if you ask if it's…

Eliezer Yudkowsky and the LessWrong forum popularized AI safety/alignment ideas. (The Effective Altruism community was originally mostly populated by people from LessWrong.) I think this article is a little awkwardly infatuated with LessWrong, I say even as a fan, but it does fit this discussion conveniently well about where they're coming at the subject from: https://unherd.com/2020/12/how-rational-have-you-been-thi…

[deleted]

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#188

Post by AnIdiotOnTheNet »

AgentME wrote: post #149
thom wrote: post #113

EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the outcomes because of the long timelines. And so you rope in more and more acolytes who both increase and accrue these same benefits. It seems unlikely that the fall from grace of prominent figures will affect the movement, given that it’s partly based on cogni…

People in EA donate lots of money to causes that help people today. AI is just one of the more interesting subjects they often talk about. AI safety isn't about Roko's basilisk, or the idea that AI will take revenge on us for our moral choices. It's about the risk that an AI smarter than us will be made and built to pursue some goal without caring about our well-being.

> It's about the risk that an AI smarter than us will be made and built to pursue some goal without caring about our well-being.

We already have those and they're called corporations. They've done significant real damage to the world already and they are still working hard to do more damage.

It makes little sense to me to focus on this potential future problem when we haven't even agreed to deal with the ones that we already have.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#189

Post by davidgerard »

aetherson wrote: post #154
precompute wrote: post #77

Earlier quoted context omitted.

Nah, HPMOR just sucks, plain and simple. There was no need for a HP fanfiction in the first place. Also, if you ever ask anyone that reads fanfiction, no one will ever have a positive opinion of HPMOR. There are millions of HP fanfics, and there are many that are much better than HPMOR. My guess is that it was set in the HP universe just so it could be popular.

My strong belief is that HPMOR is a Harry Potter fanfic not for any calculated reason, but just because the author was in a demographic maximally interested in making Harry Potter fanfic.

he has literally stated that's why he did it as a Harry Potter fanfic. "There's a large number of potential readers who would enter at least moderately familiar with the Harry Potter universe." He was also reading a lot of HP fic himself, so he knew the stuff.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#190

Post by davidgerard »

AgentME wrote: post #164
operatingthetan wrote: post #43

Earlier quoted context omitted.

He's too fragile and prone to overcompensation for that, nice try. Of course he's going to back track after getting mercilessly mocked for it and then claim it was just a joke.

The first few chapters have a song scene parodying ghostbusters and a joke about the protagonist biting a teacher in a tantrum as a young kid. There's plenty of elements of crackfic parody in it.

> a joke about the protagonist biting a teacher in a tantrum as a young kid.

wasn't that literally Yudkowsky, tho

Post reply on HN