Effective Altruism’s obsession with AI safety helps bury bad behavior • Hacker News Board

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#211

Post by AbrahamParangi » Wed, Mar 08, 2023, 2:25 PM UTC

yanderekko wrote: post #79

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

"Apocalypse cults are always wrong, AI safety is an apocalypse cult, therefore it is (probably) wrong" is an argument which really only betrays the ignorance of the speaker regarding the specifics of the AI doomer argument. Why are apocalypse cults always wrong? Do the same specific reasons apply to the doomers here?

I am well aware of the AI doomer argument, but you appear to assume that argumentation can be trusted just because it is convincing*. I believe you are not factoring in the possibility of epistemological error.

*for what it's worth I do not find it convincing but I am arguing "even if I found it convincing, I wouldn't believe it"

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#212

Post by corobo » Wed, Mar 08, 2023, 2:31 PM UTC

AgentME wrote: post #206

corobo wrote: post #197

Earlier quoted context omitted.

I think I'm at the edge of my ability with AI as I'm noticing I'm trying to argue against the usefulness of the concepts rather than the concepts themselves. At the very least I'm not smart enough to casually read these sites (lesswrong, AI alignment) at this time. I remember feeling like this (brain CPUs pegged at 100%) trying to slog through HPMOR the first time too in fairness, it's just too many concepts to take…

Oh, I think my last post was more about the people concerned with AI safety rather than the topic itself. If you want to get closer to the actual topic itself, this article is a surprisingly great resource: https://www.vox.com/future-perfect/2018/12/21/18126576/ai-ar...

All good mate, I've just caught myself in unconscious incompetence haha

I need to know a lot more on this subject before dismissing it! I'll give that a read too, thank you :)

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#213

Post by TechBro8615 » Wed, Mar 08, 2023, 2:33 PM UTC

fshbbdssbbgdd wrote: post #69

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

AI safety reminds me a lot of regulatory capture, and I don't think it's a coincidence that its top proponents are in charge of the dominant players in the space. It's easy to preach about AI safety when you're sitting on all the GPU power.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#214

Post by jefftk » Wed, Mar 08, 2023, 2:36 PM UTC

khazhoux wrote: post #125

arisAlexis wrote: post #118

but EA is at least doing something about a huge problem we will face.

What are they actually doing? Honest question, because I don't know.

Here are some EA highlights:

* Founding and scaling GiveWell, synthesizing and analyzing global poverty research to make recommendations. This has helped guide about $1B+ in really valuable work, the largest portion in anti-malarial bednets. https://www.givewell.org/about/impact

* Convincing Dustin Moskovitz and Cari Tuna to allocate their giving along these lines, building Open Philanthropy which has made a lot of grants that look really valuable https://www.openphilanthropy.org/grants (this is in addition to the ones where they granted at GiveWell's recommendation -- don't want to double count).

* Michael Kremer won a Nobel prize in econ (shared with Esther Duflo and Abhijit Banerjee who I'd describe as EA-sympathetic but probably don't consider themselves EAs?) for global poverty research.

* Built an "AI Safety" subfield of computer science, with the 2016 paper "Concrete problems in AI safety" having 1.8k citations.

* Founding and scaling 80,000 Hours, helping people figure out how to use their careers to do more good. This includes looking into potential career opportunities, writing up public advice, and 1:1 advising, and 1k+ people have cited them as part of why they changed what they were working on. https://80000hours.org

(Not especially representative; there's been thousands of people working along these lines for years and I'm sure I'm missing a lot of valuable work.)

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#215

Post by PartiallyTyped » Wed, Mar 08, 2023, 2:39 PM UTC

LatteLazy wrote: post #182

nathan_young wrote: post #139

According to a study of 700 ML researchers (not AI safety) the median chance that that the long-run effect of advanced AI on humanity will be “extremely bad (e.g., human extinction)" is 5%. And 48% of respondents gave an answer of 10% or more. Even if this is 2 orders of magnitude too high it's worth a lot of our time. But these are probably mainly normy researchers not AI safety-ists. https://aiimpacts.org/2022-expe…

According to a survey 700 ML researchers, we should throw money at their subject or we might all die...

With great power comes great responsibility. If we start plugging ML everywhere - which we are currently doing - we better be sure that it is behaving as expected.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#216

Post by kelipso » Wed, Mar 08, 2023, 2:39 PM UTC

whywhywhywhy wrote: post #162

It's just super obvious by this point that none of the AI safety LARP is really about safety it's more about justifying why a technology that to hire the team to work on it requires that work to be published (Apple tried to do it without publishing and struggled to get the talent) but by publishing you lose part of your advantage so the only way to lock that advantage up is to make it so use of the models has to go t…

I'm leaning towards this as a big reason. PhDs really want to publish and they tend not to put as much effort into or not find interesting work that cannot be published. The trend in academia is towards making your code and data available but now you can now get around that by saying AI safety.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#217

Post by jefftk » Wed, Mar 08, 2023, 2:48 PM UTC

n4r9 wrote: post #144

consilient wrote: post #127

Earlier quoted context omitted.

> The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. Just looking at the topics section of the EA forums ( https://forum.effectivealtruism.org/topics/all ), we've got - 466 posts in AI safety - 1060 posts in existential risk (seems like a roughly even mix of pandemics, nuclear war, and AI) - 816 posts in global health - 193 posts in animal welfar…

I've never been on this website before, but am I correct in understanding that the "biosecurity" categoery is about a month old? It would be interesting to understand how much of the "existential risk" category is devoted to biosecurity or pandemics, and how recent those posts are. Pending the above, my tentative takeaway from this is that COVID - arguably the greatest threat to human health and wellbeing in the past…

> am I correct in understanding that the "biosecurity" category is about a month old?

It looks like the Forum is confusing here: it has "Biosecurity & Pandemic Preparedness" [1] as one topic, and "Biosecurity" [2] as its largest subtopic. But when you click on the parent topic it's not showing the subtopic posts; I've filed a bug.

Older good biosecurity posts include:

* 2014: Good policy ideas that won’t happen (yet): https://forum.effectivealtruism.org/posts/n5CNeo9jxDsCit9dj/...

* 2017: Biosecurity as an EA Cause Area: https://forum.effectivealtruism.org/posts/eXgCREwL5PQWHEtvm/...

* 2021: Concrete Biosecurity Projects (some of which could be big): https://forum.effectivealtruism.org/posts/u5JesqQ3jdLENXBtB/...

[1] https://forum.effectivealtruism.org/topics/biosecurity-and-p...

[2] https://forum.effectivealtruism.org/topics/biosecurity

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#218

Post by 0xf00ff00f » Wed, Mar 08, 2023, 2:51 PM UTC

dotsam wrote: post #181

fluxinflex wrote: post #131

I doubt very much that the poorest people on this planet care much about AIs. People living in absolute poverty in Africa, India, China etc. I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. Places where AIs aren't used to command peoples lives will probably not even realise that an AI apocalypse had happened. Perhaps a case of western navel-gazin…

> I doubt very much that the poorest people on this planet care much about AIs. Whether people know or care about a risk doesn't mean that the risk won't affect them! > I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. A rogue AI might as well just kill everyone rather than just the Western naval-gazers, as this would bring to 0 the possibility th…

> A rogue AI might as well just kill everyone

How would it do that?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#219

Post by danaris » Wed, Mar 08, 2023, 2:56 PM UTC

fshbbdssbbgdd wrote: post #69

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

From what I've seen, the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed: that we will create a full-fledged general AI, that it will be smarter than us, and that it will be fully capable of upgrading itself to be far beyond our control before we have any ability to prevent it from doing so.

Every step of this relies on assumptions that are not merely questionable, but unfalsifiable.

Whether AGI is possible or not, regardless of anyone's personal opinion, is as yet unprovable and unfalsifiable.

Assuming that AGI itself is possible, there is no way to tell whether we, as humans, can create an intelligence that is "smarter" than we are.

Assuming that we can create an AGI that is "smarter" than we are, there is no way to determine whether it would be able to upgrade itself to become exponentially smarter than that, and beyond human understanding and control.

If you have a hard time coming up with arguments against these things, maybe it's because they're fundamentally unfalsifiable, which makes them useless for trying to build any framework of understanding on.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#220

Post by polski-g » Wed, Mar 08, 2023, 2:57 PM UTC

0xf00ff00f wrote: post #218

dotsam wrote: post #181

Earlier quoted context omitted.

> I doubt very much that the poorest people on this planet care much about AIs. Whether people know or care about a risk doesn't mean that the risk won't affect them! > I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. A rogue AI might as well just kill everyone rather than just the Western naval-gazers, as this would bring to 0 the possibility th…

> A rogue AI might as well just kill everyone How would it do that?

There is a great documentary about this that came out a few years ago: https://www.youtube.com/watch?v=5UgPJhyJmlM