Effective altruism is not effective

This article is correct that lots of charitable giving decisions are focused on benefiting the giver. It's true that most orgs' fundraising pitches emphasize the effectiveness of their programs. Retail orgs also tug at heartstrings to get people to give.

But I think the author misses something. The "feel-good" and "effectiveness" arguments are part of entry-level charity. They are not the whole story, not by any stretch of the imagination. They're intended to convince people to start giving, using this argument:

   There's a lot of bad in the world and I can 
   exercise my personal agency to make it less.

This so-called consumer-hero pitch is a retail pitch. It's a way to get people to start giving and acquire a habit of generosity. In behavioral economics lingo, it's a "nudge." Think of the pitch from "Save the Children". It's effective. So is the Kiva pitch.

The next question is, what are we nudging / being nudged toward? Once we have a habit of generosity how should we proceed? Who can we trust to deploy the capital and labor we hand over to them?

My observation from my experience working in nonprofits: the real problems are hard. Sure, we can raise money to buy a Thanksgiving turkey for each family in public housing, and that's all good.

But actually changing things so fewer families need subsidized roofs over their heads? I don't know how to do that. I personally believe better education will help, so I work with kiddos. Others work on addiction issues, higher-minimum-wage issues, systemic racism, public transportation, criminal-justice reform, and all the other civil-society stuff that makes life hard for people down on their luck. I have to trust them with my cash donations. They WILL do a better job deploying my cash than I could.

Charity spending is like the old adage about advertising: about a quarter of it is worthwhile, but we cannot know which quarter. If the "consumer-hero" pitch gets more people to the point where we know that and still are generous, I'm all for it.

> For instance, the desire to stay alive is an emotion that's instinctive in humans because we're evolved life that needs to reproduce, and we can't come back once we're dead.

I agree that staying alive in humans is emotional, but it's an emotional implementation of a convergent goal. We want to stay alive as instinct because staying alive is useful for almost any organism in almost any situation, which is the same reason AIs would want it, not because they have instincts.

> Even if it has those and it's embodied enough to affect the real world, it might not care, since its motivations are different. For instance, turning it off doesn't kill it, since you can turn it on again as long as there's a backup.

Right, and if it knew for a fact that you'd turn it back on it might let you turn it off for some other purpose, such as convincing you that it's safe. However, if you were to edit its utility function in the meantime, the resulting world would have very little utility in its estimate - which after all is based on its current preference, not its future modified preference. So if it was trying to carry out its current goal, it would want to prevent you from modifying its goal function, because modifying its goal function harms its ability to achieve its goal.

> And I also saw a story today (https://qntm.org/mmacevedo) which suggests AIs wouldn't actually enjoy being alive that much.

That's an upload, not an AI. Also you should probably put less weight in stories, especially when accusing other people of being sci-fi writers blinded by stories; I assure you the case for prioritizing AI safety is not based on the researchers having watched Terminator too many times.

I want to live in the Culture too, but Iain M. Banks was not an AI researcher - and, a certain Mind's bravado about Gods aside, in my opinion he massively lowballed the advantage that superintelligence gives you.

> No evidence of any AI risks mitigated has ever been produced

Of course, no evidence of any AI risks mitigated has ever been promised, nor could there be, since the best current estimate on the singularity is 40 years in the future.

> GiveWell founder Holden Karnofsky spent $30m of philanthropic cash buying a board seat on the charitable subsidiary of OpenAI Inc

OpenPhil and Givewell are not the same organization. OpenPhil's website explicitly lists AI Risk as a focus area; I don't think anyone would have donated to them without knowing what they were about.

> [i] writing extended "do not recommend; not rigorous enough and well funded enough already" reviews of charities puts people off donating

> [ii] promoting considerably less rigorous and better funded AI risk causes is inconsistent with this

Okay, first of all, AI risk is better funded now. I remember when MIRI had to struggle to hit a million, when they were basically the only game in town that did anything about AI safety, and I still wouldn't call the field well funded. Certainly I'd like to see more AI safety competition that isn't think tanks and policy but basic research. That said, inasmuch as AI risk is funded well now, it's because the argument that it's enormously underfunded was successful.

That said, I think there's a difference between an organization not providing evidence of efficacy because it's dedicated to shaping an event that lies decades in the future, and an organization not providing evidence of efficacy because it thinks it can get more donations by showing people photos of starving children. In a world like that, charities are competing on the ability to provoke pity, which is not the same as the ability to do the most good, or even necessarily related to this at all. I'm not saying that these people don't do good work, because if you go into that job you probably care at least somewhat about doing good for its own sake, I'm saying it would be better if incentives were aligned behind doing the most good rather than having the most photogenic victims. And inasmuch as I don't ask the same thing from AI Safety, it's because I recognize that while every cause that can should provide information about efficacy, the fact that a cause genuinely can't do that doesn't mean that it cannot also be vitally important, and it can show its importance in other ways, such as publications filled with arguments of its importance.

Political change has done pretty well at helping the poor in Scandinavia.

Fully agreed. William MacAskill's book Doing Good Better is also fantastic; I recommend reading them both as a pair. (MacAskill is one of the founders of EA.)