Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

111–120 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#111

Post by yanderekko »

KvanteKat wrote: post #108
yanderekko wrote: post #86

Earlier quoted context omitted.

>- "Paperclip maximizer". A corporation is a paperclip maximizer. If you take Milton Friedman seriously, that's what a corporation is supposed to do. As I point out occasionally, most of the ethical problems foreseen for AIs are known ethical problems with corporations. Corporations are not capable of having the "fast takeoff" which is an integral part of the doomer narrative here. Furthermore, a corporation being a…

What exactly is the argument that corporations are incapable of unbounded exponential growth contra a possible future AI? Is there just something magic about computers, or am I missing something obvious?

You can find discussions on this by googling "AI Foom" - the key idea is that an AGI that can recursively self-improve will be able to rapidly escape human control before any entity could check it, and very likely without humans even knowing that this had happened. Such an AI would likely consider humans an obstacle to its objectives, and would develop some sort of capacity to quickly destroy the world. Popular hypothesized mechanisms for doing so involve creating nanomachines or highly-lethal viruses.

That's key to the story of the paperclip maximizer - the paperclip maximizer will go about its task by trying to improve itself to be able to best solve the problem, and once it's improved enough it will decide that paperclips would be maximized by destroying humanity and would come up with a plan to achieve this outcome. However, humans may not realize that the AI is planning this until it's too late.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#112

Post by TheDong »

puzzledobserver wrote: post #106

I had previously heard of EA, and knew about paperclip maximizers, trolley problems, unintended consequences, and other risks associated with unregulated AI. But I wasn't aware of the connections between the two communities / families of ideas until recently. Because I don't consider myself sufficiently educated in these matters, I default to using GiveWell and their mosquito net recommendations as an effective low-e…

EAs premise includes basically "all human lives are equal. Saving one stranger's life is just as valuable as saving your father's life (more valuable actually if the stranger has more years left than your dad), so you should donate your kidney to a stranger, not your father"

"All human lives are equal" then expands to "and there are more unborn lives than there are currently living lives, so it's more important to save unborn kids in 200 years by diverting AI catastrophe now".

I personally don't think unborn lives are worth anything, and as such I don't follow that jump. I don't feel a need to justify it either.

I think there's two additional easy ways to veer off the EA course.

The first is "how do we know EA is right about the charities?" The answer you'll get is basically "Sure, you can donate a kidney to your father and know it saved his life, you can verify that... but the people verifying your EA kidney donation does more good are smarter than you. They're rationalists. They do math. They know selling your kidney, which would have only saved one life, to a black market and getting enough money for 50 mosquito nets, which will save 20 lives, is the right deal. The spreadsheet says so."

I do not buy that the people running and choosing EA charities are actually the smartest people, especially when it comes to having a positive impact on lives. Sure, they talk a big talk, but I know for a fact that when I give money to a local shelter, the money gets used for good. I know for a fact that when I donated to a certain EA charity I won't name, the money went into "AI research" which is to say into the pockets of a rich white dude. I know for a fact that the average EA rationalist I have met does not come off as a caring human.

A second way that one can veer off the EA course is by deciding that one values some lives more than others. For example, if I live in a small city, I may decide I value the lives of orphans in this city more than the lives of the orphans in some third-world country. Sure, feeding an orphan in africa is cheaper so EA would say do that, but I may decide that the intangible value of improving my local community is worth more than "getting the most bang for my buck". I think this is also a totally reasonable argument, and in fact I would much rather live in a world where everyone tried to improve their local community than one where everyone used a spreadsheet to try and maximize their "best improvement per dollar".

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#113

Post by thom »

EA is at least partly a moral and ethical pyramid scheme. You get to be noble and far-sighted, do relatively little real work, and you’ll never be judged on the outcomes because of the long timelines. And so you rope in more and more acolytes who both increase and accrue these same benefits. It seems unlikely that the fall from grace of prominent figures will affect the movement, given that it’s partly based on cognitive dissonance as it is.

All that said, the amount of energy we expend on AI safety probably shouldn’t be zero. But I choose to believe that if Roko’s basilisk is going to torture us over anything, it’ll be that we didn’t raise up millions of AI scientists from the cotton fields and sweatshops. Credit to the less prominent altruists doing that, or who at least have some form of compound interest built into their utility function which compels them to be kind, now.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#114

Post by geraldhh »

roenxi wrote: post #102
geraldhh wrote: post #94

Earlier quoted context omitted.

> Why are apocalypse cults always wrong? because we are still here

If there was an apocalypse cult in ancient Pompeii predicting the mountain would explode and bury everyone with ash, they would not be here. And there easily could have been. This is not an argument, it is a tautology powered by the anthropic principle.

that example is a local disaster, not an apocalypse.

but yes, there most certainly were people predicting it and they probably didn't get a chance to enjoy being right.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#115

Post by geraldhh »

yanderekko wrote: post #100
geraldhh wrote: post #94

Earlier quoted context omitted.

> Why are apocalypse cults always wrong? because we are still here

That's the consequence of them being wrong, not the cause..

denying the existence of circular dependencies does not invalidate them

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#116

Post by hiAndrewQuinn »

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

I don't have any solid arguments either, but I just get the feeling that after the first very public tragedy, like a killbot rampaging in New York City, we'll see a very, very sudden tilt in the public eye towards never wanting to touch this technology again. People are terrified of killer robots.

One of the likely outcomes like that is even stronger pivoting towards walled gardens and end to end stuff than we previously had on the Internet. Can't train a model on someone else's encrypted database without decrypting it first.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#117

Post by consilient »

KvanteKat wrote: post #108
yanderekko wrote: post #86

Earlier quoted context omitted.

>- "Paperclip maximizer". A corporation is a paperclip maximizer. If you take Milton Friedman seriously, that's what a corporation is supposed to do. As I point out occasionally, most of the ethical problems foreseen for AIs are known ethical problems with corporations. Corporations are not capable of having the "fast takeoff" which is an integral part of the doomer narrative here. Furthermore, a corporation being a…

What exactly is the argument that corporations are incapable of unbounded exponential growth contra a possible future AI? Is there just something magic about computers, or am I missing something obvious?

> What exactly is the argument that corporations are incapable of unbounded exponential growth

Individual corporations aren't capable of unbounded exponential growth because they can't keep the interests of the humans that make them up aligned with the "interests" of the corporation indefinitely. They develop cancer of the middle management and either die or settle into a comfortable steady-state monopoly.

Market systems as a whole can and do grow exponentially - and this makes them extremely dangerous. But they're not intelligent and so can't effectively resist when a world power decides to shorten the leash, as occasionally happens.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#119

Post by arisAlexis »

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

There are top level scientistis warning of this apocalypse. Sam Altman which was the CEO of the very site you are posting and the CEO of OpenAI has warned starkly that there is a probability that humanity will go extinct from AI in his latest interview. Have you seen it? You think he is somehow bluffing or is he part of the same cult?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#120

Post by arisAlexis »

Top level scientistis warning of this apocalypse. Sam Altman which was the CEO of the very site you are posting and the CEO of OpenAI has warned starkly that there is a probability that humanity will go extinct from AI in his latest interview. Have you seen it? You think he is somehow bluffing or is he part of the same cult?
Post reply on HN