Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

251–260 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#251

Post by CatWChainsaw »

hiAndrewQuinn wrote: post #116
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

I don't have any solid arguments either, but I just get the feeling that after the first very public tragedy, like a killbot rampaging in New York City, we'll see a very, very sudden tilt in the public eye towards never wanting to touch this technology again. People are terrified of killer robots. One of the likely outcomes like that is even stronger pivoting towards walled gardens and end to end stuff than we previo…

Thou shalt not make a machine in the likeness of a man's mind.

Addendum: nor his body.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#252

Post by mitthrowaway2 »

fidgewidge wrote: post #226
paulcole wrote: post #203

Earlier quoted context omitted.

> The existential risks for AI seem much higher What’s the upper bound for climate change’s existential risks? The end of human existence and society as we know it except for a relatively small number of survivors living in a world those of us alive today can barely imagine?

There is no realistic scenario provided for by current scientific knowledge that would lead to that outcome. Waterworld was a film, not a textbook.

The scenario described in this article comes the closest, detailing a mechanism that may have been responsible for the end-Permian mass extinction, wherein warming oceans become anoxic and begin to release huge quantities of hydrogen sulfide gas. The gas is directly lethal to most life in the oceans and on land, and destroys the ozone layer as a bonus. It's hard for me to imagine any way in which an agricultural human civilization could survive that scenario.

https://www.scientificamerican.com/article/impact-from-the-d...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#253

Post by michaelt »

puzzledobserver wrote: post #106

I had previously heard of EA, and knew about paperclip maximizers, trolley problems, unintended consequences, and other risks associated with unregulated AI. But I wasn't aware of the connections between the two communities / families of ideas until recently. Because I don't consider myself sufficiently educated in these matters, I default to using GiveWell and their mosquito net recommendations as an effective low-e…

> How do philosophers go from something like GiveWell to something like fending off the AI apocalypse?

It's simply that writers like to write, forum commenters like to comment, philosophy academics like to publish papers, and there's only so many ways you can rehash The Life You Can Save.

Advocating for mosquito nets and praziquantel is like advocating that programmers should use version control - it may be true, but it's not going to drive traffic to your blog or impress people with how forward-thinking you are.

The people who haven't expanded their ideology in debatable directions aren't getting to the front page of HN.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#255

Post by mitthrowaway2 »

whatshisface wrote: post #85
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions. Since very smart human beings have in history done a lot of damage, but…

Alexander the Great or Napoleon have been limited in how much damage they can do, because they're human too. They don't live for hundreds of years, they can't copy their brains to new hardware, and they can't modify their own minds to remove that pesky conscience when it slows them down. Perhaps most importantly, Napoleon's happiness, power, and continued survival depends on having a civilization of surviving humans working under his command, whereas that might only temporarily be true for AI.

At best, history serves as a counterexample to the idea that if an AI goes bad the attendants would just unplug it, seeing how often it is that dictators don't get stabbed by their aides as soon as they start causing mass deaths, instead often receiving broad popular support as the world burns.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#257

Post by mitthrowaway2 »

simple-thoughts wrote: post #145
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

> The orthogonality thesis is an unproven assumption

A safety mindset would suggest that rather than disregarding it until proven true, we should worry about it until proven false.

> meaning an intelligent being can pursue stupid goals.

It would pursue very intelligent instrumental goals, but the terminal goal is a free variable and I don't think there exists any measure by which terminal goals can be considered smart or stupid. It would be whatever is implied by its programming.

> States like that, it’s obviously wrong and laughable.

Perhaps not so obviously wrong nor so laughable as you think?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#258

Post by bakuninsbart »

aetherson wrote: post #151
Natsu wrote: post #83

Earlier quoted context omitted.

> The issue is that the effective altruism crowd are focused exclusively on AI safety. How many EA people do you know? That seems to get the most clicks, but I usually associate them with things like mosquito bed nets to help prevent malaria, because that's a far more common topic in my experience. I'm sure you can find particular people who are all about the AI risk stuff that seems pretty wild to me, too. But when…

As someone who is not an EA, but has been peripherally aware of them for years, the bed nets stuff was their big thing until maybe three years ago, but it genuinely is the case that a lot of their energy had recently redirected to AI safety and other X-risks. That isn't to say that the malaria stuff isn't still a big topic, but there has been a shift.

Yep, I think the community is pretty evenly split amongst "longtermist"-focused people, and this includes all the X-Risk stuff, as well as the more classical "Global Health & Happiness" people. At this point they are actually relatively distinct groups I'd wager. One thing missing from the article is that, at least in my experience, the vast majority of the opportunists are in the longtermist camp.

Longtermist topics are the perfect combination of high-prestige and smart sounding with a complete lack of accountability, because the impact of research or projects won't materialize for decades. Ironically, in practice it directly contradicts the founding tenants of EA, despite being a logical conclusion of the underlying moral philosophy.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#259

Post by pksebben »

whywhywhywhy wrote: post #162

It's just super obvious by this point that none of the AI safety LARP is really about safety it's more about justifying why a technology that to hire the team to work on it requires that work to be published (Apple tried to do it without publishing and struggled to get the talent) but by publishing you lose part of your advantage so the only way to lock that advantage up is to make it so use of the models has to go t…

| If you disagree then why is OpenAI's safety committee talking about how you should need a government contract to buy an A100.

Fuck, really? I hadn't heard about this. If true, it's the godfather of all ladder pulls.

Not to mention, it's categorically awful to silo this stuff and grant it to an org that's so opaque. Good luck separating the wheat from the propaganda chaff without access to the tools that produce it.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#260

Post by fidgewidge »

mitthrowaway2 wrote: post #252
fidgewidge wrote: post #226

Earlier quoted context omitted.

There is no realistic scenario provided for by current scientific knowledge that would lead to that outcome. Waterworld was a film, not a textbook.

The scenario described in this article comes the closest, detailing a mechanism that may have been responsible for the end-Permian mass extinction, wherein warming oceans become anoxic and begin to release huge quantities of hydrogen sulfide gas. The gas is directly lethal to most life in the oceans and on land, and destroys the ozone layer as a bonus. It's hard for me to imagine any way in which an agricultural huma…

By "climate change" people mean the sort of change that will be realistically caused by humans within the next few hundred years, not the sort of extinction caused by massive volcano activity that takes 10,000 years and happens once every 100 million years. Obviously if the climate changes drastically enough fast enough then everyone dies but nobody is suggesting that's the case for preset day human activity.
Post reply on HN