Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

321–329 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#321

Post by ElfinTrousers »

JohnPrine wrote: post #317
ElfinTrousers wrote: post #316

Earlier quoted context omitted.

Please see second asterisked point above.

You seem to agree that if at some point in the next few decades AI will be something we need to worry about, so I'm trying to figure out exactly what it is you oppose. Would you have opposed research into renewable energy in the 1970s since global warming was still a few decades away from being something we needed to worry about?

> You seem to agree that if at some point in the next few decades AI will be something we need to worry about

If you think this than you have misunderstood me. Now go elsewhere and bother other people.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#322

Post by JohnPrine »

ElfinTrousers wrote: post #321
JohnPrine wrote: post #317

Earlier quoted context omitted.

You seem to agree that if at some point in the next few decades AI will be something we need to worry about, so I'm trying to figure out exactly what it is you oppose. Would you have opposed research into renewable energy in the 1970s since global warming was still a few decades away from being something we needed to worry about?

> You seem to agree that if at some point in the next few decades AI will be something we need to worry about If you think this than you have misunderstood me. Now go elsewhere and bother other people.

I'm not trying to bother you. I'm genuinely curious about what makes people oppose AI safety research.

What makes you sure it's not going to be a problem?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#323

Post by ElfinTrousers »

JohnPrine wrote: post #322
ElfinTrousers wrote: post #321

Earlier quoted context omitted.

> You seem to agree that if at some point in the next few decades AI will be something we need to worry about If you think this than you have misunderstood me. Now go elsewhere and bother other people.

I'm not trying to bother you. I'm genuinely curious about what makes people oppose AI safety research. What makes you sure it's not going to be a problem?

> I'm not trying to bother you.

Your intent is irrelevant.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#324

Post by JohnPrine »

ElfinTrousers wrote: post #323
JohnPrine wrote: post #322

Earlier quoted context omitted.

I'm not trying to bother you. I'm genuinely curious about what makes people oppose AI safety research. What makes you sure it's not going to be a problem?

> I'm not trying to bother you. Your intent is irrelevant.

Why are you bothered by a simple line of follow up questions about a belief you expressed publicly?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#325

Post by fidgewidge »

mitthrowaway2 wrote: post #284
fidgewidge wrote: post #277

Earlier quoted context omitted.

"It doesn't really matter whether that carbon comes from supervolcanoes or from human emissions." It matters a lot. Whilst I was eating dinner I've now read the article you cited, the paper that it cites (Kump, Arthur & Pavlov, 2015). I also cross checked the argument and numbers against several other papers. Ward's argument is extremely slippery. This is sadly what I'm coming to expect from academics with unverifiab…

The solubility of gasses in water decreases with rising temperature. For ocean absorption of O2 from the atmosphere, the surface temperature is particularly important. This is why even if atmospheric O2 does not decrease, this could still be a concern -- albeit one with a different threshold than a scenario where the ocean warms and atmospheric O2 concentrations decrease at the same time. Still, I certainly do hope t…

You're welcome!

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#326

Post by AnIdiotOnTheNet »

JohnPrine wrote: post #312
AnIdiotOnTheNet wrote: post #183

Earlier quoted context omitted.

There are many, but the biggest one is just that the possibility space is vast and unknowable and while drawing a line straight through it to AI doom is of course possible it doesn't say anything at all about likelihood, and it can't. It makes a ton of assumptions and expects everyone to just go along with them to reach its desired conclusion. AI certainly has risks, I don't think any reasonable human being doubts th…

> AI doom cult seems to think the worst outcomes are near certainties without really backing that up What are you basing this on? There have been tons of arguments written about why these worst outcomes are likely. Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics.

> What are you basing this on?

Many conversations with AI doomers. They gloss over and make assumptions about intelligence that aren't really backed by priors and when this is pointed out they hand wave and say "but computer".

> Read Bostrom's Superintelligence for example, or Yudkowsky's Intelligence Explosion Microeconomics.

I don't really have any interest in doing so, and if I'm honest have a particularly unfavorable read of Yudkowsky as a person based on his cultish following.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#327

Post by d0mine »

skybrian wrote: post #298
d0mine wrote: post #276

Earlier quoted context omitted.

It looks like a hit piece. Someone is against rationality regarding the question of AI safety (any real practical consequences is to censor AI under _pretense_ of AI safety, Perhaps, the articles are the preventive strikes to discredit the AI safety experts that might point out the obvious).

That's a rather specific accusation to make without evidence. Possibly, a journalist might have other reasons to cover a story? Not necessarily good ones, but still...

The article itself is the evidence: read it and apply common sense.

I know almost nothing about EAs but what I know makes me think that it is likely P(SA)>P(SA|EA)>0.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#328

Post by space_fountain »

js8 wrote: post #138
space_fountain wrote: post #87

Earlier quoted context omitted.

Humans ability to push others out of the circle of concern is near limitless. We literally bought and sold people. I think there is reason for hope. We stopped doing some of those terrible things and only partially for economic reasons, but ai still fills me with some great fear. It shrinks the circle that the wealthy absolutely must keep in their circle, in their tribe, to go on and live happy lives and people of th…

> Humans ability to push others out of the circle of concern is near limitless In theory, yes. In practice, the circle has been continually expanding with technological progress. Stated differently: What's the point of being powerful if you cannot rule over other people? What's the point of having thousands of empty palaces, when no other humans see them or care?

> In theory, yes. In practice, the circle has been continually expanding with technological progress.

Only if by continually you mean in sum. The holocaust and the genocide that have come since were pretty huge back slides and the tran Atlantic slave trade was a fairly new innovation in scale and kind even if slavery had existed before. Both of those seem if anything enabled by technology. And certainly the rich and powerful are unlikely to to kill absolutely everyone else, but I thought earlier you were arguing that AI would see the end of the rich and powerful. Now you're just saying they'll want to have pets

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#329

Post by csense »

danaris wrote: post #219
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

From what I've seen, the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed: that we will create a full-fledged general AI, that it will be smarter than us, and that it will be fully capable of upgrading itself to be far beyond our control before we have any ability to prevent it from doing so. Every step of this relies on assumptions that are not merely questionabl…

> the AI safety arguments basically rely on assuming that the singularity is not merely real, but guaranteed

The argument goes like this: If you want to save the world, then doomsday scenario S is the best place for you to invest your resources if S has a nonzero probability, and currently we are investing less resources in S than in other existential risk scenarios (per basis point of probability).

"AI risk" is a pretty good candidate for S (especially back in 2010-2015 when the movement was just starting.)

> Whether AGI is possible or not, regardless of anyone's personal opinion, is as yet unprovable and unfalsifiable.

"AGI is impossible" is certainly falsifiable: All I have to do is build an AGI and show it to you.

Further, there is no theoretical reason AGI is impossible. Rather the reverse; consider these Well Settled Scientific Facts:

- (1) Human Mind = Human Brain

- (2) Human Brain obeys the laws of physics

- (3) We understand the laws of physics well enough to simulate them in a computer

If you accept these three facts, then in theory AGI is possible: You could implement AGI by building a machine implementation of the Human Brain by fully simulating the underlying physics.

> Assuming that AGI itself is possible, there is no way to tell whether we, as humans, can create an intelligence that is "smarter" than we are.

> Assuming that we can create an AGI that is "smarter" than we are, there is no way to determine whether it would be able to upgrade itself to become exponentially smarter than that, and beyond human understanding and control.

An AI safety person would say to this: "You're right, we don't know -- and that's exactly the problem."

What we can do is assign a probability based on our confidence. How likely do you think it is that we can do those things? 20%? 2%? 0.000002%?

If you say "0.000002%", what makes it so extraordinarily certain it's impossible? If you say "2%" or "20%" then as a matter of self-preservation, shouldn't our society be devoting a lot of money and smart people's time and attention to figuring out how to make sure it doesn't happen?

Post reply on HN