Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

241–250 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#241

Post by dotsam »

0xf00ff00f wrote: post #218
dotsam wrote: post #181

Earlier quoted context omitted.

> I doubt very much that the poorest people on this planet care much about AIs. Whether people know or care about a risk doesn't mean that the risk won't affect them! > I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. A rogue AI might as well just kill everyone rather than just the Western naval-gazers, as this would bring to 0 the possibility th…

> A rogue AI might as well just kill everyone How would it do that?

Get access to the internet, get money. Money buys it power to get people to do things in the real world. Or it can hack through systems and blackmail and coerce people. Now it can act in the real world by getting people to do things it wants.

As one example it could design, synthesize and spread a highly contagious and lethal pathogen.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#243

Post by n4r9 »

jefftk wrote: post #217
n4r9 wrote: post #144

Earlier quoted context omitted.

I've never been on this website before, but am I correct in understanding that the "biosecurity" categoery is about a month old? It would be interesting to understand how much of the "existential risk" category is devoted to biosecurity or pandemics, and how recent those posts are. Pending the above, my tentative takeaway from this is that COVID - arguably the greatest threat to human health and wellbeing in the past…

> am I correct in understanding that the "biosecurity" category is about a month old? It looks like the Forum is confusing here: it has "Biosecurity & Pandemic Preparedness" [1] as one topic, and "Biosecurity" [2] as its largest subtopic. But when you click on the parent topic it's not showing the subtopic posts; I've filed a bug. Older good biosecurity posts include: * 2014: Good policy ideas that won’t happen (yet)…

Thanks for the reading. The 2017 talk/article is interesting in light of COVID:

"Some of the reasons I'm skeptical of natural risks are that first of all, they've never really happened before. Humans have obviously never been caused to go extinct by a natural risk, otherwise we would not be here talking. It doesn't seem like human civilization has come close to the brink of collapse because of a natural risk, especially in the recent past.

"You can argue about some things like the Black Death, which certainly caused very severe effects on civilization in certain areas in the past. But this implies a fairly low base rate. We should think in any given decade, there's a relatively low chance of some disease just emerging that could have such a devastating impact. Similarly, it seems like it rarely happens with nonhuman animals that a pathogen emerges that causes them to go extinct. I know there's one confirmed case in mammals. I don't know of any others. This scarcity of cases also implies that this isn't something that happens very frequently, so in any given decade, we should probably start with a prior that there's a low probability of a catastrophically bad natural pathogen occurring."

I wonder if this is a case of extrapolation fallacy? Modern human dynamical behaviour is significantly different to both animal behaviour and less modern behaviour. Viruses spread more easily in the age of global travel; those that used to be too deadly to spread very far suddenly have more opportunity.

EDIT: Reading this more carefully, the speaker does actually address globalisation, but seems to dismiss it as a counterargument and I'm not really sure why.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#244

Post by YeGoblynQueenne »

Could a kind user please have a second look at the comment I posted earlier in this thread and which has been flagged to death? That was meant as a riddle.

Are riddles not OK for HN? Or should I have made it more clear that it is a riddle?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#245

Post by weatherlight »

AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

Extinction is the rule, Survivability is the exception.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#246

Post by CatWChainsaw »

Quarrelsome wrote: post #134
arisAlexis wrote: post #118

but EA is at least doing something about a huge problem we will face.

we _will_ face? Are we forecasting AGI as a given?

Personally, I'm forecasting mass human disassociation/suffering as a result of pursuing AGI as a given. :)

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#247

Post by skinnymuch »

orzig wrote: post #137
Nexialist wrote: post #8

If you'd like to read more about the roots of Effective Altruism, ranging from old mailing lists, LessWrong, bitcoin, cults, AI and the surprising connections to many well known figures in tech, I highly recommend the 7-part Extropia's Childen series from Jon Evans [1] 1. https://aiascendant.substack.com/p/extropias-children-chapte...

This feels like the endless debate about whether America is forever tainted by the sins of its founding fathers. I’m pro-American because I believe in its stated values. Moreover, I believe that despite original and continuing failures to perfectly reflect those values, there is large and concrete progress in aggregate. I feel the same way about EA: I aspire to their ideals (including the continued discussion and rec…

I can’t imagine having the optimism you have. My rude mind is thinking this person is naive but you know I’m pretty naive too haha!

I disagree and think both the US and institutions like EA are too tainted. The US hasn’t done anything about slavery except make things worse still. That’s not on the founding fathers any more only. It’s on the US since the civil war. EA will take money from future multimillionaire and billionaires. Much like the US, they are both completely controlled by capitalism. I am in the US but not pro US.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#248

Post by ShredKazoo »

whatshisface wrote: post #85
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions. Since very smart human beings have in history done a lot of damage, but…

>I don't think it's impossible to come up with arguments against them. For one thing, extrapolating the current gradual rate of progress forwards means that we will get to see several minor "intelligence spills" before the hypothetical big and last one, and by observing what went wrong, humanity will have the opportunity to come up with solutions.

This has been discussed under "warning shots" https://www.lesswrong.com/posts/idipkijjz5PoxAwju/warning-sh...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#249

Post by CatWChainsaw »

fluxinflex wrote: post #131

I doubt very much that the poorest people on this planet care much about AIs. People living in absolute poverty in Africa, India, China etc. I also doubt they would be - negatively - affected if a rogue AI would turn out the lights in the electronicified world. Places where AIs aren't used to command peoples lives will probably not even realise that an AI apocalypse had happened. Perhaps a case of western navel-gazin…

By that logic, uncontacted tribes shouldn't be negatively affected by the modern world, and the Sentinelese don't need to concern themselves with sea level rise even if their island is completely swallowed.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#250

Post by ShredKazoo »

simple-thoughts wrote: post #145
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

>In real life, intelligence is associated with picking better goals.

"Better" according to what metric?

It may be the case that there is a tendency for high-intelligence humans to pick "more enlightened" goals. Perhaps there is a natural "enlightened goals" attractor for our species.

However I don't think we can extrapolate from that to a fundamentally alien AI.

I think even if this statistical tendency exists, it has clear counterexamples -- consider that 2 genius chess players may have opposite goals, of beating one another. And we shouldn't bet the future of humanity on this statistical tendency extrapolating outside of the original distribution of human species.

Here are some intuition pumps on how diverse goals can be even across intelligent species:

* Orcas killing sharks for their livers: https://www.livescience.com/2-orcas-slaughter-19-sharks-in-a... I don't believe dolphins show the same level of violence, even though both are smart cetaceans

* Intelligent dogs bred to be responsive and attentive to human needs -- unlike close cousins like the wolf

* Chimpanzees and bonobos are both related to humans, both highly intelligent, but with very different culture and goals https://www.youtube.com/watch?v=c6Ko0Hzi47U

Post reply on HN