Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

231–240 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#231

Post by scarmig »

mb7733 wrote: post #201
roenxi wrote: post #103

Earlier quoted context omitted.

Corporations, when first developed, took out large chunks of Asia before anyone could stop them. The main difference is that corporations have to be powered by people while an AI could theoretically work with complete independence.

> Corporations, when first developed, took out large chunks of Asia before anyone could stop them. What do you mean by this?

British East India Company, probably?

At some point it had more soldiers and controlled a greater subject population than the government of Britain itself.

I'm not sure it was ever consistently profitable, though, which is the main paperclip corporations ostensibly maximize. Which itself speaks to the problem: superficial objective functions don't always determine outcomes or behavior.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#232

Post by HDThoreaun »

aetherson wrote: post #151
Natsu wrote: post #83

Earlier quoted context omitted.

> The issue is that the effective altruism crowd are focused exclusively on AI safety. How many EA people do you know? That seems to get the most clicks, but I usually associate them with things like mosquito bed nets to help prevent malaria, because that's a far more common topic in my experience. I'm sure you can find particular people who are all about the AI risk stuff that seems pretty wild to me, too. But when…

As someone who is not an EA, but has been peripherally aware of them for years, the bed nets stuff was their big thing until maybe three years ago, but it genuinely is the case that a lot of their energy had recently redirected to AI safety and other X-risks. That isn't to say that the malaria stuff isn't still a big topic, but there has been a shift.

Givewell is still where most EA money flows and they are almost exclusively focused on mosquito nets and other malaria intervention.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#233

Post by HDThoreaun »

Vecr wrote: post #74
fshbbdssbbgdd wrote: post #69

Earlier quoted context omitted.

Personally, I find the AI safety arguments sound pretty far-fetched, but I can’t come up with solid arguments against them. Also, when I look for a good ”debunking”, I don’t find anything. Mostly all I find are mockery, ad-hominem and the like. Frankly, the people who argue climate change isn’t real do a better job than the people who argue AI risk isn’t real, and they’re wrong!

Yeah, I can't really argue against the whole idea. My "P(doom)"(language from the article), at least as related to agentic AI, is pretty low. About 7 or 8 percent before I die, and I'm not really old. There's probably not a lot I could do about it anyway, given that most AI jobs are alleged to just make it worse. That said, I can't really go around telling people with different numbers they should be working on antiv…

7 or 8 percent chance of apocalypse is low?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#234

Post by scarmig »

dr_dshiv wrote: post #48
xkcd-sucks wrote: post #42

Earlier quoted context omitted.

2. "An ounce of prevention is worth a pound of cure"; "measure twice, cut once"; "it's easier to fuck it than to unfuck it"; etc. Not exactly novel

To clarify— I mean the rationalist reference to angels and demons. It seems to be more than metaphorical.

Angel here would be an "aligned" AI: an AI whose behavior is in line with human values. A demon is simply an unaligned one.

The key point would be that it needs not be actively hostile to human values, but indifferent to them. Think of a Lovecraftian Elder God, not AI Hitler.

The supporting hypotheses would be that it's much easier to create an unaligned AI than an aligned one, as an aligned one is an unaligned one with additional constraints, and that unaligned AIs will wreak massive destruction.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#235

Post by HDThoreaun »

pksebben wrote: post #156
karaterobot wrote: post #15

Earlier quoted context omitted.

Harry Potter and the Methods of Rationality . I'm glad such a thing exists, but I could never work up the interest to read it myself. https://en.wikipedia.org/wiki/Harry_Potter_and_the_Methods_o...

wait, really? I scrolled right past someone posting about "Harry Potter rationality fanfiction" and thought, "huh, oddly specific burn but it's chuckle worthy". it's an actual thing ? This is a joke, right? tongue in cheek or something. Or should I go get some Brawndo to water my basil...

It's not only a thing, it's almost 2000 pages long.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#236

Post by scotty79 »

JamesBarney wrote: post #50
AbrahamParangi wrote: post #32

AI safety is an apocalypse cult and we should recognize that everyone who has ever been in an apocalypse cult thinks that "no, this time it's really the end of the world". Everyone who believes crazy shit thinks they're being reasonable, so just because we're super sure we're being reasonable doesn't mean what we believe isn't crazy. Do we believe that we alone, of all people who have believed in the coming end, have…

There have been people for a long time who think the world is going to end. We can use your argument to dismiss anyone who fears any existential risk like asteroids, nuclear war, climate change, AI, etc... And since the world exists every group will be wrong except one. I think the risk of human extermination is much lower than many in this group believe because I think AI is much more compute bound than algorithm bo…

> And since the world exists every group will be wrong except one.

Except at most one.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#237

Post by HDThoreaun »

Vecr wrote: post #92
5cott0 wrote: post #71

what’s so bad about reading some Harry Potter fanfic in the polycule?

I'm assuming you read it, so why does Harry Potter (a 11 year old in 1991/1992) think polyamory might (or probably would be, looking at the text right now) be a good idea in a future where people live a long time, in said fanfic? I think it's pretty dubious, but the other 11 and 12 year old girls in the same fanfic also seem to be fans of the idea of yaoi like situations, so all semblances of actual historical events…

Harry has voldemort's soul in him the HPMOR which is the explanation given for why he acts like an evil adult.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#238

Post by LatteLazy »

PartiallyTyped wrote: post #215
LatteLazy wrote: post #182

Earlier quoted context omitted.

According to a survey 700 ML researchers, we should throw money at their subject or we might all die...

With great power comes great responsibility. If we start plugging ML everywhere - which we are currently doing - we better be sure that it is behaving as expected.

So far we have seen no power at all from ML. So spending large amounts of money making it "safe" (not that anyone knows how or even what it would look like) seems like an over reaction.

With no power, comes no responsibility.

But of course, if ML is a person's pet interest, they will likely think we should increase spending on it. That doesn't make that a smart decision...

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#239

Post by dotsam »

esperent wrote: post #223
dotsam wrote: post #158

Earlier quoted context omitted.

Climate change is unlikely to kill everyone on earth. Other risks, such as an engineered pandemic, asteroid impact, or an AI apocalypse have the possibility of killing ~everyone. This is not saying that climate change is not a real issue.

So what you're saying is that if a risk can totally upend our society, destroy most of our cities, make most of our farmland unusable, destabilize geopolitics in a way that's almost certain to lead to war between nuclear powers (who also have the ability to engineer deadly diseases), and massively disrupt every ecosystem on earth, and that there's enough evidence to say with almost complete certainty that this will c…

The cascading risks you mention are certainly real and serious, and are worthy of our best and urgent efforts to solve. Effective altruists are rightly concerned about these effects e.g. https://80000hours.org/problem-profiles/climate-change/

My comment was written with the summary of that article in mind (I didn't make this clear):

> Climate change is going to significantly and negatively impact the world. Its impacts on the poorest people in our society and our planet’s biodiversity are cause for particular concern. Looking at the worst possible scenarios, it could be an important factor that increases existential threats from other sources, like great power conflicts, nuclear war, or pandemics. But because the worst potential consequences seem to run through those other sources, and these other risks seem larger and more neglected, we think most readers can have a greater impact in expectation working directly on one of these other risks.

There is an excellent chapter on the existential risks associated with climate change in Toby Ord's book The Precipice, which you can get a free copy of at https://80000hours.org/the-precipice/

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#240

Post by naasking »

esperent wrote: post #70
JamesBarney wrote: post #50

Earlier quoted context omitted.

There have been people for a long time who think the world is going to end. We can use your argument to dismiss anyone who fears any existential risk like asteroids, nuclear war, climate change, AI, etc... And since the world exists every group will be wrong except one. I think the risk of human extermination is much lower than many in this group believe because I think AI is much more compute bound than algorithm bo…

The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. If they talked about all possible threats to humanity and how to solve them, from AI, to climate change, to nuclear bombs, to asteroids, to disease, they would be an interesting and important group. They don't. They hardly talk about anything else except AI. From the outside it looks a lot like a…

> They don't. They hardly talk about anything else except AI.

Even if that were true, there are lots of people already looking at those other existential risks. The EA crowd was the only group concerned about AI for a long, long time. Seeing an existential risk that nobody else does leads to evangelism and hyper focus so people start paying attention.

Post reply on HN