Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Effective Altruism’s obsession with AI safety helps bury bad behavior

bloomberg.com

Post reply on HN
Print view View original

281–290 of 329 posts

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#281

Post by AbrahamParangi »

mitthrowaway2 wrote: post #264
AbrahamParangi wrote: post #211

Earlier quoted context omitted.

I am well aware of the AI doomer argument, but you appear to assume that argumentation can be trusted just because it is convincing*. I believe you are not factoring in the possibility of epistemological error. *for what it's worth I do not find it convincing but I am arguing "even if I found it convincing, I wouldn't believe it"

What's your approach? Do you only believe arguments that sound unconvincing? Or just ignore all arguments, and try to adopt the beliefs that the popular kids in the schoolyard talk about?

The first step would be to recognize the error bounds on epistemological error and redo one’s estimates, but the tl;dr is that one’s certainty on edge case predictions should drop by a lot.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#282

Post by eldenring »

renewiltord wrote: post #99
eldenring wrote: post #72

Earlier quoted context omitted.

what an evil and disgusting comment.

I deserve this since sarcastic comments are really the lowest class of Internet comments.

oh god sorry I didn't even realize. Its a bit sad that nowadays a comment like that can even be mistaken for a serious one.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#283

Post by mitthrowaway2 »

AbrahamParangi wrote: post #281
mitthrowaway2 wrote: post #264

Earlier quoted context omitted.

What's your approach? Do you only believe arguments that sound unconvincing? Or just ignore all arguments, and try to adopt the beliefs that the popular kids in the schoolyard talk about?

The first step would be to recognize the error bounds on epistemological error and redo one’s estimates, but the tl;dr is that one’s certainty on edge case predictions should drop by a lot.

Having low certainty on edge case predictions is fine, but it also should result in correspondingly larger updates when those edge-case predictions come true, which one had previously doubted. For me, that was the case with AlphaGo, AlphaFold, and now ChatGPT. In all three cases I was highly skeptical that, in my lifetime, AI would ever beat humans at Go, adapt the same architecture to problems like protein folding, and blow the Turing Test right out of the water.

I've had to update accordingly, and now I'm less skeptical that the barriers ahead will be harder to break than the ones behind us.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#284

Post by mitthrowaway2 »

fidgewidge wrote: post #277
mitthrowaway2 wrote: post #266

Earlier quoted context omitted.

In terms of timeline, the key sentence would be "The so-called thermal extinction at the end of the Paleocene began when atmospheric CO2 was just under 1,000 parts per million (ppm)". The end-Permian was likely around 2500 ppm[1]. It doesn't really matter whether that carbon comes from supervolcanoes or from human emissions. When the article was written, CO2 concentration was 385 ppm. Today it's 421 ppm. If CO2 conce…

"It doesn't really matter whether that carbon comes from supervolcanoes or from human emissions." It matters a lot. Whilst I was eating dinner I've now read the article you cited, the paper that it cites (Kump, Arthur & Pavlov, 2015). I also cross checked the argument and numbers against several other papers. Ward's argument is extremely slippery. This is sadly what I'm coming to expect from academics with unverifiab…

The solubility of gasses in water decreases with rising temperature. For ocean absorption of O2 from the atmosphere, the surface temperature is particularly important. This is why even if atmospheric O2 does not decrease, this could still be a concern -- albeit one with a different threshold than a scenario where the ocean warms and atmospheric O2 concentrations decrease at the same time.

Still, I certainly do hope that you're correct! And thank you for the thoughtful and well-considered reply!

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#285

Post by arisAlexis »

pksebben wrote: post #261
AgentME wrote: post #167

Earlier quoted context omitted.

A recent poll of published AI researchers showed a median estimate of a 5% chance AI causes human extinction, with half believing the chance is greater than 10%. https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ A poll of NLP researchers showed 37% agreed that catastrophic outcomes of AI comparable to nuclear war were plausible. https://arxiv.org/abs/2208.12852

it's a little annoying that the "AI Impacts" paper seems to be three MIRIs in a trenchcoat, considering that this article is about EA's AI agenda and by extension Singularity/MIRI. There is a slight conflict of interest here - as the publishers of the paper are also taking funding specifically to "deal with the threat" they're publishing on.

Yes if we have scientists that are getting paid to research a certain risk it’s inevitable that they have conflict of interest. Otherwise how would it be done?

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#286

Post by postultimate »

jasonhansel wrote: post #19

Reminder that one of EA's core supporters is a fan of scientific racism: https://twitter.com/ArsonAtDennys/status/1362153191102677001 For some more context: https://rationalwiki.org/wiki/Scott_Alexander#Race_and_IQ

Good for him. Nobody's racism should be unscientific.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#287

Post by simple-thoughts »

mitthrowaway2 wrote: post #257
simple-thoughts wrote: post #145

Earlier quoted context omitted.

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

> The orthogonality thesis is an unproven assumption A safety mindset would suggest that rather than disregarding it until proven true, we should worry about it until proven false. > meaning an intelligent being can pursue stupid goals. It would pursue very intelligent instrumental goals , but the terminal goal is a free variable and I don't think there exists any measure by which terminal goals can be considered sma…

This “safety mindset” is the same as Pascal’s wager and has the same flaws.

There’s no clear delineation in real entities between instrumental and terminal goals.

Yeah if you’re talking to a brainwashed religious fanatic, definitely it can seem counterintuitive to the doctrines they believe.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#288

Post by simple-thoughts »

ShredKazoo wrote: post #250
simple-thoughts wrote: post #145

Earlier quoted context omitted.

The ai safety arguments rely on something called the “orthoganality thesis” which is a huge, unintuitive assumption. In real life, intelligence is associated with picking better goals. Generally, an entity with higher intelligence will pick different goals than a being with lower intelligence. The orthoganility thesis is an unproven assumption that intelligence and goals are not corellated, meaning an intelligent bei…

>In real life, intelligence is associated with picking better goals. "Better" according to what metric? It may be the case that there is a tendency for high-intelligence humans to pick "more enlightened" goals. Perhaps there is a natural "enlightened goals" attractor for our species. However I don't think we can extrapolate from that to a fundamentally alien AI. I think even if this statistical tendency exists, it ha…

> “Better” by what metrics?

As soon as you phrase goal selection in terms of metrics, you’re assuming that goal selection is based on some other goal - that is you’re already assuming the orthoganality thesis. Your logic is fully circular.

One thing that’s interesting to note about all of the examples you picked - every single one of those species shows cooperative behaviors. They share many other behaviors that are more similar than they are different. To reject the orthoganality thesis it’s sufficient to show that there is an empirical general association between intelligence and certain goals - then we can extrapolate an AI although of course it will function differently and may have many unusual behaviors will tend to follow those goals more directly. For instance intelligence is associated with : cooperation, empathy, inter and intra species communication, curiosity, etc. all of the species you mentioned exhibit these more than less intelligent species. Meanwhile something like “hunting to eat” is observed across the intelligence spectrum.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#289

Post by low_tech_love »

esperent wrote: post #70
JamesBarney wrote: post #50

Earlier quoted context omitted.

There have been people for a long time who think the world is going to end. We can use your argument to dismiss anyone who fears any existential risk like asteroids, nuclear war, climate change, AI, etc... And since the world exists every group will be wrong except one. I think the risk of human extermination is much lower than many in this group believe because I think AI is much more compute bound than algorithm bo…

The issue is that the effective altruism crowd are focused exclusively on AI safety, at least in what they publish online. If they talked about all possible threats to humanity and how to solve them, from AI, to climate change, to nuclear bombs, to asteroids, to disease, they would be an interesting and important group. They don't. They hardly talk about anything else except AI. From the outside it looks a lot like a…

AI safety and "longtermism" (I feel slightly more stupid just by writing this) is an escape route for the group's realization that (a) they cannot really make a difference and (b) even if they could, it would require a lot of actual work. The person who goes out in the middle of a cold night and cooks soup for homeless people can quite easily count how many mouths he fed; sounds quite effective and quantitative to me. Or maybe the volunteer who is helping fight malaria in a desert somewhere outside of America/Europe; actually, they don't need to count, their exhaustion at the end of a hard day is proof enough of their altruism.

But since the real goals of the EAs are not to do any of that (it's just a bunch of socially-awkward geeks looking to feel accepted in some kind of community and find sexual partners; in other words, trying to feel like normal humans), they fooled themselves into believing that such short term actions are "ineffective"; actually, any actual measurable hard work has to necessarily be ineffective, or else it would actually have to be done. So they keep pushing their agenda to longer and longer term issues, more abstract and less well-defined as possible, which cannot be actually measured, so they can continue to build this weird, meaningless, ridiculous fantasy that they live in (and which supports their social hierarchy) without having to actually do anything concrete.

Re: Effective Altruism’s obsession with AI safety helps bury bad behavior

#290

Post by low_tech_love »

dotsam wrote: post #158
pksebben wrote: post #143

Earlier quoted context omitted.

didn't really have much of a dog in this fight, but looking at that breakdown I wonder which oil company is bankrolling them. I know I'm jumping the gun just a little, but c'mon. An apocalypse list without climate?

Climate change is unlikely to kill everyone on earth. Other risks, such as an engineered pandemic, asteroid impact, or an AI apocalypse have the possibility of killing ~everyone. This is not saying that climate change is not a real issue.

Says whose crystal ball?
Post reply on HN