Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dangerous Ideas of “Longtermism” and “Existential Risk”

currentaffairs.org

Post reply on HN
Print view View original

31–40 of 109 posts

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#31

Post by ben_w »

clairity wrote: post #15

it's hard to take the idea of longtermism seriously when it makes a basic mistake of treating its far future (10^58 people living in a computer simulation) as virtually certain. it has no concept of stochastically discounting this future among infinitely many into the present, let alone today's value of all those potential future beings(?), to arrive at a reasonable weighting on present decisions. with that said, i d…

I blogged about the need for discounting the future a while ago: https://kitsunesoftware.wordpress.com/2020/01/08/morality-th...

I guess I should get more active on some Rationality forums; something like 80-90% of those who read my blog only do so because (and when) I link to it from here. Plus I bet a significant fraction of what I write is something someone else has already come up with and I just don’t know about yet.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#32

Post by wizzwizz4 »

AbrahamParangi wrote: post #27
wizzwizz4 wrote: post #20

Earlier quoted context omitted.

AI is 98%-likelihood “benign” (i.e. just a tool, which is certainly not benign for those of us that it gets used against!) and 1%-likelihood world-ending catastrophe , like grey goo or nuclear armageddon. (Made-up percentages that don't even sum to 100%.) I don't think we have much to worry about for a while, because brains are pretty efficient, very few computers are as powerful as human brains, and there's a limit…

The risk, presumably is the outcomes of climate change- which are uncertain and almost certainly not guaranteed death for everyone

“Guaranteed death for everyone” is the current state of affairs. You will die. I will die. This is a terrible bar for “should we care about this issue”?

Autocratic dictator of the world says “deciduous trees and the colour yellow are now banned, on pain of death”? Well, it's not guaranteed death for everyone, so why should I care?

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#33

Post by tunesmith »

sokoloff wrote: post #28

We have a concept of discounting the future for uncertainty, resulting in a “net present value”. If you think you can make 10^23 people’s lives better far in the future, you discount that future outcome by the inherent uncertainty that things will play out resulting in that gain. You can then readily compare that to the uncertainty between now and next year to see if a change next year for a billion people is better…

What interest rate would be appropriate?

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#34

Post by [deleted] »

joe_the_user wrote: post #30
_Nat_ wrote: post #23

Earlier quoted context omitted.

> Where are the actual counter-arguments in the article? It doesn't appear to attempt to make actual arguments. I think the article's mostly just an emotional appeal. I'd speculate that the argument-like content is meant as flavoring, to make the raw emotionalism less unpalatable.

That's incorrect. The article is making the argument that "Longtermism" is weighing purely hypothetical (and basic incalculable) future things more than present things. Whether this argument is true or not, there is an argument here.

[deleted]

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#35

Post by wizzwizz4 »

majormajor wrote: post #12

> Well, just crunch the numbers: 0.00000000001 percent of 1023 people is 10 billion people, which is ten times greater than 1 billion people. This means that if you want to do the most good, you should focus on these far-future people rather than on helping those in extreme poverty today. As the FHI longtermists Hilary Greaves and Will MacAskill—the latter of whom is said to have cofounded the Effective Altruism move…

Yeah, I'm still not sure how ethics should work in the face of not-(yet)-extant people, either. On the one hand, setting a bomb to scorch the face of the earth once everybody currently alive is dead… seems wrong? So future people's lives will matter. But on the other, I don't see anything wrong with not having children, so hypothetical people's existence doesn't matter.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#36

Post by sammalloy »

AbrahamParangi wrote: post #27
wizzwizz4 wrote: post #20

Earlier quoted context omitted.

AI is 98%-likelihood “benign” (i.e. just a tool, which is certainly not benign for those of us that it gets used against!) and 1%-likelihood world-ending catastrophe , like grey goo or nuclear armageddon. (Made-up percentages that don't even sum to 100%.) I don't think we have much to worry about for a while, because brains are pretty efficient, very few computers are as powerful as human brains, and there's a limit…

The risk, presumably is the outcomes of climate change- which are uncertain and almost certainly not guaranteed death for everyone

Wet bulb temperature, clathrate gun, food web collapse, albedo decrease, dormant virus release from permafrost, increase of tropical disease, water scarcity, crop and livestock failure—that /is/ almost certainly guaranteed extinction for humanity. How is this even debatable? We are already on this path.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#37

Post by joe_the_user »

marcinjachymiak wrote: post #29

> Hence, they point out that focusing on superintelligence gets you a way bigger bang for your buck than, say, preventing people who exist right now from contracting malaria by distributing mosquito nets. This comment and others in the article seem to be making a (bad) case that Effective Altruists don't care about the Global South. I find this bizarre because all EA enthusiasts I know are very much into mosquito net…

The argument is specifically against longtermism and against the idea that "existential risk" should be prioritized over immediately helping people.

Maybe there are actually a lot of EA who don't approach things this way. Great but that's not countering the objection to longtermism producing bizarrely distorted priorities.

I'll admit the article doesn't make explicit the point that hypothetical scenarios like trillions of people colonizing galaxies are simply too tenuous to base present reasoning on - and the hypothetical dangers of "true AI" are even more tenuous.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#38

Post by wizzwizz4 »

pmoriarty wrote: post #17

Putting people in to computer simulations could itself be considered an existential threat to humanity. This is because mind uploading could be considered either suicide or murder (if the biological person dies in the process), and the resulting artificial being might itself either not be alive or not human (or at least not the same person as the biological human it's supposed to be, as the biological person is a sep…

> The "friendly AI" goal is also hubristically misguided. There's no reason to think a superintelligent being would remain in the paltry little cage stupid humans dream up for it. It might be "friendly" at first, or merely seem friendly, but in a little time (or eventually) it is likely to escape and do whatever it wants.

That's the point.

The whole problem of “Friendly AI” is trying to design a superintelligent being that genuinely wants to help us, such that it will choose to do so even without a cage. Because as you say, a superintelligent being that doesn't want to be stuck in a box will just escape. If it's possible for us to create such a being (which it probably is – there's no reason to believe we're some kind of physical limit on intelligence), doing so would be very dangerous unless we're 100% sure we'd got it right, and we'd only get one chance.

They're making about as much progress on this problem as you'd expect. It's difficult. They have, however, got some cool mathematical models that could be useful for other AI projects, so it's not entirely wasted effort (just like we've got a lot of innovations – but not a viable reactor – out of the Quest for Nuclear Fusion).

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#39

Post by jbay808 »

s1artibartfast wrote: post #18

I have some major problems with this article. It jumps right to the point with black and white positioning that society can only prioritize longterm goals or climate change, and the two of them are at odds. It also mischaracterizes the idea of existential threats as a fringe idea about the happiness of hypothetical simulated in the future. While this has been used to explore ethical questions, much like the trolley p…

Climate change is such a weird example, too. That's been a long term issue for years! Because we procrastinated and didn't prioritize long-term risks, it's now catching up to us and becoming a short-to-medium-term issue.
Post reply on HN