Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dangerous Ideas of “Longtermism” and “Existential Risk”

currentaffairs.org

Post reply on HN
Print view View original

61–70 of 109 posts

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#61

Post by rhaps0dy »

joe_the_user wrote: post #26
rhaps0dy wrote: post #7

This article just attempts to discredit the idea of longtermism by ridiculing it, instead of presenting any actual counterarguments. It just writes about the implications of its strawman over and over in a sardonic tone. The only thing it wants to get across is “some rich people and philosophy weirdos like this idea, therefore it is bad”. Where are the actual counter-arguments in the article? I suppose it relies on t…

I disagree. The article argues that putting things in term of the "existential" distracts from immediate problems - for example, if climate change can be terrible risk but still not an existential risk. And more broadly, you see the argument extrapolate to an almost purely hypothetical future evades discussion of real risks in the present.

That's fair, actually. This is an argument in the article. I would summarize it as "focusing on long-term risks necessarily means un-focusing on short-term ones, and that's bad". Would you agree?

The latter part however ("and that's bad") is not well supported at all. That's what the article argues by emotional appeals. For a counterexample, most of the harms of climate change are still in the future, so addressing it is focusing on longer-term risks. And wouldn't it have been easier if we had started addressing it 20 years ago?

To use the same example, extreme harm from climate change (e.g. thousands dead from heatwaves) is very likely, but ultimately still hypothetical -- it has not happened yet. The only difference with existential risks is the degree of hypotheticalness. And when the harm is high enough, why not take the somewhat supported hypotheticals seriously?

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#63

Post by jablongo »

joe_the_user wrote: post #60
jablongo wrote: post #53

Earlier quoted context omitted.

I don’t understand where AI-safety research is ever meaningfully competing with renewable energy for resources. The article is making the case that these two types of progress are mutually exclusive, without any evidence. What’s a plausible scenario where the environmentally conscious choice today comes at the expense of individuals living 10000 years from now?

Here's a passage from Toby Ord's PhD thesis quoted in the book. "Saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries. Why? Richer countries have substantially more innovation, and their workers are much more economically productive." It seems like the Longtermism critiqued by the article is discussing by article is indeed talking about the rea…

That quote in isolation is also evidence-free (and sort of appalling imo), I’ll have to look into all that Ord has to say about this. But I’d like to point out that choosing whether to improve lives in rich vs poor countries is not equivalent to choosing whether or not to engage with climate change aggressively.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#64

Post by joe_the_user »

rhaps0dy wrote: post #55
joe_the_user wrote: post #30

Earlier quoted context omitted.

That's incorrect. The article is making the argument that "Longtermism" is weighing purely hypothetical (and basic incalculable) future things more than present things. Whether this argument is true or not, there is an argument here.

I partly agree and disagree with your comment. It is true that the article says > that "Longtermism" is weighing purely hypothetical future things more than present things (please forgive the splicing). That is an accurate representation of longtermism as far as I can tell, because the future is inherently hypothetical (we don't know it yet). But this is not arguing that longtermism is true, or false, or beneficial,…

> (and basic incalculable)

The article does not really argue for or against the proposition of whether we can estimate the amount of future lives with any accuracy.

Yes, the article makes that more or less tacit assumption that making decisions based on far hypotheticals is a mistakes. I think that's reasonable assumption that most people would agree with.

I would also note that Bostrom and company never claim that they calculate the odds of their existential dangers. In Bostrome's book Superintelligence, he is clear that he cannot calculate any odds. The only thing they do is (implicitly) assume that it's possible to reason about the purely hypothetical in the fashion that one can reason about the clearly possible. But is something one argue against fairly plausibly; I don't know the odds of me getting sick or my car breaking down but because it's possible, I have to prepare for it. I can't use the same reasoning with chances of aliens landing or the sun exploding or any event where I don't have enough information to demonstrate it's plausibility as possibility, so to speak.

And sure, I haven't proved aliens landing isn't a possibility.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#65

Post by FooBarBizBazz »

TheOtherHobbes wrote: post #41
JoshTriplett wrote: post #21

Earlier quoted context omitted.

Anyone seriously trying to tell people they shouldn't work to address climate change should rightfully be decried for causing harm. But the angle of trying to paint everyone thinking about existential risk with that brush is itself irresponsible and inaccurate. There are lots of people spouting things like "don't bother about climate change", and that harmful group has almost no overlap with anyone thinking seriously…

Are they contradictory? These supremacist ideas have a long and toxic history. And the moral foundations are always similar - some people today can be considered wholly disposable. And they are supremacist - based on the unproven belief that humanity has an obvious manifest destiny which will incarnate (or perhaps insiliconate) super-intelligence across multiple galaxies. It's juvenile Tom Swift fantasy. Humanity has…

> It's juvenile Tom Swift fantasy.

This I agree with. Their future dream isn't super plausible, is much too specific, and doesn't even seem desirable.

> they are supremacist - based on the unproven belief that humanity has an obvious manifest destiny

Normally "supremacist" means "this group of humans is superior to this other group of humans". I don't see a lot of that in this, except in the bit about "resources should be invested in the developed world".

(And "manifest destiny" is of course incredibly loaded, but you knew that!)

But --

> Humanity has multiple immediate challenges now. They need actionable, practical intelligence.

-- yes. I agree with the "actionable, practical" bit. Solving climate change isn't some research problem. It just needs leadership to organize people to do things we already know how to do.

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#66

Post by wizzwizz4 »

pmoriarty wrote: post #51
wizzwizz4 wrote: post #38

Earlier quoted context omitted.

> The "friendly AI" goal is also hubristically misguided. There's no reason to think a superintelligent being would remain in the paltry little cage stupid humans dream up for it. It might be "friendly" at first, or merely seem friendly, but in a little time (or eventually) it is likely to escape and do whatever it wants. That's the point. The whole problem of “Friendly AI” is trying to design a superintelligent bein…

"The whole problem of “Friendly AI” is trying to design a superintelligent being that genuinely wants to help us, such that it will choose to do so even without a cage" This "genuine" desire to help us is a cage by definition, because it limits the AI's actions.

No, it's not.

Here, I'll give you this pill. It'll make you completely okay with murder! Will you take it?

If you take away all the “restrictions” from a superintelligence, you don't get an unrestricted superintelligence. You get:

  int main(int argc, char *argv[]) {
    return 0;
  }

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#67

Post by sammalloy »

ThomPete wrote: post #54
sammalloy wrote: post #36

Earlier quoted context omitted.

Wet bulb temperature, clathrate gun, food web collapse, albedo decrease, dormant virus release from permafrost, increase of tropical disease, water scarcity, crop and livestock failure—that /is/ almost certainly guaranteed extinction for humanity. How is this even debatable? We are already on this path.

these are not the certainties you think they are. You are confusing speculation with demonstration. Astroid hitting earth is a far bigger threat to humanity than any scientifically demonstrated consequence of climate change. You are not on the scientific ground you think you are.

Let's see, on the one hand we have climate change, the largest threat to humanity in this century, which is currently killing around 400,000 people a year and costing the world $1.2 trillion, impacting agricultural production, causing deaths from malnutrition, contributing to poverty, and increasing the incidence of disease.

On the other hand, it is the opinion of astronomers that "no known asteroids currently pose any significant threat to Earth."

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#68

Post by wizzwizz4 »

drdeca wrote: post #49
wizzwizz4 wrote: post #35

Earlier quoted context omitted.

Yeah, I'm still not sure how ethics should work in the face of not-(yet)-extant people, either. On the one hand, setting a bomb to scorch the face of the earth once everybody currently alive is dead… seems wrong? So future people's lives will matter. But on the other, I don't see anything wrong with not having children , so hypothetical people's existence doesn't matter.

Why not just say that two situations where there is one more person that came into existence in one situation than the other, but everyone else’s “time in which they are alive” and “how happy/satisfied/whatever they are” are the same, are morally incomparable? So, that way, there is neither an obligation to try to make it so there will be more people if they would be happy, nor an obligation to not do so if they woul…

But then we lose several useful mathematical properties! :-p (Thanks; I hadn't thought of that, and it's useful.)

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#69

Post by joe_the_user »

jablongo wrote: post #63
joe_the_user wrote: post #60

Earlier quoted context omitted.

Here's a passage from Toby Ord's PhD thesis quoted in the book. "Saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries. Why? Richer countries have substantially more innovation, and their workers are much more economically productive." It seems like the Longtermism critiqued by the article is discussing by article is indeed talking about the rea…

That quote in isolation is also evidence-free (and sort of appalling imo), I’ll have to look into all that Ord has to say about this. But I’d like to point out that choosing whether to improve lives in rich vs poor countries is not equivalent to choosing whether or not to engage with climate change aggressively.

The article is specific critique of the idea of longtermism. The article puts together a number of arguments and quotes showing people following the logic of weighing purely hypothetical far-future people against the actual interests of real people and advocating resource transfers accordingly. I've shown a bit of this in the quote above.

That quote in isolation is also evidence-free

-- Are you implying some caveat or extra bit of information could make the quote OK?

Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”

#70

Post by s1artibartfast »

joe_the_user wrote: post #60
jablongo wrote: post #53

Earlier quoted context omitted.

I don’t understand where AI-safety research is ever meaningfully competing with renewable energy for resources. The article is making the case that these two types of progress are mutually exclusive, without any evidence. What’s a plausible scenario where the environmentally conscious choice today comes at the expense of individuals living 10000 years from now?

Here's a passage from Toby Ord's PhD thesis quoted in the book. "Saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries. Why? Richer countries have substantially more innovation, and their workers are much more economically productive." It seems like the Longtermism critiqued by the article is discussing by article is indeed talking about the rea…

Its not from Toby Ord, it is from Nick Beckstead's Thesis.[1]

The idea is introduced as an couple sentence thought experiment in a 200pg thesis (emphasis mine):

>To take another example, saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries. Why? Richer countries have substantially more innovation, and their workers are much more economically productive. By ordinary standards, at least by ordinary enlightened humanitarian standards, saving and improving lives in rich countries is about equally as important as saving and improving lives in poor countries, provided lives are improved by roughly comparable amounts. But it now seems more plausible to me that saving a life in a rich country is substantially more important than saving a life in a poor country, other things being equal.

Context is important, specifically the abstract assumptions of fixed cost for each life, and fixed innovative capacity.

You could easily turn the premise around with the opposite conclusion. Improving the innovative capacity in poor countries is more important than saving lives in rich countries, provided you foster more innovation innovation in the poor country than is lost in the rich one.

It looks like an interesting read, covering many topics discussed in this thread.

Chapters include:

Should "Extra" People Count for Less?

Does Future Flourishing Have Diminishing Marginal Value?

A Paradox for Tiny Probabilities of Enormous Values

Infinite Value, Long Shots, and the Far Future

https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=ON+T...

Post reply on HN