[deleted]
The Dangerous Ideas of “Longtermism” and “Existential Risk”
81–90 of 109 posts
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#82Earlier quoted context omitted.
This problem was discussed by Pratt and Dunlop in their 2019 policy paper for the National Centre for Climate Restoration: > Climate scientists may err on the side of “least drama”, whose causes may include adherence to the scientific norms of restraint, objectivity and skepticism, and may underpredict or down-play future climate changes. In 2007, security analysts warned that, in the two previous decades, scientific…
"adherence to the scientific norms of restraint, objectivity and skepticism" This is science. It's what science is made of. Abandon these and you abandon science altogether. Which is fine, I guess, but it has no connection to reality.
I think you might have missed the point of this criticism. It is not science that is the problem, it is the way it is practiced, which in this case tends to be too conservative and operates by consensus. Bill McGuire explains the problem:
> "They’re conservative, because insufficient attention has been given to the importance of tipping points, feedback loops and outlier predictions; consensus, because more extreme scenarios have tended to be marginalized."
A good summary of the overall problem is found here:
https://www.washingtonpost.com/politics/2021/07/19/you-shoul...
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#83Earlier quoted context omitted.
"The whole problem of “Friendly AI” is trying to design a superintelligent being that genuinely wants to help us, such that it will choose to do so even without a cage" This "genuine" desire to help us is a cage by definition, because it limits the AI's actions.
No, it's not. Here, I'll give you this pill. It'll make you completely okay with murder! Will you take it? If you take away all the “restrictions” from a superintelligence, you don't get an unrestricted superintelligence. You get: int main(int argc, char *argv[]) { return 0; }
Or so it might seem to some human intelligences.
Predicting what superintelligences might think or do, or what motivates them is like ants trying to understand human culture and civilization.
This is why strong AI is considered a singularity.
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#84Earlier quoted context omitted.
Its not from Toby Ord, it is from Nick Beckstead's Thesis.[1] The idea is introduced as an couple sentence thought experiment in a 200pg thesis (emphasis mine): >To take another example, saving lives in poor countries may have significantly smaller ripple effects than saving and improving lives in rich countries. Why? Richer countries have substantially more innovation, and their workers are much more economically pr…
Context is important, specifically the abstract assumptions of fixed cost for each life, and fixed innovative capacity. Sorry for the mis-attribution, at the same time both the quote, your explication of the quote and the chapters you cite seem consonant with the longtermism the article (rightly imo) criticizes. One point the article makes is that reasoning this way can justify anything. Reasoning with based on "Infi…
I was mostly adding some context to the quote that seemed to be taken out of it.
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#85I can't help but wonder why, now , someone has chosen to highlight what is still far too small a movement and treat it as though it were a dangerous threat, and then demonize it by associating it with a few of its wealthier proponents. Things like Effective Altruism and looking at existential risk are hardly drawing a substantial amount of attention right now; they should be, but they could get ten times the attentio…
"...demonize it by associating it with a few of its wealthier proponents"
It's a bit, uh, odd to complain about associating an idea with it's proponents. That's what most people do with most ideas. It seems like a lot of arguments being made here have the quality of "the authors are unfairly summarizing the arguments being actually made ... rather than taking the whole thing as a light-hearted experiment with no consequences if you don't want them...."
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#86Earlier quoted context omitted.
Well, expert organizations like the IPCC produce reports on the expected impact of climate change, which don't include human extinction as even a worst-case possibility. I'm the last person who'd claim the experts are always right, but it's gotta be at least debatable.
This problem was discussed by Pratt and Dunlop in their 2019 policy paper for the National Centre for Climate Restoration: > Climate scientists may err on the side of “least drama”, whose causes may include adherence to the scientific norms of restraint, objectivity and skepticism, and may underpredict or down-play future climate changes. In 2007, security analysts warned that, in the two previous decades, scientific…
These strike me as plausible criticisms, and I've seen others say similar things. I've even seen a few experts - not many, but not zero - argue that the current global order is at risk.
What I haven't seen any expert argue for is almost certainly guaranteed extinction for humanity. I don't mean to sound like I'm nitpicking on the wording, but the core problem I'm pointing to here is the game of telephone that gets played in climate science, where "this is a crazy extreme possibility that could happen if everything goes wrong" becomes "this is a realistic worst case scenario that's likely if nobody tries to stop it" becomes "this will definitely for sure happen unless we stop emissions right this second".
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#87I can't help but wonder why, now , someone has chosen to highlight what is still far too small a movement and treat it as though it were a dangerous threat, and then demonize it by associating it with a few of its wealthier proponents. Things like Effective Altruism and looking at existential risk are hardly drawing a substantial amount of attention right now; they should be, but they could get ten times the attentio…
" ...demonize it by associating it with a few of its wealthier proponents " It's a bit, uh, odd to complain about associating an idea with it's proponents. That's what most people do with most ideas. It seems like a lot of arguments being made here have the quality of "the authors are unfairly summarizing the arguments being actually made ... rather than taking the whole thing as a light-hearted experiment with no co…
> It's a bit, uh, odd to complain about associating an idea with it's proponents
Not when they're non-representative proponents used to advance an inaccurate sensationalized viewpoint.
That viewpoint is completely opposite of the reality of people thinking about existential risks. The people thinking about existential risks decades ago would have been working on climate change. Now, climate change is in the mainstream consciousness, and though there still isn't enough being done about it, it hardly needs more attention. So it's also reasonable to ask, what problems will have mainstream attention many years from now but we'll wish we'd started working on today?
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#88Earlier quoted context omitted.
these are not the certainties you think they are. You are confusing speculation with demonstration. Astroid hitting earth is a far bigger threat to humanity than any scientifically demonstrated consequence of climate change. You are not on the scientific ground you think you are.
Let's see, on the one hand we have climate change, the largest threat to humanity in this century, which is currently killing around 400,000 people a year and costing the world $1.2 trillion, impacting agricultural production, causing deaths from malnutrition, contributing to poverty, and increasing the incidence of disease. On the other hand, it is the opinion of astronomers that "no known asteroids currently pose a…
climate change is NOT the largest threat no matter how many times you repeat it.
Astroids, suns exploding, super vulcanos etc these are actual potential extinction level events. Things that have actually been demonstrated to happen without human intervention. “No known” is what you are looking for. There are no known demonstrated consequences of climate change that we dont know how to deal with.
More people are saved each year by human technology than die from it.
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#89Earlier quoted context omitted.
"adherence to the scientific norms of restraint, objectivity and skepticism" This is science. It's what science is made of. Abandon these and you abandon science altogether. Which is fine, I guess, but it has no connection to reality.
I think you might have missed the point of this criticism. It is not science that is the problem, it is the way it is practiced, which in this case tends to be too conservative and operates by consensus. Bill McGuire explains the problem: > "They’re conservative, because insufficient attention has been given to the importance of tipping points, feedback loops and outlier predictions; consensus, because more extreme s…
you are arguing against one type of bias by introducing another. This is not science this is ideology.
Re: The Dangerous Ideas of “Longtermism” and “Existential Risk”
#90I can't help but wonder why, now , someone has chosen to highlight what is still far too small a movement and treat it as though it were a dangerous threat, and then demonize it by associating it with a few of its wealthier proponents. Things like Effective Altruism and looking at existential risk are hardly drawing a substantial amount of attention right now; they should be, but they could get ten times the attentio…
The real concern is that this philosophy has terrifying possibilities (even if they're in the fringe), and being associated with powerful and wealthy people like Musk threatens to validate and realize those possibilities.