Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The rational conclusion of doomerism is violence

campbellramble.ai

Post reply on HN
Print view View original

31–40 of 203 posts

Re: The rational conclusion of doomerism is violence

#31

Post by aaroninsf »

Hot take:

I assume the author wrote this with the expectation that much of the readsherp gasp, and react with "the natural horror all right thinking folk would have in response to violence of any kind."

Sorry, lol, no.

The appropriate question for "all right thinking" folk is very different: if argumentation has no impact and it's obvious that it shall have none—what other avenue do you expect opponents, who take the risks seriously, to take...?

That's not a rhetorical question.

To put it bluntly: the machinery of contemporary capitalism, especially as practiced by our industry, very clearly leaves no avenue.

How many days ago was Ronan Farrow here doing an AMA on his critique of Altman—whose connection to this specific community is I assume common knowledge...?

How many of you have carried, or worked beneath, the banner, move fast and break things...?

What message does that ethos convey, about their the extent to which "tech" is going respect community standards, regulation—the law?

And on the other edge: what does this ethos enshrine about how best to accomplish one's aims?

One of the bigger domestic stories this past week which has inflamed a certain side of Reddit, is the "disgruntled employee torches warehouse" one.

Consider also—and I'm deadly serious—the broader frame narrative we are all laboring within today: that the new contract of the capitalist class—including and perhaps especially those in "tech," e.g. in the Peter Thiel circles—seems very much to be, "social stability via surveillance and a police state, rather than through equity and discourse."

When code is law, the law is buggy.

When there is no recourse through the law, you get violence.

Re: The rational conclusion of doomerism is violence

#32

Post by tcoff91 »

irishcoffee wrote: post #19
tcoff91 wrote: post #15

I have a different perspective on this given that I view climate change as the biggest threat we face as a species. I feel like robotics is the only hope we have to be able to scale action against climate change. It's clear that emissions reduction is just not going to happen, and catastrophic warming is inevitable. Therefore we will have to do an extraordinary amount of labor in order to modify our environment to sa…

https://www.howeandhowe.com/civil/thermite The firefighting robots of which you speak already exist.

Hell yeah, those look awesome. I look forward to the autonomous versions that don't require fully manual remote operation. It'd be great if coordinators could have like an RTS-style view and command these like they're starcraft units.

Re: The rational conclusion of doomerism is violence

#33

Post by jmull »

People are basing their entire world view on not understanding the nature of exponential phenomena.

Exponential phenomena only begin in a medium that holds the potential for that phenomena, and necessarily consume that medium.

That is, exponential phenomena are inherently self-limiting. The bateria reaches the edge of the petri dish. When the all the nitroglycerin is broken up the dynamite is done exploding.

That doesn't mean exponential phenomena aren't dangerous -- of course they can be. I mentioned dynamite, after all. And there are nukes.

But it's really far from "AI is improving exponentially now" to "AI will destroy everyone".

I see AI companies consuming cash at unsustainable rates. Since their motive is profit, this is necessarily limiting. Cash, meanwhile is a proxy for actual resources, which have their own, non-exponential limitations -- employees, data centers, electricity, venture capitalist with capital, etc.

AI isn't going to keep improving exponentially -- it can't. Like every other exponential phenomenon, it will consume the medium of potential that supports it (and rather quickly).

Re: The rational conclusion of doomerism is violence

#34

Post by dpark »

squigz wrote: post #26
hax0ron3 wrote: post #7

I don't agree with Yudkowsky, but I think there's certainly a chance that he's right about AI destroying humanity. I just don't think the likelihood of that happening is as high as he thinks it is. But there certainly is a chance. The problem with trying to stop it is, how? Even if you killed every single AI company leader and every single top AI engineer, it would almost certainly just slow down the rate of progress…

> I can think of no example in history of the entire world deciding to just forsake the development of a technology because it seemed like it could prove to be too dangerous. The same psychological logic always applies. Can't you? Haven't many (most?) countries agreed to nuclear disarmament? What about biological weapons? Even anti-personnel mines, I think?

> Haven't many (most?) countries agreed to nuclear disarmament?

This misses the point. He specifically said the entire world because the point is that someone will develop AGI (theoretically; I’m not making a statement about how close we are to this).

9 nations have nuclear weapons despite non proliferation agreements and supposed disarmament. It’s not enough for most countries to agree not to build nuclear weapons if the goal is to have no nuclear weapons. Same for AGI. If it can be developed, you need all nations to agree not to develop it if it don’t want it to exist. Otherwise it will simply be developed by nations that don’t agree with you.

(Also arguably the only reason most nations don’t have nuclear weapons is the threat of destruction from nations that already have them if they try.)

Re: The rational conclusion of doomerism is violence

#35

Post by ACCount37 »

tcoff91 wrote: post #29
derektank wrote: post #17

Earlier quoted context omitted.

Wouldn’t geoengineering through stratospheric aerosol engineering (likely with sulfates) be both cheaper and less technically challenging than changing the built environment? If we’re accepting massive climate changes anyways, it seems like taking the risk with solar radiation modifications would be the next step

That would require global consensus and could ignite wars if there isn't global consensus. Seems very likely that this could have unanticipated consequences that could be worse, but admittedly this is an area I don't really know much about.

No one gives a shit about "global consensus". As demonstrated in 2020s by multiple countries taking major unilateral actions unopposed.

If a nuclear power starts SAI, what is everyone else going to do? Shake their fists at the sky, realistically.

Re: The rational conclusion of doomerism is violence

#36

Post by greenavocado »

jmull wrote: post #33

People are basing their entire world view on not understanding the nature of exponential phenomena. Exponential phenomena only begin in a medium that holds the potential for that phenomena, and necessarily consume that medium. That is, exponential phenomena are inherently self-limiting. The bateria reaches the edge of the petri dish. When the all the nitroglycerin is broken up the dynamite is done exploding. That doe…

> People are basing their entire world view [on things getting worse because their leadership is abandoning them or actively working against their interests]

We understand hard times and are willing to work together to solve problems, but not when leadership is actively harmful.

Fixed that for you.

Re: The rational conclusion of doomerism is violence

#37

Post by morningsam »

hax0ron3 wrote: post #7

I don't agree with Yudkowsky, but I think there's certainly a chance that he's right about AI destroying humanity. I just don't think the likelihood of that happening is as high as he thinks it is. But there certainly is a chance. The problem with trying to stop it is, how? Even if you killed every single AI company leader and every single top AI engineer, it would almost certainly just slow down the rate of progress…

>The technology is pushed forward by a simple psychological logic: every key global actor knows that if they don't build the technology, they will be outcompeted by other actors who do build the technology. No key actor thinks that they have the luxury of not building the technology even if they wanted to not build it.

I don't remember who, but someone made an interesting point about this around the time GPT-4 was released: If the major nuclear powers all understand this, doesn't that make nuclear war more likely the closer any of them get to AGI/ASI? After all, if the other side getting there first guarantees the complete and total defeat of one's own side, a leader may conclude that they don't have anything to lose anymore and launch a nuclear first strike. There are a few arguments for why this would be irrational (e.g. total defeat may, in expectation, be less bad than mutual genocide), but I think it's worth keeping in mind as a possibility.

Re: The rational conclusion of doomerism is violence

#38

Post by arduanika »

tcoff91 wrote: post #15

I have a different perspective on this given that I view climate change as the biggest threat we face as a species. I feel like robotics is the only hope we have to be able to scale action against climate change. It's clear that emissions reduction is just not going to happen, and catastrophic warming is inevitable. Therefore we will have to do an extraordinary amount of labor in order to modify our environment to sa…

Upvoted because this is an interesting take, but I disagree at least somewhat. I think you should be wary whenever you've narrowed down your options to, "in order to solve the top-priority problem X, our only hope is solution Y."

I agree that some technological solution might be the key to dealing with the climate, and that maybe robots would be part of such a solution, maybe powered by similar techniques as the current wave of AI. It's not an insane scenario, but it's worth keeping your perspective open to other possible developments.

Re: The rational conclusion of doomerism is violence

#39

Post by dweinus »

hax0ron3 wrote: post #27
squigz wrote: post #26

Earlier quoted context omitted.

> I can think of no example in history of the entire world deciding to just forsake the development of a technology because it seemed like it could prove to be too dangerous. The same psychological logic always applies. Can't you? Haven't many (most?) countries agreed to nuclear disarmament? What about biological weapons? Even anti-personnel mines, I think?

Those weapons are still all being developed and would be brought out in any actually existential war where they seemed useful. The agreements would last only as long as the wars were not existential, or as long as the various countries involved believed that use of them, and the resulting retaliation in kind, would be more destructive than not using them. But one way or another, countries still develop them.

I don't think it needs to be a binary to be effective. Yes, those weapons still exist, but understanding of existential risk and political pressures have slowed them considerably and resulted in a safer, more cautious world.

Re: The rational conclusion of doomerism is violence

#40

Post by MostlyStable »

Joker_vD wrote: post #11
MostlyStable wrote: post #8

It is completely coherent to both think that an extremely bad thing is coming, and yet that does not justify any particular action. "The ends don't justify the means" and literal entire religions have been built on this concept. It is not irrational or incoherent to believe that even something as serious as extinction does not justify arbitrary action. Someone _may_ decide that it does, but it is not a necessary conc…

> "The ends don't justify the means" Eh. The ends do justify the means, but only inasmuch as those means actually do help to achieve the ends — astonishingly often, they don't (and rarer, but also often, actually bring you in the opposite direction from those end goals), and so remain unjustified.

I personally believe quite strongly that some things are just immoral on their face and that I would rather fail/die without using them than succeed/live while using them. I agree that in very many cases where people do these things, they are, in the long run, counter productive, but I also believe that even if could be conclusively proven that this wasn't the case, I would still advocate against their use.
Post reply on HN