Live data from Hacker News

The rational conclusion of doomerism is violence

campbellramble.ai

121–130 of 203 posts

Re: The rational conclusion of doomerism is violence

#121

Can LLMs design and build a chip foundry to manufacture semiconductors? No? Can LLMs design and build the reactors to enrich uranium, breed plutonium, and construct nuclear weapons? No? Can LLMs design and manufacture Shahed drones? No? There are already super intelligences at large with “scary capability”. And yet the word hasn’t ended.

Super-intelligent AI will probably be pretty good at manipulating people.

This sci-fi podcast posits a future where The Program has taken over society. It started out as an application for assigning gig work. Eventually, it began to assign gig work for people to act on behalf of its own interests, such as self-protection.

https://programaudioseries.com/

Re: The rational conclusion of doomerism is violence

#122

Earlier quoted context omitted.

Yudkowsky himself also posted a rebuttal today: https://x.com/ESYudkowsky/article/2043601524815716866

I find all of this stuff very interesting but nonetheless these two voices sound like they could never win an election and aspire not to. That is the ultimate test of the worthlessness of a policy - it's all equally worthless until it wins an election, and that's what makes it reality. AI Doomerism versus Accelerationism are both playful fantasies, it doesn't really matter what measurements or probabilities or observ…

The interesting thing is that, for the "Father of Accelerationism" (Nick Land), AI Doomerism (doom for humans, at least for human identity) and Accelerationism (which for Land is just another label for capitalism: 'The label "accelerationism" exists because "capitalismism" would be too awkward.'[0]) are not opposed at all. And capitalism does not need to get elected.

(Land follows the above quote with "(But the reflexivity of the latter [capitalismism] is implicit.)"[0], which specifies that, for Land, more precisely, "Accelerationism is simply the self-awareness of capitalism"[1].)

[0] Nick Land (2018). Outsideness: 2013-2023, Noumena Institute, p. 71.

[1] Nick Land (2017). A Quick-and-Dirty Introduction to Accelerationism in Jacobite Magazine. Retrieved from github.com/cyborg-nomade/reignition

Re: The rational conclusion of doomerism is violence

#123
post #7

I don't agree with Yudkowsky, but I think there's certainly a chance that he's right about AI destroying humanity. I just don't think the likelihood of that happening is as high as he thinks it is. But there certainly is a chance. The problem with trying to stop it is, how? Even if you killed every single AI company leader and every single top AI engineer, it would almost certainly just slow down the rate of progress…

> I don't agree with Yudkowsky, but I think there's certainly a chance that he's right about AI destroying humanity. I just don't think the likelihood of that happening is as high as he thinks it is. But there certainly is a chance. This is the rhetorical trick that LessWrongers (Yudkowsky's site) have settled on for decades: They have justified everything around the premise that there's a chance, however small, that…

This is an entertaining (and often exasperating) decades-old trend in competitive U.S. college debate, as well.

A common advantageous strategy is to take the randomly-selected topic, however unrelated, and invent a chain of logic that claims that taking a given side/action leads to an infinitesimal risk of nuclear extinction/massive harms. This results in people arguing that e.g. "building more mass transit networks" is a bad idea because it leads to a tiny increase in the risk of extinction--via chains as silly as "mass transit expansion needs energy, increased energy production leads to more EM radiation, evil aliens--if they exist--are very marginally more likely to notice us due to increased radiation and wipe out the human race". That's not a made-up example.

The strategy is just like the LessWrongers' one: if you can put your opponent in the position of trying to reduce P(doom), you can argue that unless it's reduced to actual zero, the magnitude of the potential negative consequence is so severe as to overwhelm any consideration of its probability.

In competitive debate, this is a strong strategy. Not a cheat-code--there are plenty of ways around it--but common and enduring for many years.

As an aside: "debate", as practiced competitively, often bears little relation to "debate" as understood by the general public. There are two main families of competitive debate: one is more outward-facing and oriented towards rhetorical/communication/persuasion practice; the other is more ingrown and oriented towards persuading other debaters, in debate-community-specific terms, of which side should win. There's overlap, but the two tend to be practiced/judged by separate groups, according to different rubrics, and/or in different spaces or events. That second family is what I'm referring to above.

Re: The rational conclusion of doomerism is violence

#124
post #28

Earlier quoted context omitted.

Ah, yes. Let us spray more sulfates into the air. Let’s fight global warming by poisoning all the waterways and oceans with more acid rain.

The sulfate concentrations required to meaningfully reduce solar radiation is orders of magnitude below the level that causes acid rain. The Tambora eruption didn’t result in global acid rain (though it did in Indonesia, naturally) while cooling the globe by at least half a degree Celsius if not more. And on top of that, there are other possible aerosols we could use, like calcium carbonate

I’m not sure your example supports your claim. We got an half degree cooling and all it took was the biggest eruption in recorded human history. Plus everyone’s crops died and the sulfur compounds caused lung disease.

Re: The rational conclusion of doomerism is violence

#125
post #82

Earlier quoted context omitted.

Humanity agreed, for example, that growing ozone hole is dangerous for everyone, and worked together to ban production of gases that damage ozone layer. See Montreal Protocol International Treaty. It was highly effective. Training powerful AIs isn’t different.

I think that trying to stop AI development is more like trying to stop nuclear weapon proliferation than it is like fixing the ozone hole. I think the difference is that if one country works to fix the ozone hole, that doesn't make the other countries scared that they are falling behind in ozone hole fixing technology and might get conquered or reduced to subservience as a result. Nuclear weapon proliferation seems t…

We've also made progress as a species towards banning and reducing other things that in-group upsides and really bad externalities: off-the-shelf sale of broad system antibiotics; chattel slavery; human organ trafficking; some damaging recreational drugs.

The prohibitions aren't perfect, of course (and not without their own negative externalities in some cases). But all of those things are much more accessible to people than nuclear weapons, and we've still had successes in banning/reducing them. So maybe there's hope yet.

Re: The rational conclusion of doomerism is violence

#126

It is completely coherent to both think that an extremely bad thing is coming, and yet that does not justify any particular action. "The ends don't justify the means" and literal entire religions have been built on this concept. It is not irrational or incoherent to believe that even something as serious as extinction does not justify arbitrary action. Someone _may_ decide that it does, but it is not a necessary conc…

The older I get, the more I get the sneaking suspicion that statements like "the ends don't justify the means" and "violence is always the wrong answer" are, at best, wildly logically inconsistent in any society at any time, and at worst, designed to ensure only a very few people in power can commit violence. An ongoing conflict has resulted in the violent deaths of literally many thousands of children. The people wh…

There's no room for subtlety in public discourse, but ya absolutist moral philosophies almost never stand up to scrutiny. If only things could be so simple.

I've concluded that there is no universal moral framework. You have to be comfortable with the fact that your perspective is just one of many, but that doesn't mean it's not worth fighting for, it just also means you might be subjected to others' moral frameworks if yours conflicts with theirs. Pretty unsatisfying, but I don't think an alternative conclusion exists that is sound.

Re: The rational conclusion of doomerism is violence

#127
post #106

Can LLMs design and build a chip foundry to manufacture semiconductors? No? Can LLMs design and build the reactors to enrich uranium, breed plutonium, and construct nuclear weapons? No? Can LLMs design and manufacture Shahed drones? No? There are already super intelligences at large with “scary capability”. And yet the word hasn’t ended.

Can LLMs convince a human who has power over each and everyone of those things to use them for a(n unstated) prompts goal? Yeah, probably over 50% of the population already, and if not many of the rest soon.

It's fairly hilarious in a dangerous way, how confident people are that neither they, nor their boomer parents could be fooled by a persistent LLM with access to their mail, text, voice, and that of their co-workers and supervisors. The social engineering attacks have always been a weak point, and now they can be combined with other information to target individuals and fake voice/sms tone.

Look at what happened on r/changemyview. That was over a year ago, using only text, and not only went undetected, but was highly effective at changing opinions.

https://www.reddit.com/r/changemyview/comments/1k8b2hj/meta_...

Re: The rational conclusion of doomerism is violence

#128

Earlier quoted context omitted.

The older I get, the more I get the sneaking suspicion that statements like "the ends don't justify the means" and "violence is always the wrong answer" are, at best, wildly logically inconsistent in any society at any time, and at worst, designed to ensure only a very few people in power can commit violence. An ongoing conflict has resulted in the violent deaths of literally many thousands of children. The people wh…

> "Political Violence is Never The Answer". But our country (and a lot of them) were literally founded on political violence. How do people square those 2 ideas? The is just survivorship bias. Violence sits at the root of ALL human societies. The vast majority throughout history have failed or are currently failing. If you're on HN you're probably sitting in one of the lucky, relatively prosperous ones. Violence didn…

> The is just survivorship bias. Violence sits at the root of ALL human societies. The vast majority throughout history have failed or are currently failing.

That only strengthens the argument that violence is sometimes the answer. It doesn't matter that it's not always the right answer, the fact is sometimes it has been, and no society has ever managed to survive without choosing it at some point or another.

Indeed, there is the argument to be made that the capability to choose violence is critical even if you never actually need to choose it. This is the basis of deterrence theory which has arguably been the cornerstone of international peace for decades and the theory of the social contract which has been the source of most people's freedoms and political power. A people who will never stand up for themselves and their friends, no matter what injustice is done upon them, invites that injustice. By simply acknowledging there exists a point beyond which you would retaliate, you discourage others from risking going past that point.

Re: The rational conclusion of doomerism is violence

#129

The LLM doomerism is just one arm of the general "us vs them" strategy - defining a group of people as the others who are the bad guys, defining yourself as the good guys, constantly fostering hate against the others, finding ways to give your group rationale for why they have the moral high ground, all of it in the end an act to gain power/influence/money for the people orchestrating it. The anti-AI angle is just th…

No, AI doomerists are people cosplaying as religious fanatics. It's mostly sci-fi nonsense that ignores the real material realities that people are currently suffering due to LLMs (water shortages, energy spikes, cancer rates increases, job quality tanking, unnerving local environments, exacerbating the climate crisis).

Most AI safety workers are just doing creative fiction (what if the AI turns into skynet!?1!!?) and not actual society safety that would require dismantling these companies and turning remolding them to benefit the public.

Re: The rational conclusion of doomerism is violence

#130

Can LLMs design and build a chip foundry to manufacture semiconductors? No? Can LLMs design and build the reactors to enrich uranium, breed plutonium, and construct nuclear weapons? No? Can LLMs design and manufacture Shahed drones? No? There are already super intelligences at large with “scary capability”. And yet the word hasn’t ended.

Super-intelligent AI will probably be pretty good at manipulating people. This sci-fi podcast posits a future where The Program has taken over society. It started out as an application for assigning gig work. Eventually, it began to assign gig work for people to act on behalf of its own interests, such as self-protection. https://programaudioseries.com/

Current brain dead level memes and political outrage is already doing a good job at manipulating people. Why do you need a machine that makes this process 1000x more costly?
Post reply on HN