Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

151–160 of 200 posts

Re: The contagion of fear

#151

Post by ShinyLeftPad »

voidhorse wrote: post #120
rcr-anti wrote: post #105

I tend to believe what people actually do over what that say. If you earnestly believe over 10 percent chance, or say minus 1 billion human lives expected value, well I struggle to understand how they'd rationalize their current course of action of business as usual. Terminator 2's depiction of Sarah Connor comes to mind for what I'd expect, a logical consequence if you seriously believe and internalize the consequen…

This. If you truly believe the thing you are building is going to kill you and your family, your decision is not "let's keep going" unless you are a complete nihilist. The fact that people buy the whole "well we have to because if we don't someone else will" argument shows just how pervasive and complete irrationality has become. People eat up content and do not even apply a microsecond's wort of critical thought to…

This fixation on "entire humanity" and "truly believe".

How about this, they think it's possible LLM could totally facilitate a mass murder of never seen before scale, but they continue to work on it because they're rich and sheltered enough they are not that worried that their family personally will be killed. The rest of humanity doesn't matter compared to making more money.

Re: The contagion of fear

#153

Post by Mentlo »

This piece, like many anti-doom pieces - is grounded in what Ai does today. Doom scenarios are, however, all extrapolations of multiple exponential curves.

It’s hard to think up the exponential. It’s even harder to communicate an inference one is making across multiple exponentials.

We last had this is early 2020, where Doomers were stockpiling food and medicine and the anti-doomers were ridiculing them. Anti-doomers were focusing on the single exponential, whereas doomers were modelling virus evolution, monitoring and sequencing lag and social dynamics against the exponential. The latter was very hard to communicate before the fact as it was a combination of deep intuition and grappling with the exponential.

I am not saying covid is proof that ai doomers are right, I am saying it’s an example of the known property of human cognition - which is that it struggles with exponentials. Covid was 2 exponentials, AI I can rhink of at least 4 relevant ones.

To me - the fact that 3-4 generations from now AI will have superhuman hacking ability and superhuman persuasive ability (for intuition transfer - think of superhuman persuasive ability as superhuman ability to hack human systems) materialises bio risks swiftly. We already have technology to make robotic systems (mini drone swarms) that can kill humans en-masse with no credible defensive vector bar an EMP. Climbing up those exponentials for further 6 years makes me want to stockpile food and medicine.

Re: The contagion of fear

#154

Post by fasterik »

ShinyLeftPad wrote: post #148
fasterik wrote: post #73

Earlier quoted context omitted.

Of course. Even in an all-out nuclear war, the probability of human extinction is close to zero.

Then it's all fine right? /s Killing even 10% of humanity should be unacceptable, can't believe this argument is even happening

Of course it would be unacceptable. Honestly, this thread feels like talking to people who have the reading comprehension of a 5th grader.

Re: The contagion of fear

#155

Post by rakel_rakel »

I enjoyed this post.

I don't know if this is the correct thread for it, but am I the only one thinking that this call for regulation is nothing but a way for the American AI companies to try and mitigate being overrun by the competition from open models? To me that's obvious, but I'm very often wrong, and very likely that's the case now too.

Re: The contagion of fear

#156

Post by ShinyLeftPad »

fasterik wrote: post #154
ShinyLeftPad wrote: post #148

Earlier quoted context omitted.

Then it's all fine right? /s Killing even 10% of humanity should be unacceptable, can't believe this argument is even happening

Of course it would be unacceptable. Honestly, this thread feels like talking to people who have the reading comprehension of a 5th grader.

Someone said "AI might kill everybody soon" and prevailing argument is "it's wrong because it's not gonna be like everybody everybody".

Re: The contagion of fear

#157

Post by fasterik »

ShinyLeftPad wrote: post #156
fasterik wrote: post #154

Earlier quoted context omitted.

Of course it would be unacceptable. Honestly, this thread feels like talking to people who have the reading comprehension of a 5th grader.

Someone said "AI might kill everybody soon" and prevailing argument is "it's wrong because it's not gonna be like everybody everybody".

It's the AI doomers who have chosen to frame the argument in terms of extinction. Even if they were only claiming that 1% of the world population will be killed in the next 10 years, it would still be an extraordinary claim that hasn't been established with any amount of rigor. I'll reiterate the original point: these people are not serious, they're spreading alarmist nonsense, and they're undermining the public's trust in experts.

Re: The contagion of fear

#158

Post by henrymerrilees »

Mentlo wrote: post #153

This piece, like many anti-doom pieces - is grounded in what Ai does today. Doom scenarios are, however, all extrapolations of multiple exponential curves. It’s hard to think up the exponential. It’s even harder to communicate an inference one is making across multiple exponentials. We last had this is early 2020, where Doomers were stockpiling food and medicine and the anti-doomers were ridiculing them. Anti-doomers…

Part of our general inability to reason about exponential growth is the inability to recognize that a supposedly exponential growth is actually sigmoid. The horizontal asymptote could just as well be 8b, so it is not a case against doomerism in itself, but as is the case with COVID, to say nothing discounting its horror, that number is far less.

There is no need to count exponentials—it would only be a matter of time. The need to sum multiple factors betrays the finite limits of what is actually sigmoid growth. Reasoning about specific effects is unfortunately subject to counter-evidence and so struggles for traction against abstract handwaving about exponential growth.

There is much uncertainty, certainly not exclusive to AI. The benefit of hyper-vigilance in each case must be weighed against the cost of indulging every similar panic.

Re: The contagion of fear

#159

Post by ShinyLeftPad »

fasterik wrote: post #157
ShinyLeftPad wrote: post #156

Earlier quoted context omitted.

Someone said "AI might kill everybody soon" and prevailing argument is "it's wrong because it's not gonna be like everybody everybody".

It's the AI doomers who have chosen to frame the argument in terms of extinction. Even if they were only claiming that 1% of the world population will be killed in the next 10 years, it would still be an extraordinary claim that hasn't been established with any amount of rigor. I'll reiterate the original point: these people are not serious, they're spreading alarmist nonsense, and they're undermining the public's tr…

There is no proof possible. There is no experiment that can show something humanity scale will happen. There are even no remotely similar precedents in the past to extrapolate.

The actual established real evidence is that this tech is showing unprecedented capabilities (including destructive) and its safety guardrails are lacking.

Re: The contagion of fear

#160

Post by henrymerrilees »

ElProlactin wrote: post #136
mplanchard wrote: post #130

Earlier quoted context omitted.

> If the experts can't even agree on what's happening "inside of" these LLMs, how can they give laypeople an assessment of the risk? The answer, if you are a responsible expert in the field, is to convey the range of possibilities and the uncertainty.

Isn't that what they're doing? "There's a better than 0 chance this thing could kill everyone in the next decade." That might be too imprecise for the HN set but it's realistic for laypeople. And none of the AI people talking about the risk are running into rooms full of people telling them Claude has gone mad and yelling at them to disconnect from the internet and turn off their devices immediately.

A key facet of the suggested risk is that these programs would attempt to mask whatever behavior would bring about the end of the world. A world in which AI doom could be detected before it’s too late and stopped by metaphorically running into rooms full of people telling people to unplug is less extreme than that in which doomers argue we are already living.

Alarm takes severity and scale. Forecasting double-digit odds of near-term human extinction on national television to a lay audience is more extreme than yelling to unplug rogue AI. For what it’s worth, the latter has already occurred numerous times at containable scales in leading labs, mitigated by the physical realities of computation, notwithstanding the competence of involved personnel. People running into rooms yelling is probably not a hypothetical.

Post reply on HN