Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

141–150 of 200 posts

Re: The contagion of fear

#141

Post by emil-lp »

cowthulhu wrote: post #111
timr wrote: post #108

Earlier quoted context omitted.

No. The person making the improbable argument is required to provide an affirmative case for that argument. It's not my job to make the argument for them. For what it's worth, I have done your suggested exercise, and I find every causal link (including the ones brought up by luminaries like Amodei) to be outrageous and poorly argued. But it's not my job expend effort to make their outrageous arguments better.

I'm not making an affirmative case - I'm looking for others to introspect about this, and anchor their expectations. If we have a bioweapon close call, would you consider AI an existential risk at that point? If not, how close would we need to get? You don't need to answer here, or disclose anything publicly - just think and remember.

> If we have a bioweapon close call, would you consider AI an existential risk at that point? If not, how close would we need to get?

The thing is, this doesn't really mean anything.

What is a bioweapon close call? What is the process in which a bioweapon close call happens? An AI that hacks all cell phones to emitt anthrax?

Re: The contagion of fear

#142

Post by halostatue »

cowthulhu wrote: post #106

Everyone writing off AI-related risk in this thread should think about what would need to happen for them to consider AI a serious existential risk. It can be outlandish and unlikely (close call with a bioweapon? unsupervised persistence in a data center?) but people should honestly call their shots and then stick with them.

At this point, the most likely scenario for AI producing a serious existential risk is making us all want to bash our heads in because of "the honest options" and other claude-isms.

Re: The contagion of fear

#143

Post by jandrewrogers »

shimman wrote: post #112
foobarian wrote: post #99

Earlier quoted context omitted.

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

I mean it's likely a nuclear war would probably be a species extinction level event at the least, humans would definitely be reduced by 99%. We don't know how bad the resulting nuclear cooling would be, but we do know how humans act under extreme desperation. They lash out and attack, when was the last time a majority of human settles were truly under desperate acts of survival simultaneously? Maybe what ~74,000 year…

Nuclear winter scenarios have largely been debunked. Several different events in the 20th century had atmospheric impact on the same scale as modeled in a full nuclear exchange and the effects were modest.

People have kind of mythologized nuclear weapons far beyond their reality. One of the arguments against the use of nukes in policy circles is that this mythology is useful. The limited adverse environmental consequences in practice if demonstrated will greatly lower the threshold for subsequent use.

It might set society back a century, but with substantial knowledge of what was lost. Extinction from nuclear weapons is not remotely plausible.

Re: The contagion of fear

#144

Post by JoshTriplett »

JackFr wrote: post #103
GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

I don’t struggle. I find Dario and Sam disingenuous. “Frontier models are so dangerous we need to slow down.” Ok. Slow down. You’re the CEO, just do it. Oh, wait what you really want is a gov’t mandated oligopoly. Because there’s no moat you can find. If you’re truly afraid, and want regulation, support nationalization. It’s the only way we can be safe.

[dead]

Re: The contagion of fear

#145

Post by maxaw »

I’m not worried about a fully autonomous attack, I’m worried about an attack by a crazy human coached by a frontier llm with no guardrails on how to make a bioweapon

Re: The contagion of fear

#146

Post by [deleted] »

halostatue wrote: post #142
cowthulhu wrote: post #106

Everyone writing off AI-related risk in this thread should think about what would need to happen for them to consider AI a serious existential risk. It can be outlandish and unlikely (close call with a bioweapon? unsupervised persistence in a data center?) but people should honestly call their shots and then stick with them.

At this point, the most likely scenario for AI producing a serious existential risk is making us all want to bash our heads in because of "the honest options" and other claude-isms.

[deleted]

Re: The contagion of fear

#147

Post by ShinyLeftPad »

goatlover wrote: post #57
fasterik wrote: post #41

This is an excellent piece. Note that he is not saying that AI doesn't pose a risk. He's saying that it's irresponsible to make sensational, maximalist claims without strong evidence. If someone says that there's a 10% chance of human extinction by 2036, you can and should immediately stop taking them seriously.

They should also be able to give detailed reasons experts in the relevant fields can verify as to how they arrived at a 10% chance all humanity goes extinct. Not a science fiction narrative which likely does not take into account the relevant physical facts limiting such scenarios. Such as how exactly an AI would build a bioweapon capable of killing 8 billion humans across the planet.

This tool is not known for "how exactly" to be a question people can answer.

And answering it in detail is giving tools to people who may want to do it.

"releasing a very transmittable and deadly respiratory virus with long incubation period" is just one of many ways.

Re: The contagion of fear

#148

Post by ShinyLeftPad »

fasterik wrote: post #73
mcmcmc wrote: post #48

Earlier quoted context omitted.

Would you have said the same thing during the Cold War when nuclear weapons were proliferating? That’s the equivalent of what the developing offensive capabilities of models, basically cyber nukes. Or WMDs in general. OpenAI is accidentally hacking people, if someone made the decision to deliberately direct an agent swarm to attack national infrastructure you don’t think they could do much worse? Human extinction is…

Of course. Even in an all-out nuclear war, the probability of human extinction is close to zero.

Then it's all fine right? /s

Killing even 10% of humanity should be unacceptable, can't believe this argument is even happening

Re: The contagion of fear

#149

Post by ShinyLeftPad »

sobellian wrote: post #70

Any claim that AI will, or could, destroy humanity reduces to a claim that any sufficiently intelligent being - even a human - could destroy humanity. I find that much of the x-risk thought relies on religious thinking. Take for example: https://x.com/paulg/status/1660404244174782464 . If this line of thinking is taken too literally, we can never falsify it. Any specific hypothesis - nukes, bioweapons, spontaneously…

First don't get fixated on the "entire" humanity part. Even 10% should be unacceptable period.

Second you can't compare it to just one person's capability.

And it's never a doubt that really smart and insane people absolutely could kill a lot of humans considering modern technology. The reason they don't is because our society was historically built so that smart people don't want to or get stopped. For example ethics, religion, mental hospitals, self preservation instinct, police etc.

Re: The contagion of fear

#150

Post by timr »

cowthulhu wrote: post #115
dullcrisp wrote: post #114

Earlier quoted context omitted.

I don’t really understand this exercise of predicting what I’ll be thinking in the future.

To avoid shifting goalposts

But it doesn't do that. "Imagine how you'll be wrong in the future" is just as shiftable as anything else, and has the distinct rhetorical advantage of being unfalsifiable.

The substantive difference is that you're asking me to dream up ridiculously improbable scenarios, which is probably the actual point. Just like "The End is Near Accept Jesus" guy on the street corner gets what he wants as soon as I engage.

To quote a famous movie about doomy AI scenarios: "the only winning move is not to play."

Post reply on HN