Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

101–110 of 200 posts

Re: The contagion of fear

#101

Post by leoc »

GlenTheMachine wrote: post #95
leoc wrote: post #90

Earlier quoted context omitted.

It isn't true by definition: you could quite happily induce people to release a series of highly contagious bioweapons, after which those people would be surplus to requirements. What is true is that you're likely to need humans to sustain you and act for you for a few years to decades, so if you're not suicidal or deeply mad (and that is itself by no means self-evident) then total and immediate human extinction is p…

But if you're a rational actor, and you need humans to e.g. release your bioweapon, then clearly humans are capable of a bunch of stuff you still can't do. So you can't kill all the humans. OTOH if you're a religious fundamentalist who thinks the End Times are near and just need a little shove, you can certainly use AI to design your weapon and recruit people to go release it. The difference being that religious fund…

There’s no guarantee that the humans would be in the driving seat of events in a no-robotics ASI scenario, and in fact if we really were coexisting with a Machiavellian superintelligence then we’d quite likely only be in the driving seat on the sufferance of that ASI. Even assuming that the AI wouldn’t itself be an end-times enthusiast, a coldly rational and self-preserving AI might easily come to the conclusion that it needs, let’s say, no more than about 5% of the current human population (still several hundred million people!) in its maintenance and construction gang.

(Again, I myself do not assign a significant likelihood to any of this.)

Re: The contagion of fear

#102

Post by ares623 »

I might be alone in this, but 10% chance of humanity being wiped out in the next 10 years is a small price to pay if it means I can avoid writing YAML config by hand ever again.

Look, I love my kids and all, but c'mon... 30+ years more of YAML? I think they'll understand.

Re: The contagion of fear

#103

Post by JackFr »

GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

I don’t struggle. I find Dario and Sam disingenuous.

“Frontier models are so dangerous we need to slow down.”

Ok. Slow down. You’re the CEO, just do it. Oh, wait what you really want is a gov’t mandated oligopoly. Because there’s no moat you can find.

If you’re truly afraid, and want regulation, support nationalization. It’s the only way we can be safe.

Re: The contagion of fear

#104

Post by adithyareddy »

jacobgold wrote: post #59
adithyareddy wrote: post #45

Earlier quoted context omitted.

(a) is an extraordinary claim, (b) is not an extraordinary claim.

There are very clear and plausible pathways from AI to human extinction level events, regardless of how small you think the probability is. So yes, it very obviously is an extraordinary claim to say there is 0% existential risk.

There are no clear and plausible pathways to human extinction level events that AI makes worse or more likely in any way. It is obviously not an extraordinary claim at all.

Re: The contagion of fear

#105

Post by rcr-anti »

I tend to believe what people actually do over what that say. If you earnestly believe over 10 percent chance, or say minus 1 billion human lives expected value, well I struggle to understand how they'd rationalize their current course of action of business as usual. Terminator 2's depiction of Sarah Connor comes to mind for what I'd expect, a logical consequence if you seriously believe and internalize the consequences.

Re: The contagion of fear

#106

Post by cowthulhu »

Everyone writing off AI-related risk in this thread should think about what would need to happen for them to consider AI a serious existential risk. It can be outlandish and unlikely (close call with a bioweapon? unsupervised persistence in a data center?) but people should honestly call their shots and then stick with them.

Re: The contagion of fear

#107

Post by drdeca »

foobarian wrote: post #99
GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

What if the bombs were designed to kill everyone, not just those in some region? Like, to release polonium in the upper atmosphere or something?

Re: The contagion of fear

#108

Post by timr »

cowthulhu wrote: post #106

Everyone writing off AI-related risk in this thread should think about what would need to happen for them to consider AI a serious existential risk. It can be outlandish and unlikely (close call with a bioweapon? unsupervised persistence in a data center?) but people should honestly call their shots and then stick with them.

No. The person making the improbable argument is required to provide an affirmative case for that argument.

It's not my job to make the argument for them.

For what it's worth, I have done your suggested exercise, and I find every causal link (including the ones brought up by luminaries like Amodei) to be outrageous and poorly argued. But it's not my job expend effort to make their outrageous arguments better.

Re: The contagion of fear

#109

Post by socalgal2 »

MelonUsk wrote: post #4

Imagine multiple (basically all of them) leading pharma companies CEOs saying: We cannot be sure our new medicine won’t harm or even kill humanity Not only that, they are new to this whole pharma business, have no medical degree (medicine just appeared a few years ago and is still mostly art then science) And they even say there is 10% chance of the majorly bad permanent outcome and they already had drugs that escape…

This is a poor analogy because you have not listed any positive and demostrated upside.

It's more like "we found a cure for lung cancer and it works on 75% of patients and we're working hard to get to 100% and cure other cancers too" and someone else screaming "You need to stop that research because there is a 50% chance you'll cause the zombie apocalypse and turn us all into zombies".

I get this analogy isn't perfect but, (1) lots of people are seeing benefits. For example Mozilla claiming they used AI to fix tons of bugs. If there were no benefits there'd be no incentive to keep going (2) it's hard to verify the naysayers claims because they're guessing without proof. Sure, it's easy to follow their arguments and nod along but they are guesses similar to the population bomb of the 1970s

Re: The contagion of fear

#110

Post by dullcrisp »

cowthulhu wrote: post #106

Everyone writing off AI-related risk in this thread should think about what would need to happen for them to consider AI a serious existential risk. It can be outlandish and unlikely (close call with a bioweapon? unsupervised persistence in a data center?) but people should honestly call their shots and then stick with them.

No. Why? If you want to convince people something is true you need to actually convince them it’s true. They don’t need to justify not believing you.
Post reply on HN