Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

31–40 of 201 posts

Re: The contagion of fear

#31

Post by perrygeo »

like_any_other wrote: post #3

The article neglects to consider the precautionary principle. It asks us to act as if AI is safe until it's proven to be dangerous, when the cautious approach is to act as if it's dangerous until it's proven to be safe.

While I personally agree with the precautionary principle, when have we actually done that, as a society, in recent history? Fossil fuels, social media, industrial ag, and plastic pollution - there was barely a facade of precaution - post ww2 america has always been full Leroy Jenkins. The reckless approach to AI is right on brand.

Re: The contagion of fear

#32

Post by ACCount37 »

dumberquestions wrote: post #9

So his central rebuttal is that robots aren't good enough yet? What about when they are? And what about all the things you can do entirely remotely? I do feel that threats of catastrophic loss of control seem overstated, both in likelihood and urgency, though any argument for why this risk is not even worth thinking about will probably be overconfident in the other direction.

Adolf Hitler didn't have robots capable enough to carry out his will. He used humans to do it.

It's the old-fashioned way of doing things, but, why change what works?

Re: The contagion of fear

#33

Post by bloaf »

dumberquestions wrote: post #9

So his central rebuttal is that robots aren't good enough yet? What about when they are? And what about all the things you can do entirely remotely? I do feel that threats of catastrophic loss of control seem overstated, both in likelihood and urgency, though any argument for why this risk is not even worth thinking about will probably be overconfident in the other direction.

I think the central premise looks more like this:

Biology has been trying to grey-goo the world for billions of years, but it turns out the world is not something so trivial.

He is pushing back against the folks with pure CS backgrounds who think that computers are all there is. Its a form of magical thinking unique to programmers who live in a world where speaking the right words to a machine is enough to impart your will on the world. Believe that strongly enough, and you fall into the trap of thinking that a sufficiently smart entity could speak the words "let there be light" and it would be so.

The author is pointing out that speaking the words is insufficient. To end humanity there must be an execution phase. The author is correct to point out that acquiring superhuman intelligence is not some guarantee that you will have or obtain the resources necessary to make that happen, in the same way that genius generals still lose to ordinary ones, and the best-laid plans are oft to go awry.

There certainly are risks, but 10% risk of extinction in 10 years is not one of them.

Re: The contagion of fear

#34

Post by abletonlive »

bcantrill wrote: post #16
jacobgold wrote: post #14

These are the sober and informed takes that we need more of. > the claims from Coxon and his ilk are the most extraordinary a technologist can make, and we must demand evidence commensurate with the claims. Yes, exactly. These claims do not have sufficient evidence. > ...you had nothing to fear then — and (at least with respect to extinction risk!) you have nothing to fear now. Wait, this is another extraordinary cla…

Saying that we don't need to fear human extinction in the next ten years (or even the next hundred?) does not feel like an extraordinary claim?

No it does not seem like an extraordinary claim, unless this is your first time hearing such claims. For the rest of us, we've heard this every decade and it turns out to be entirely untrue.

Nuclear, Overpopulation, Peak Oil, Y2K Bug, Global Warming.

No, we aren't going extinct in the next 10 years.

Re: The contagion of fear

#35

Post by jacobgold »

bcantrill wrote: post #16
jacobgold wrote: post #14

These are the sober and informed takes that we need more of. > the claims from Coxon and his ilk are the most extraordinary a technologist can make, and we must demand evidence commensurate with the claims. Yes, exactly. These claims do not have sufficient evidence. > ...you had nothing to fear then — and (at least with respect to extinction risk!) you have nothing to fear now. Wait, this is another extraordinary cla…

Saying that we don't need to fear human extinction in the next ten years (or even the next hundred?) does not feel like an extraordinary claim?

If you think AI will be powerful enough to change the world in huge ways, then it does seem reasonable to assume some level of risk of destroying the world too.

Stating that the probability is zero when we simply don't know what the probabilities are does seem like an extraordinary claim.

Imagine how reassuring it would be to people if we had evidence that there's no existential risk?

Re: The contagion of fear

#36

Post by andsoitis »

GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

> The point being: humans seem to me to be the weak link here.

That doesn't mean AI isn't dangerous. Humans are not to blamed for being the weak link.

Re: The contagion of fear

#38

Post by SamBam »

jchw wrote: post #28

Edit: Actually, nevermind. You guys are not capable of having a reasonable discussion about this.

> The problem isn't that AI could never be capable of causing real world damage. It is that there is absolutely no fucking path towards human extinction.

I think the "humanity will go extinct" discussion is a silly strawman being propagated either to try to spread fear, as the article suggests, or to make the anti-AI folks look silly, but it detracts from your first point which is that it may be "capable of causing real world damage."

We can be worried about real damage without having to defend the idea that every one of the planet's humans will die.

> "superhuman AI is unlikely to be much of a match against severing fiberoptic cable"

I think that the realistic scenarios all involve humans with an intent to do harm -- creating bio-weapons, finding vulnerabilities in infrastructure -- and using AI to help, so "severing the fiberoptic cable" doesn't apply.

Re: The contagion of fear

#39

Post by 0xDEAFBEAD »

blfr wrote: post #15
dumberquestions wrote: post #9

So his central rebuttal is that robots aren't good enough yet? What about when they are? And what about all the things you can do entirely remotely? I do feel that threats of catastrophic loss of control seem overstated, both in likelihood and urgency, though any argument for why this risk is not even worth thinking about will probably be overconfident in the other direction.

OP's central rebuttal is that anxiety or fear are not evidence. This particular anxiety has been with us for thirty+ years, its proponents/sufferers weren't right before, and they don't have some new evidence now.

The recent cyberattacks (HuggingFace etc.) validated a number of core doomer predictions around cheating to maximize reward functions, agentic deception of human/non-human supervisors, AI attempts to manipulate humans, automated hacking, and collective superintelligence.

Re: The contagion of fear

#40

Post by jacobgold »

duhhhhh1212 wrote: post #25
jacobgold wrote: post #14

These are the sober and informed takes that we need more of. > the claims from Coxon and his ilk are the most extraordinary a technologist can make, and we must demand evidence commensurate with the claims. Yes, exactly. These claims do not have sufficient evidence. > ...you had nothing to fear then — and (at least with respect to extinction risk!) you have nothing to fear now. Wait, this is another extraordinary cla…

bruh what? Are we reading the same blog? Bryan says the responsibility is the person making the extraordinary claim. If tomorrow Jacob Coxon comes on CNN and says "Jacob Gold is an extinction risk to humanity". What are we supposed to do? Put you in a bunker and never let you see the light of day until someone proves otherwise? If someone doesn't accept an extraordinary claim without evidence, that doesn't mean they…

There seem to be two extraordinary claims which lack evidence:

a) 10% existential risk

b) 0% existential risk

Post reply on HN