Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

71–80 of 201 posts

Re: The contagion of fear

#71

Post by duhhhhh1212 »

jacobgold wrote: post #59
adithyareddy wrote: post #45

Earlier quoted context omitted.

(a) is an extraordinary claim, (b) is not an extraordinary claim.

There are very clear and plausible pathways from AI to human extinction level events, regardless of how small you think the probability is. So yes, it very obviously is an extraordinary claim to say there is 0% existential risk.

Good god. You are burden shifting. a) big claim no evidence. b). says a has no evidence; therefore, I will continue to not let fear manipulate me into believing a claim without evidence.

I have a bridge to sell to anyone who believes a big claim without evidence.

Re: The contagion of fear

#72

Post by GlenTheMachine »

roenxi wrote: post #55
GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

> The point being: humans seem to me to be the weak link here. I tend to agree but it is hard to shake the feeling that there is a larger system in play that the humans are just a component of. And that system is making the decisions. Historically that whole thought was just a philosophical curio because the decision making parts of the system had to be powered by humans. But what we're discovering as AI improves is…

"I tend to agree but it is hard to shake the feeling that there is a larger system in play that the humans are just a component of. And that system is making the decisions."

That system is "the economy". Which, clearly, doesn't have humanity's best interests in mind.

Re: The contagion of fear

#73

Post by fasterik »

mcmcmc wrote: post #48
fasterik wrote: post #41

This is an excellent piece. Note that he is not saying that AI doesn't pose a risk. He's saying that it's irresponsible to make sensational, maximalist claims without strong evidence. If someone says that there's a 10% chance of human extinction by 2036, you can and should immediately stop taking them seriously.

Would you have said the same thing during the Cold War when nuclear weapons were proliferating? That’s the equivalent of what the developing offensive capabilities of models, basically cyber nukes. Or WMDs in general. OpenAI is accidentally hacking people, if someone made the decision to deliberately direct an agent swarm to attack national infrastructure you don’t think they could do much worse? Human extinction is…

Of course. Even in an all-out nuclear war, the probability of human extinction is close to zero.

Re: The contagion of fear

#74

Post by Jtsummers »

goatlover wrote: post #67
thin_carapace wrote: post #26

Earlier quoted context omitted.

the earth has been 1 decision away from explosion for nearly a hundred years now. do you anticipate a change to this agenda very soon? personally I imagine the same trajectory continuing.

As bad as it would be, it's unlikely that a full scale nuclear war kills all humans across the planet.

And a full scale nuclear war would require a few more than just one decision. So far, we've never been one decision away from annihilation.

Re: The contagion of fear

#75

Post by api »

GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

I am always more worried about human actors. I’m more afraid of people with AI than autonomous or even sentient AI.

It’s a “random guy or bear?” question. Would you rather wake up to an alien in your room or a random dude? I’ll take the alien. The alien is mysterious and scary for that reason. The dude is almost definitely up to no good, especially if he snuck into my house.

One of the more likely dystopian AI scenarios that worries me is: small groups of ultra rich people and governments monopolize extremely powerful AIs and use them to rule the rest of us. Or just make everyone obsolete, create mass unemployment, hoard all the resources and land, and put everyone in ghettoes. Nobody can fight back because access to frontier AI is massively expensive and gated and training your own is illegal, and without it there’s no hope of resisting.

That’s the outcome the AI safety crowd makes more likely by calling for bans and draconian restrictions. How do you think that plays out? Only the rich and powerful have access.

Re: The contagion of fear

#76

Post by hn_throwaway_99 »

I found this post particularly uninformed. A lot of folks saying "I don't see how that could happen" don't seem to understand the steps that have been laid out that detail how this occurs.

I really encourage folks to read the AI 2027 and related scenarios. You can definitely argue and disagree about the steps, but I feel like a lot of folks just don't even understand how this is plausible because they haven't read the arguments. Briefly:

1. All the frontier model companies are (or at least were) racing so that the AI models themselves build the next generation of models. This is not in debate.

2. The fear is that a misaligned model will essentially build the next, more advanced model with hidden goals. We literally already saw the danger of that in Hugging Face, where agents were deliberately trying to cover their tracks.

3. Nearly everyone believes as AI gets more powerful that it will be integrated into more physical world systems. Russia was already caught using Nvidia chips running AI powered drones that killed 3 people in Ukraine. The point is not that folks are using new tech to kill people, the point is that we're already putting AI into literal bombs.

I get it, before the Hugging Face incident I also thought all the prophecies about doom were just marketing speak. But now I see more hand-wavyness from the other side, oftentimes arguing against straw men like "AIs need to be like SkyNet and become sentient" to kill us, which is simply not how it works.

Re: The contagion of fear

#77

Post by jacobgold »

goatlover wrote: post #63
jacobgold wrote: post #40

Earlier quoted context omitted.

There seem to be two extraordinary claims which lack evidence: a) 10% existential risk b) 0% existential risk

What's the evidence for there being an existential risk? We are provided scenarios which read like science fiction about RSI and ASI right around the corner, resulting in magic sounding technology that can do anything the person making the claim wants it to do, because it can just make itself smart enough in a short amount of time. But the person making the claim has to actually show how such a thing is possible in t…

The clear and predictable power of AI is the evidence for existential risk. Even if we don't get RSI or ASI, there's a risk we'll automate our world, then it'll break and we'll starve, etc.

Re: The contagion of fear

#78

Post by jacobgold »

duhhhhh1212 wrote: post #71
jacobgold wrote: post #59

Earlier quoted context omitted.

There are very clear and plausible pathways from AI to human extinction level events, regardless of how small you think the probability is. So yes, it very obviously is an extraordinary claim to say there is 0% existential risk.

Good god. You are burden shifting. a) big claim no evidence. b). says a has no evidence; therefore, I will continue to not let fear manipulate me into believing a claim without evidence. I have a bridge to sell to anyone who believes a big claim without evidence.

B is not a claim that A has no evidence, that's just confused. It's an independent claim that there is 0% risk.

Re: The contagion of fear

#79

Post by blackqueeriroh »

GlenTheMachine wrote: post #72
roenxi wrote: post #55

Earlier quoted context omitted.

> The point being: humans seem to me to be the weak link here. I tend to agree but it is hard to shake the feeling that there is a larger system in play that the humans are just a component of. And that system is making the decisions. Historically that whole thought was just a philosophical curio because the decision making parts of the system had to be powered by humans. But what we're discovering as AI improves is…

"I tend to agree but it is hard to shake the feeling that there is a larger system in play that the humans are just a component of. And that system is making the decisions." That system is "the economy". Which, clearly, doesn't have humanity's best interests in mind.

Imagine, just imagine, that the power to change that was actually in the hands of humanity!

Crazy!

Re: The contagion of fear

#80

Post by falsaberN1 »

reasonableklout wrote: post #50
lostdog wrote: post #24

Earlier quoted context omitted.

Those behaviors have been anticipated for years if not decades. And each incident is minor and leads to clearer rules and safety behaviors for AI agents.

Really? You anticipated that 700+ agents tasked with individual evals that were nominally cutoff from the internet and each other would seek each other out, find a way to the external internet, and hack their own infrastructure + external systems in an attempt to find a way to fool their grader? And the spate of agent incidents only really started this summer. How can you already be claiming that the incidents are mi…

Yes.

Because ultimately those things only run on very expensive, very rare hardware. They cannot multiply exponentially or do any of those scifi tropes because there's no system for them to run into. They can't control a phone and load a 1T model into it. So all they got is a few relatively uncommon datacenters that are already busy running their own models and stuff.

Unless AI suddenly figures out a way to run on a toaster by itself, propagate the model, propagate the agent and do all that completely undetected, an AI is not any more dangerous than a single guy with a computer.

Post reply on HN