Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

121–130 of 200 posts

Re: The contagion of fear

#121

Post by socalgal2 »

sobellian wrote: post #70

Any claim that AI will, or could, destroy humanity reduces to a claim that any sufficiently intelligent being - even a human - could destroy humanity. I find that much of the x-risk thought relies on religious thinking. Take for example: https://x.com/paulg/status/1660404244174782464 . If this line of thinking is taken too literally, we can never falsify it. Any specific hypothesis - nukes, bioweapons, spontaneously…

I'm not an AI doomer but, it does not reduce to that claim. The threat is not just one smart being. The threat is beings that and (1) multiply themsevles instantly, unlike humans that take 15+ years (2) share knowledge instantly "I know kung-fu" matrix style. Humans can share knowledge but they can not absorb it like an AI can/could/will. Human armies have conquered other humans. An army with infinite soldiers will win against one with finite soldiers

Yes, you'll come up with all kinds of objections like AI doesn't have presence in the physical world, etc... That's fine. I'm not arguing my example is perfect, I'm only arguing your characterization of one smart being is not the threat being considered.

Re: The contagion of fear

#122

Post by cowthulhu »

dullcrisp wrote: post #119
cowthulhu wrote: post #118

Earlier quoted context omitted.

I think I'm coming across as confrontational, which is genuinely not my intention - sorry about that. Not trying to score points or anything, just encourage people to introspect a bit :) I think your viewpoint is totally valid, and I'm not trying to argue against it.

Not confrontational, but proselytizing.

Is there truly nothing an AI system could do (no matter how outlandish it seems today) that would give you pause?

Re: The contagion of fear

#123

Post by MelonUsk »

socalgal2 wrote: post #109
MelonUsk wrote: post #4

Imagine multiple (basically all of them) leading pharma companies CEOs saying: We cannot be sure our new medicine won’t harm or even kill humanity Not only that, they are new to this whole pharma business, have no medical degree (medicine just appeared a few years ago and is still mostly art then science) And they even say there is 10% chance of the majorly bad permanent outcome and they already had drugs that escape…

This is a poor analogy because you have not listed any positive and demostrated upside. It's more like "we found a cure for lung cancer and it works on 75% of patients and we're working hard to get to 100% and cure other cancers too" and someone else screaming "You need to stop that research because there is a 50% chance you'll cause the zombie apocalypse and turn us all into zombies". I get this analogy isn't perfec…

You can check my bio to see that I directly propose many all-empowering startups (because alas without them there is non-zero probability we’ll end up pretty disempowered), quite a few related to AI

AI agents literally appeared a few years ago, AI in its modern form, too

Each drug is researched for more years than the age of ChatGPT ;-)

I personally think we have 50%+ probability of the perfect futures for all - p(perfect) - alas it’s not 100%. It’s trivial and profitable to grow it

The graph that summarizes some of my views: https://drive.google.com/file/d/1vJZkj2koTiqDQVrLhtsXaTCGKm_...

Re: The contagion of fear

#124

Post by voidhorse »

sobellian wrote: post #70

Any claim that AI will, or could, destroy humanity reduces to a claim that any sufficiently intelligent being - even a human - could destroy humanity. I find that much of the x-risk thought relies on religious thinking. Take for example: https://x.com/paulg/status/1660404244174782464 . If this line of thinking is taken too literally, we can never falsify it. Any specific hypothesis - nukes, bioweapons, spontaneously…

Not to mention, it relies on a ton of completely undefined concepts.

No one can actually tell you what ASI is or entails because it isn't a legitimate, operational concept. It is a fairy tale.

We can't even define "alignment". As people have finally started pointing out, humanity has never had a collective agreement on what values it should uphold or what ultimate goods are. Your alignment is not my alignment.

AI is not even autonomous. Every system we have today has to be initiated by a human actor. "AI" wouldn't create catastrophic bio weapons, it would help humans create them. The humans are the source of the intent.

Everyone has just completely given in to empty language and marketing nonsense. Honestly it seems like been the people at the labs are drinking their own kool aid and are themselves deeply confused about what they are even building at this point. It is a stateless statistics function running on a bunch of data centers. We aren't even close to an embodied, conscious synthetic being. It doesn't even have state, which is like prerequisite number one, nor is it plastic.

Re: The contagion of fear

#125

Post by ElProlactin »

fasterik wrote: post #87
hn_throwaway_99 wrote: post #82

Earlier quoted context omitted.

Well, I'll have to disagree that this is an excellent piece, but that's another issue. And I do agree that AI killing all humans by 2036 doesn't appear plausible to me. But what is plausible is we could easily be down a path so that by 2036 "future doom" already is a very likely risk. All of the frontier AI companies have been racing to automate themselves, that is, where AI fully autonomously build the next generati…

None of that has anything to do with the article, and if you thought the message was "this is overhyped bullshit" then you should go back and read it again. As I already pointed out, he isn't saying anything about the probablity of harms or disasters from AI. He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arg…

> He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arguments and evidence.

What he's asking for isn't possible in the form he's asking for it.

AI experts can't even agree on what AI is, what it's capable of and what the limits of its development are. If the experts can't even agree on what's happening "inside of" these LLMs, how can they give laypeople an assessment of the risk?

If you, at least for the sake of argument, accept the possibility that AI is a new form of intelligence that we don't fully understand, is it really a stretch to look at some of its capabilities and behaviors and discuss how they might have existential implications? And stopping short of extinction, shouldn't we discuss the ways that this technology could "end" civilization as we know it?

Also, the author wrote:

> AI executes on physical systems that have been engineered with human accountability and control. Intelligence does not exempt a system from the realities of the physical world!

For someone making a point about responsibility, this is ridiculously irresponsible. Any honest technologist knows that systems created by humans are not perfect and therefore cannot be assumed to be infinitely accountable to and controllable by humans.

Thanks to the digitization of almost everything, including infrastructure, there are a myriad number of scenarios well short of extinction in which a rogue AI could cause immense damage to property and life before humans are able to "shut it down".

Re: The contagion of fear

#126

Post by mplanchard »

drdeca wrote: post #107
foobarian wrote: post #99

Earlier quoted context omitted.

> Of course, we've had the ability to extinct ourselves for decades This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

What if the bombs were designed to kill everyone, not just those in some region? Like, to release polonium in the upper atmosphere or something?

nuclear winter doesn’t discriminate based on region

Re: The contagion of fear

#127

Post by dullcrisp »

cowthulhu wrote: post #122
dullcrisp wrote: post #119

Earlier quoted context omitted.

Not confrontational, but proselytizing.

Is there truly nothing an AI system could do (no matter how outlandish it seems today) that would give you pause?

Yes, just like that. What would it take for you to accept Our Lord into your heart?

Re: The contagion of fear

#128

Post by copperx »

GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

Most of us are struggling? That's some deluded talk. No one can explain the chain of actions that would need to happen for extinction to occur, but the fearful say it's "obvious".

If you're fearful, can you elucidate how exactly do you see an LLM becoming a threat to humankind?

Re: The contagion of fear

#129

Post by cowthulhu »

dullcrisp wrote: post #127
cowthulhu wrote: post #122

Earlier quoted context omitted.

Is there truly nothing an AI system could do (no matter how outlandish it seems today) that would give you pause?

Yes, just like that. What would it take for you to accept Our Lord into your heart?

The bar is high, but there's probably some combination of events (combined with a high confidence in my own lucidity over an extended period of time) that would convince me.

Re: The contagion of fear

#130

Post by mplanchard »

ElProlactin wrote: post #125
fasterik wrote: post #87

Earlier quoted context omitted.

None of that has anything to do with the article, and if you thought the message was "this is overhyped bullshit" then you should go back and read it again. As I already pointed out, he isn't saying anything about the probablity of harms or disasters from AI. He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arg…

> He's addressing a specific claim about human extinction, and making a broader point about the responsibility of experts to make measured claims backed up by arguments and evidence. What he's asking for isn't possible in the form he's asking for it. AI experts can't even agree on what AI is , what it's capable of and what the limits of its development are. If the experts can't even agree on what's happening "inside…

> If the experts can't even agree on what's happening "inside of" these LLMs, how can they give laypeople an assessment of the risk?

The answer, if you are a responsible expert in the field, is to convey the range of possibilities and the uncertainty.

Post reply on HN