Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The contagion of fear

bcantrill.dtrace.org

Post reply on HN
Print view View original

91–100 of 200 posts

Re: The contagion of fear

#91

Post by ACCount38 »

GlenTheMachine wrote: post #54
ACCount37 wrote: post #42

Earlier quoted context omitted.

If you're in the field, then you know: modern robotics is an AI problem more than anything else. If we have a rogue AI trying to get into a self-improvement loop and gunning for ASI? I'd expect that to be accompanied by a massive change in how capable robots are. Driven by all the existing frames suddenly getting vastly improved AI to back them. If an AI can take a reasonable crack at autonomous operationalized RSI,…

"If you're in the field, then you know: modern robotics is an AI problem more than anything else." It is not. Certainly AI is a big part of why robotics is hard, but it is by no means the biggest. You can fall into one of two camps: you either think that robots will need to work in human-engineered spaces, doing jobs by replacing humans; or you think that we need to change our infrastructure in order to be roboticall…

[dead]

Re: The contagion of fear

#92

Post by ACCount38 »

GlenTheMachine wrote: post #69
leoc wrote: post #64

Earlier quoted context omitted.

Right. A hypothetical superhuman AI wouldn’t have to master robotics to affect the physical world. It could simply bribe, blackmail, manipulate and play politics with humans. As others have pointed out, our political leaders have already been playing these games since forever ago https://news.ycombinator.com/item?id=49689978 and a super-AI would be better at it. At the cost of being seen to cite a SF novel in defence…

By definition, if you're taking over the world by bribing humans to be your hands, you aren't killing all the humans. I"m not saying it's obviously going to be great. I'm saying that "extinction event" has a very specific definition, and this isn't it.

[dead]

Re: The contagion of fear

#94

Post by dumberquestions »

bloaf wrote: post #68
dumberquestions wrote: post #44

Earlier quoted context omitted.

Biology is extremely slow, try to extrapolate the previous 3 years of AI progress 10 years into the future, it's not a comparable trajectory.

Spoken like someone with a somewhat limited understanding of what it actually takes to calculate the fitness function of arbitrary organisms in an arbitrary environments.

Yeah it takes long because natural selection is a blind process starting from absolute zero and is only concerned with fitness, AI is a human guided process, has all of human progress as a starting point and is concerned with any number of goals we can create reward signals for, there's almost nothing you can extrapolate from one to the other.

Re: The contagion of fear

#95

Post by GlenTheMachine »

leoc wrote: post #90
GlenTheMachine wrote: post #69

Earlier quoted context omitted.

By definition, if you're taking over the world by bribing humans to be your hands, you aren't killing all the humans. I"m not saying it's obviously going to be great. I'm saying that "extinction event" has a very specific definition, and this isn't it.

It isn't true by definition: you could quite happily induce people to release a series of highly contagious bioweapons, after which those people would be surplus to requirements. What is true is that you're likely to need humans to sustain you and act for you for a few years to decades, so if you're not suicidal or deeply mad (and that is itself by no means self-evident) then total and immediate human extinction is p…

But if you're a rational actor, and you need humans to e.g. release your bioweapon, then clearly humans are capable of a bunch of stuff you still can't do. So you can't kill all the humans.

OTOH if you're a religious fundamentalist who thinks the End Times are near and just need a little shove, you can certainly use AI to design your weapon and recruit people to go release it. The difference being that religious fundamentalists aren't rational actors and aren't interested in self preservation.

Re: The contagion of fear

#96

Post by cindyllm »

deepwoods wrote: post #47
like_any_other wrote: post #3

The article neglects to consider the precautionary principle. It asks us to act as if AI is safe until it's proven to be dangerous, when the cautious approach is to act as if it's dangerous until it's proven to be safe.

This is a sensible approach for policymakers. For the average person, who might be panicking about this, opting not to have children or save for retirement, etc. as a result of doomerism in the media, it does not make a whole lot of sense. That is the problem here: Coxon and other concerned researchers - through no fault of their own - have no idea how the media works, and have just reached for the biggest possible m…

[dead]

Re: The contagion of fear

#97

Post by keeda »

I can't understand the stance that we do NOT have extraordinary evidence. I cannot stress enough that just ~4 years ago the concept of general purpose AI models that can do everything they are doing today was pure sci-fi.

And since then they have grown even more powerful than they were predicted to be, which, note, also faced a lot of skepticism at the time. The Hugging Face hacks and recent steamrolling of longstanding Math problems are just two recent pieces of extraordinary evidence.

And worse, people trust this technology because it behaves like people, but it actually works in ways nobody really understands, even exhibiting deeply weird and even disturbing characteristics (https://news.ycombinator.com/item?id=49635518) -- each of those quirks is extraordinary in itself.

And now we're rushing to give it control over the real world while deploying this powerful, quasi-chaotic technology in an infinite variety of ways everywhere in this highly vulnerable society.

I don't know what the standards for "extraordinary evidence" should be, but given such extreme unpredictability and rapid change, I fear it may end up being "an actual catastrophe".

Re: The contagion of fear

#98

Post by thin_carapace »

Jtsummers wrote: post #89
thin_carapace wrote: post #83

Earlier quoted context omitted.

certainly I behave like the article's author and yourself all the time, this isn't meant to be some sort of moral point. watching all these smart people talk so confidently regarding things nobody knows about, it reminds of the confidence ai shows in hallucination. would you say that the singular choice of vasili arkhipov did not prevent annihilation?

> would you say that the singular choice of vasili arkhipov did not prevent annihilation? He prevented a catastrophe, but not annihilation. We were not at risk of that in 1962 and even if he had decided to go with the others, more decisions would have been needed (not just his) to fully escalate to full scale nuclear war. I will reiterate: We have never been one decision away from full scale nuclear war. But in case…

I like the way you frame your opinion as a truth with an obligation to be understood. still I disagree that a decision can only be said to affect its direct successors.

Re: The contagion of fear

#99

Post by foobarian »

GlenTheMachine wrote: post #27

Like (I assume) most of you, I have been struggling with this. And where I currently come down is that 1) I am very worried, but 2) I am more worried about human actors. "AI" by itself won't kill us in the next ten years. I think. The reason I think that is that ten years from now, the tech economy won't be completely automated. I say this as a roboticist: as was adequately stated on a post earlier this week, robots…

> Of course, we've had the ability to extinct ourselves for decades

This gets mentioned often in various doomer narratives, but I question how true it is. A global thermonuclear war would be terrible and would bring us back to the stone age, but I reckon it would come far far short of causing mankind to go extinct.

Re: The contagion of fear

#100

Post by dash2 »

goatlover wrote: post #57
fasterik wrote: post #41

This is an excellent piece. Note that he is not saying that AI doesn't pose a risk. He's saying that it's irresponsible to make sensational, maximalist claims without strong evidence. If someone says that there's a 10% chance of human extinction by 2036, you can and should immediately stop taking them seriously.

They should also be able to give detailed reasons experts in the relevant fields can verify as to how they arrived at a 10% chance all humanity goes extinct. Not a science fiction narrative which likely does not take into account the relevant physical facts limiting such scenarios. Such as how exactly an AI would build a bioweapon capable of killing 8 billion humans across the planet.

What do you think of the AI-2027 scenario?
Post reply on HN