Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

401–410 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#401

Post by tanseydavid »

oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

>> You are saying that steam power and electricity is causing way too many changes way too fast so they put a moratorium on it until the year 2500.

There is an earlier post that casually calls the internal combustion engine a "moronic invention."

I do not have words.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#402

Post by mitthrowaway2 »

tanseydavid wrote: post #398
mitthrowaway2 wrote: post #317

Earlier quoted context omitted.

> Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. EY frequently does propose possible sequences of events, but he also very correctly points out, every time, that any specific and detailed story is very unlikely to be correct because P(A*B*C*D) So here's one method, ju…

>> With the last humans gone, the AI takes ownership of its bank account...and begins crediting money into it freely. What possible need would the AGI have for money once all of the humans are gone? (asking for a friend ;)

How should it know? Increasing its bank balance was merely the task it dutifully set out to accomplish, per instructions. Everything else is just a means to that end.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#403

Post by nullsense »

sorokod wrote: post #322
qgin wrote: post #200

Earlier quoted context omitted.

That’s true, when have humans as a whole ever been consistently aligned with the interests of humans?

when mutually assured destruction became a thing.

It's kind of a low bar that the only thing we can all agree on is that we don't kill absolutely everyone. A lot of people is fine, but not everyone.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#404

Post by G_z9 »

13years wrote: post #389
G_z9 wrote: post #370

Earlier quoted context omitted.

[flagged]

> stop brushing aside real solutions without any logical reason for doing so Sure, point me to the research papers and discourse on the solution you propose.

There is no discourse you fucking idiot because everyone is rejecting the obvious. That’s like saying you discovered fire and then they say you didn’t because where is the discourse. All things start small. Think from first principles. Use your fucking head

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#405

Post by zarzavat »

bick_nyers wrote: post #368
zarzavat wrote: post #137

Earlier quoted context omitted.

Yudkowsky’s example of superintelligence is a chess computer. You can play against stockfish, but you will always lose, even if you are Magnus Carlsen. If you think you are ahead against stockfish, you are wrong. You win a rook, but it has already calculated that it wins it back 10 moves later. Stockfish is superintelligence in a very narrow domain. A superintelligent AGI is that concept applied to general intelligen…

"Anything that you can consider, it can also consider but in more depth." I think it's important to note the distinction between "it can also consider" and "it did also consider". Super Intelligence is not the same as Infinite Intelligence, there are still physical limitations and time components that can still get in the way. It would be helpful to be able to quantify the speed of intelligence, and the idea surface…

Yudkowsky makes this distinction. Stockfish is not always correct: it can be beaten by next year’s Stockfish for example. In some sense it is making mistakes all the time. It’s just that those mistakes are not accessible to us humans. It is operating in a much higher plane of understanding compared to us.

A “mistake” to stockfish looks like: I searched 30 ply down but my opponent searched 35 ply down and found a superior sequence of moves.

For stockfish to make the kinds of chess mistakes that humans make, it would similar to if I failed to calculate 123+123=246. It’s not that 123+123 is particularly easy on the grand scale of intelligence: animals cannot do it. But it’s completely inconceivable that I could make that kind of mistake.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#406

Post by CryptoChronicon »

Eliezer makes some very compelling logical arguments on the case for approaching AI with extreme caution. It stands out to me as an odd signal that that he chose to refrain from signing the letter because it asks for too little. If one were truly convinced that AI would end humanity, signing the letter to put the research on hold seems to be one tangible step that could be made in the direction of saving the human race. As someone who is very concerned about safety, to me this detail stands out and appears as grandstanding.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#407

Post by calf »

jamestimmins wrote: post #29

I've used GPT-4, and while extremely impressive, it doesn't feel like we're all that much closer to super-intelligent AI than we were last month or last year. It feels like Google on steroids, but the gap between GPT-4 and AGI still feels massive. This seems like so much putting the cart way ahead of the horse.

The concern is that there could be an exponential advancement in AI over the next few decades. So GPT looks stupid right now, stochastic parrot and all, but after 15, 30 years? I'm on board with some kind of precautionary principle applied on an international scale.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#408

Post by FeepingCreature »

dougmwne wrote: post #14

The Butlerian Jihad is officially no longer science fiction. Man may not be replaced. “The Letter” was obviously self serving drivel from people who want time to get in the game. Google does care about AI existential risk, they care about beating Microsoft by any means possible, including declaring a moratorium, but continuing to make progress behind the scenes. This guy is the real deal. I can imagine he would perso…

> I doubt super intelligent AGI is possible anyway. If it were, it would be the solution to the Fermi paradox and all matter in our galaxy would be paperclips already. The Anthropic Principle saves the day.

Can't help but notice we seem to be the first species in our lightcone to evolve, wonder why that is...

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#409

Post by FeepingCreature »

kvetching wrote: post #217
oifjsidjf wrote: post #8

Earlier quoted context omitted.

This is the one moment he has been waiting his entire life for. He literally waited his entire life for this moment, obviously he's gonna milk it to every drop.

The thing is, there's no evidence that the current path will lead to superintelligence. It could be the case that despite feeding a trillion dollars in compute / data to a model, that we still need to hold it's hand to do anything useful. He jumped the gun and he's really tarnished his reputation. How can anyone take him seriously after the insane rhetoric and hyperbole of this article?

How can you see the clear jumps in intelligence from GPT-2 to 3 to 4 and not only not believe that this leads to superintelligence, but see no evidence that this leads to superintelligence?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#410

Post by FeepingCreature »

mhh__ wrote: post #330
yreg wrote: post #191

Earlier quoted context omitted.

What credentials beyond "thinking about this problem a lot" would satisfy you? He works at his own research institute dedicated to this topic, wrote some books on it and other (self proclaimed) alignment experts take him seriously.

I think he's a pseud with no real connection to AI research. I'm not saying he's stupid or anything like that I just don't see any useful scientific output from him.

I mean, his whole thing is "progress on AI research leads to the apocalypse." He's done some work on decision theory, but they're not connected to DL, presumably because he thought "well if DL works out in the short term we're all dead anyways." Which seems to be what's happening.
Post reply on HN