Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

191–200 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#191

Post by yreg »

mhh__ wrote: post #175
ftxbro wrote: post #5

If anyone can't get the article, this one is by Eliezer Yudkowsky. He has pretty much devoted his life to thinking about how to not let AI get powerful and destroy all humans, and now there is this wild race among the largest corporations and possibly governments to be the first to get the most powerful superhuman AI system. Obviously he is disturbed by such a development.

Thinking about or thinking about thinking about, does he have any credentials at all other than basically being early

What credentials beyond "thinking about this problem a lot" would satisfy you? He works at his own research institute dedicated to this topic, wrote some books on it and other (self proclaimed) alignment experts take him seriously.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#192

Post by flangola7 »

VirusNewbie wrote: post #114
ilaksh wrote: post #34

Earlier quoted context omitted.

What does the word "super" mean to you? In some ways, GPT4 is already superintelligent. So is ChatGPT 3.5. Do you know anyone who can translate natural language to code as fast as GPT 3.5, or, given a few paragraphs of reference text, perfectly tutor a child on any subject under the sun at a moment's notice like GPT4 is doing for Khan Academy? How many artists do you know who can produce almost any style of artwork w…

But there are plenty of intellectual things it cannot do, that I can.

There are other animals (mammals) that have better memory than us. There are animals strongly suspected to have deeper and more sophisticated social relationships than us. We are not the apex in every intellectual ability, but the ones in which we are grant us absolute power over the future of all other lifeforms.

A cognitive entity does not have to best you at all things. There are standardized education tests it may never reach above 10th percentile on, just as humans will never reach above 10th percentile in the short term memory tasks that apes are masters of. But we are the 100th percentile for tasks like industrialized destruction of them and their habitats and capturing and using them for painful medical experiments - the apes are wholly outclassed when it comes to that.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#193

Post by aimbivalent »

The cat isn't going back into the bag. Doubly so if AGI really just requires a bigger neural net.

Let says we had, globally, the political will to "ban" AI. How would that even work in practical terms? Are we going to control the production and distribution of GPUs? Is that going to work better or worse than controlling nuclear proliferation given that there are billions of processors already out there?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#195

Post by layer8 »

f0ld wrote: post #155

They really hate millennials don’t they? They’re really serious about it. Can’t let them do anything. Need to ‘stop’ what you’re doing LMFAO if something has to shut down it’s privatized ICBM.

Yudkowsky is almost a millennial (off by two years).

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#196

Post by layer8 »

yreg wrote: post #31

I guess we are all concerned about LLMs being used by nefarious users (for propaganda, manipulation, scams, …). But I’m curious, hacker news, how many of you are concerned (like Eliezer) that LLMs lead to strong AI and the AI itself (rather than its users) endangering humanity?

AGI will likely incorporate LLMs as a significant element. The issue is that we are far from having any good understanding of how and why LLMs achieve what they achieve, and unless that changes, the same will be true for anything built on them or of their future evolution. If we continue to pursue progress by only caring about results and disregarding that we don't really understand the mechanisms, then we may very well run into the fatal scenario described by Yudkowsky.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#197

Post by andrewstuart »

Man people are getting really sci fi about this stuff.

Take a chill pill dude, in five years time we won’t think of it much differently to how we thought of Google last year…. a tool for getting something done.

These sci fi fantasies are really a total overreaction.

If you like the story in this Time magazine article go watch The Terminator, same thing.

Obligatory perfect Simpsons clip: https://youtu.be/RzybAS7zltE

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#198

Post by maxerickson »

Analemma_ wrote: post #46
Imnimo wrote: post #42

Earlier quoted context omitted.

I think if I seriously thought all biological life was about to be destroyed, I could get over my distaste for bombing data centers.

No, that doesn't follow. It's possible to think that all biological life was about to be destroyed but also believe some combination of 1) it's hopeless to bomb data centers because I'd quickly be stopped by the authorities and remaining data centers would beef up security, and nothing would be accomplished, 2) in the past, other people who have had beliefs of this sort ("violence is the only answer") have been wrong…

I think of this as the inordinate power of advocacy.

It can seem useless to advocate some position that is far out there, but say you convince a million people to use a little less plastic. That's gonna reduce plastic use more than anything you can personally accomplish.

Of course in this scenario you don't have to convince some people, you have to convince everyone, and then especially the people you think are the worst actors, so it's sort of unlikely to be effective.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#199

Post by VeninVidiaVicii »

nullsense wrote: post #122
13years wrote: post #16

I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…

What was that quote... "Provably unfriendly natural intelligence wants to build unprovably friendly artificial intelligence"

The quote you’re thinking of is from computer scientist and researcher Stuart Russell:

"The point of [AI alignment] is to ensure that the machines we create will be aligned with human values. And the reason we have to worry about it is that if we create machines that are more intelligent than we are, it's quite possible that those machines will have goals that are not aligned with our goals. In fact, they may not have any goals at all that we can understand or influence. This is the so-called 'provably unfriendly' scenario, where the machine has no motivation to do what we want, but is able to prevent us from interfering with its goals. The problem is that if we build machines that are provably unfriendly, then we will never be able to build machines that are 'provably friendly', because the unfriendly machines will always be able to prevent us from proving that they are friendly."

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#200

Post by qgin »

nullsense wrote: post #122
13years wrote: post #16

I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…

What was that quote... "Provably unfriendly natural intelligence wants to build unprovably friendly artificial intelligence"

That’s true, when have humans as a whole ever been consistently aligned with the interests of humans?
Post reply on HN