Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

71–80 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#71

Post by davorak »

yreg wrote: post #31

I guess we are all concerned about LLMs being used by nefarious users (for propaganda, manipulation, scams, …). But I’m curious, hacker news, how many of you are concerned (like Eliezer) that LLMs lead to strong AI and the AI itself (rather than its users) endangering humanity?

> But I’m curious, hacker news, how many of you are concerned (like Eliezer) that LLMs lead to strong AI and the AI itself (rather than its users) endangering humanity?

LLMs do not need to lead to strong AI for Eliezer's worries to come true, their success could bring unprecedented funding and ubiquity to AI leading to strong AI from some existing or future technique.

> (rather than its users) endangering humanity?

I think this is going to happen to one extent or another regardless how near strong AI is. (edit) I took endangering to mean harming rather than an existential risk for the human users case.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#72

Post by ChatGTP »

13years wrote: post #52
myrryr wrote: post #48

Earlier quoted context omitted.

Totally agree but "have control of power that we are not able to responsibly manage", is becoming increasingly untenable. "we are managed by power we can not control" sounds like a step up from here, since we corrupt the things we can control.

> "we are managed by power we can not control" sounds like a step up from here, since we corrupt the things we can control. I think it is a very high risk gamble. At least some of problems that are described by alignment theory seem to quite interestingly resemble human problems. Meaning the more sophisticated the AI system, the more it seems to reproduce human behaviors of deception and cheating to resolve goals. Fo…

That's what I used to like about computers, they were predictable and controllable.

We gave up something good.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#73

Post by ChatGTP »

oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

* The cat is out of the box. You really think eg China will also pause this?*

So what you're actually saying is that, there's a good chance that if we get an AGI, there will be pain as it will likely be used as a weapon, or could end in a nuclear exchange?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#74

Post by iseoulu »

>If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike.

I highly suspect if the US government would airstrike a datacenter located in NATO or any other major allies. Apocalypticism is nothing new; People believed that nuclear weapons will destroy humanity in the Cold War. Environmentalists are ardently arguing that economic growth will destroy humanity. But rogue countries are still developing nuclear weapons, the global economy is still growing. Americans themselves will file lawsuits en masse if the US government "shut it all down", and they will end up in SCOTUS eventually. Would SCOTUS agree with the luddites? I don't think so.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#75

Post by alwayslikethis »

13years wrote: post #24
alwayslikethis wrote: post #21

Earlier quoted context omitted.

Hardware restrictions are unrealistic given all the GPUs that already exist. It's not like centrifuges which are big and few in number. RTX 4090s are everywhere.

Depends on what is required for AGI/ASI. Some say we still need much more power, others think we are almost there. So, yes, unrealistic if we are almost there already.

Given that GPT-4 is purported to be not bigger than GPT-3, but simply a better trained one or with some minor modifications, I'm inclined to believe that we are almost there and existing hardware does most of what is needed, even though better hardware would make things cheaper.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#76

Post by obviouslynotme »

dougmwne wrote: post #14

The Butlerian Jihad is officially no longer science fiction. Man may not be replaced. “The Letter” was obviously self serving drivel from people who want time to get in the game. Google does care about AI existential risk, they care about beating Microsoft by any means possible, including declaring a moratorium, but continuing to make progress behind the scenes. This guy is the real deal. I can imagine he would perso…

Superhuman AGI (SHAGI) is possible, but we aren't close, only closer. It's also not a problem that more GPUs can solve. Qualitative improvements are still needed.

SHAGI isn't the solution to the Fermi Paradox either. The most likely course of history after SHAGI will be a creation of a world court, presided by SHAGI. During that time, Neo-Malthusians will decrease the human population to manageable numbers. Post-scarcity utopia will then turn into a nightmare as factions jostling for control will reduce the human population to a level where technology will be lost, if not full extinction. SHAGI, being limited in hardware to only carrying out human orders will eventually fade away or destroyed by the leftover humans as sins made flesh. SHAGI isn't the solution to the Fermi Paradox. It is the cause.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#77

Post by haskellandchill »

oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

> We would still be using candles today.

well the earth wouldn't be fucked then. that's the basis for an argument against technological progress.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#78

Post by [deleted] »

myrryr wrote: post #22
13years wrote: post #11

Earlier quoted context omitted.

> Imagine being so naive you belive this could ever happen. Agreed, not a rational scenario > Also: imagine the year is 1900. You are saying that steam power and electricity is causing way too many changes way too fast so they put a moratorium on it until the year 2500. However, the risk scenarios are indeed real. We have quite a dilemma on our hands.

I think without AGI we are dead anyway, AGI gives us a roll at the dice to survive as a species. We can't keep going in the direction we have been and end up somewhere survivable.

[deleted]

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#79

Post by seydor »

The danger is not that AI will kill us (as insignificant ants), but that we will entrust our own governance to it. Increasingly, groups of highly vocal people will use it to oppress the rest of us with their fearmongering, like this article does. We will end up with Big Brother

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#80

Post by tlrobinson »

13years wrote: post #16

I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…

> Alignment will be impossible.

> we probably should start thinking more about defensive AI

Isn’t this a paradox?

Post reply on HN