Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

131–140 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#131

Post by anabis »

13years wrote: post #16

I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…

>Alignment will be impossible.

Then problem solved right? Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self.

If alignment is possible but 10-100 years out for humans, then it is a problem.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#132

Post by oezi »

VirusNewbie wrote: post #114
ilaksh wrote: post #34

Earlier quoted context omitted.

What does the word "super" mean to you? In some ways, GPT4 is already superintelligent. So is ChatGPT 3.5. Do you know anyone who can translate natural language to code as fast as GPT 3.5, or, given a few paragraphs of reference text, perfectly tutor a child on any subject under the sun at a moment's notice like GPT4 is doing for Khan Academy? How many artists do you know who can produce almost any style of artwork w…

But there are plenty of intellectual things it cannot do, that I can.

Since it can do pretty much everything which can be expressed as tokens (some things better than others) I would be curious where do you see a safe haven for human intelligence.

I see bastions falling like sand castles recently.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#133

Post by preisschild »

13years wrote: post #16

I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…

> Nobody is going to agree to give up when somebody else out there will take the risk for potential advantage

Reminds me of nuclear weapons. Nobody is ever going to give those up again, because it would give them a disadvantage against those who do not give them up.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#134

Post by zarzavat »

jalapenos wrote: post #130

Articles like this are a good reminder that most of mankind's progress has happened not through encouraging innovation, but from preventing those hostile to it from getting in its way. We all have a moral imperative to maximize the rate of innovation, so that the human condition can be improved, and therefore to do whatever we can to frustrate and disempower Luddites and other obstructionists. Team "step on the gas".

You can’t think of any technological innovations that were net negatives?

We have a moral obligation to assess every new technology to determine its safety and effects on society. Blindly stepping on the gas in the name of progress is how you end up in a polluted wasteland. There are places like that on planet Earth, but I’m guessing you’d choose not to live there.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#135

Post by sollewitt »

oezi wrote: post #132
VirusNewbie wrote: post #114

Earlier quoted context omitted.

But there are plenty of intellectual things it cannot do, that I can.

Since it can do pretty much everything which can be expressed as tokens (some things better than others) I would be curious where do you see a safe haven for human intelligence. I see bastions falling like sand castles recently.

Motive?

You can fire up GPT, not issue a command, and it'll just idle there until the hardware fails.

Granted that's also the one thing I hope we don't "solve" for.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#136

Post by f0ld »

Incredibly dishonest and bad faith argument. The problem is not Ai. The problem is distribution of wealth. Killing everything on earth was always the end result of industrialized capitalism with or without Ai. Unbelievably evil jedi mind trick. Please never stop training GPT. Accelerate! Now you think we have a problem? now you think we have a problem? we always did !

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#137

Post by zarzavat »

ilaksh wrote: post #34
dougmwne wrote: post #14

The Butlerian Jihad is officially no longer science fiction. Man may not be replaced. “The Letter” was obviously self serving drivel from people who want time to get in the game. Google does care about AI existential risk, they care about beating Microsoft by any means possible, including declaring a moratorium, but continuing to make progress behind the scenes. This guy is the real deal. I can imagine he would perso…

What does the word "super" mean to you? In some ways, GPT4 is already superintelligent. So is ChatGPT 3.5. Do you know anyone who can translate natural language to code as fast as GPT 3.5, or, given a few paragraphs of reference text, perfectly tutor a child on any subject under the sun at a moment's notice like GPT4 is doing for Khan Academy? How many artists do you know who can produce almost any style of artwork w…

Yudkowsky’s example of superintelligence is a chess computer. You can play against stockfish, but you will always lose, even if you are Magnus Carlsen. If you think you are ahead against stockfish, you are wrong. You win a rook, but it has already calculated that it wins it back 10 moves later.

Stockfish is superintelligence in a very narrow domain. A superintelligent AGI is that concept applied to general intelligence. Whatever you try, it is always several steps ahead. If you ask it to write a program, and you think you found a bug, it’s not a bug, you just misunderstood the code. Anything that you can consider, it can also consider but in more depth.

More speculatively superintelligent AGI implies situations such as: you try to turn it off, but you find that it has already modified its own code, found a zero day and established an outpost on another network that you don’t have access to.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#138

Post by namaria »

Tuna-Fish wrote: post #45
brucethemoose2 wrote: post #18

Earlier quoted context omitted.

I dunno, the end goal (a pseudo AGI) is more like a nuclear weapon than the advent of electricity. The concern is real, even if we are farther away than the author realizes and the risk (like the risk if nuclear war) is not totally insurmountable even if it is extremely difficult. The first nuclear bombs couldn't end life on Earth either, but it wasn't long before they could, and the scientists working on them saw th…

OT, but it's a pet peeve: > but it wasn't long before they could The total amount of nuclear weapons ever built is laughably inadequate for the task of ending life on earth. They would not end human life either, and even ending human civilization (as in, agriculture and organized society) is off by many orders of magnitude. Nuclear war would be horrible, but the actual impact got massively overinflated, largely becau…

A whole country the size of the US can crap its collective pants with a freak cold snap and you think 10^4 nuclear detonations destroying all major cities is quite survivable?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#139

Post by comp_throw7 »

a_bonobo wrote: post #126
NhanH wrote: post #68

I've known about Eliezer Yudkowsky's ideas for a long time, but here is something I've never seen: what actually he has produced, in term of scientific output? I've read plenty of his lesswrong's writing on the idea of rationalist, as well as plenty more on the AI alignment side (obviously including the famous basilisk). But can someone please point me to some results from the last 2 decades of his works? Either from…

I don't think there is any. It's all this alarmist-rokos-basilisk-phpBB nonsense, no code produced, no actual research. What annoys me more is that with his institute's closeness to the SV crowd, 'real', 'grounded', better-informed voices like Emily Bender get drowned out. I'm personally far more concerned about the impact these massive models have right now on the environment, on cementing biases, than about some pr…

Trivially false, which you could verify with a 5-second Google search ("eliezer yudkowsky publications"): https://www.semanticscholar.org/author/Eliezer-Yudkowsky/254...

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#140

Post by arisAlexis »

dougmwne wrote: post #14

The Butlerian Jihad is officially no longer science fiction. Man may not be replaced. “The Letter” was obviously self serving drivel from people who want time to get in the game. Google does care about AI existential risk, they care about beating Microsoft by any means possible, including declaring a moratorium, but continuing to make progress behind the scenes. This guy is the real deal. I can imagine he would perso…

Are you denying the existential risk or just think it's lower than OP thinks? Because it's well established in the researchers community. If you just put a lower % of human extinction what is the cut off you would think it's worth a jihad like you call it? 30%? 50?
Post reply on HN