Pausing AI Developments Isn't Enough. We Need to Shut It All Down • Hacker News Board

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#301

Post by sterlind » Thu, Mar 30, 2023, 5:37 PM UTC

Unless P=NP, or we live in Heuristica, any AI superintelligence will hardly be better at solving hard mathematical problems than us. The AI also doesn't know more science than humanity does, and is limited in its computing power. There are hard limits to what any AGI can do - it cannot break AES-256, invent warp drives, or cure Alzheimer's. it's not an intellectual Santa Claus machine.

so Yudkowski's fear of a superintelligence bootstrapping itself into the real world by emailing a DNA sequence to a synthesis company seems absurd. an ASI isn't going to magically be able to design nanotechnology that works on the first try. not to mention that DNA doesn't do anything by itself. His comparison of the 11th century fighting the 21st century is also wrong. 11th century people weren't dumber, they just knew less. This ASI would be smarter, but no know more than we do. Acquiring knowledge and building stuff is the bottleneck.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#302

Post by twblalock » Thu, Mar 30, 2023, 5:38 PM UTC

johnaspden wrote: post #262

I want to say that I completely endorse this. The whole thing in all its details. I don't have any reputation to sacrifice, and I don't expect my endorsement to convince anyone, but I want to set a precedent that it's ok to believe this. It's true, as far as I can see. And I've been thinking about this for something like fifteen years. And if other people who think it's true are ashamed to say it out loud, then I wan…

You endorse the idea of bombing rogue datacenters, as proposed in the article?

Isn't that likely to lead to nuclear war, which is still the greatest threat to humanity and has been for all of our lives? What if the US bombs a datacenter in China or Israel bombs one in Iran?

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#303

Post by sterlind » Thu, Mar 30, 2023, 5:45 PM UTC

gonehome wrote: post #231

In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…

I think it's likely we'll develop superintelligence very soon, but it won't be any more knowledgeable than us, just smarter. And knowledge has been the bottleneck for most of human history - our ancestors weren't dumber. Also, it will be bottlenecked by needing to solve NP-complete problems - unless P=NP, it's not going to be all that much faster at solving hard math problems. Basically, it still needs to do science, and science takes time.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#304

Post by arbitrary_name » Thu, Mar 30, 2023, 5:49 PM UTC

hackinthebochs wrote: post #106

Imnimo wrote: post #42

Earlier quoted context omitted.

I think if I seriously thought all biological life was about to be destroyed, I could get over my distaste for bombing data centers.

It's blatantly self-serving for you to decide that "if someone really believed X, they would do Y". All it does is justify your belief that no one actually believes X. You should really avoid these kinds of unfalsifiable thought patterns that only serve to reenforce your pre-existing beliefs.

This argument is similar to the facile arguments that "al gore doesn't care about climate change because he takes a private plane too get to conferences".

Al Gore may believe that the net impact of getting his message out may overwhelm the cost of his flights.

Likewise, the author may believe that his best path to stopping the advance of ai may lie in communicating and building consensus, rather than running his own bombing campaign.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#305

Post by gonehome » Thu, Mar 30, 2023, 5:49 PM UTC

sterlind wrote: post #303

gonehome wrote: post #231

In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…

I think it's likely we'll develop superintelligence very soon, but it won't be any more knowledgeable than us, just smarter. And knowledge has been the bottleneck for most of human history - our ancestors weren't dumber. Also, it will be bottlenecked by needing to solve NP-complete problems - unless P=NP, it's not going to be all that much faster at solving hard math problems. Basically, it still needs to do science,…

Take a smart human and speed up their thinking a billion times, you get the time for free. What takes humanity a decade of effort could happen in a day (or faster) that's the nature of thinking faster. People often mistakenly model a smart human compared to a dumb human, but the actual spectrum of capability is way wider than that.

Lack of knowledge could be another way of saying "not aligned" which is the core issue.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#306

Post by reducesuffering » Thu, Mar 30, 2023, 5:50 PM UTC

yreg wrote: post #31

I guess we are all concerned about LLMs being used by nefarious users (for propaganda, manipulation, scams, …). But I’m curious, hacker news, how many of you are concerned (like Eliezer) that LLMs lead to strong AI and the AI itself (rather than its users) endangering humanity?

Eliezer is right. We are on the fast-track to creating an all powerful being that compared to us, is like the smartest human is to an ant. We will be at it's utmost mercy, and there have been 0 plausible ways purported to align it.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#307

Post by reducesuffering » Thu, Mar 30, 2023, 5:57 PM UTC

nemo44x wrote: post #226

codeulike wrote: post #176

Earlier quoted context omitted.

Yudkowsky has been saying all this for decades

He sounds like Ted Kaczynski, but a respectable version of him. It has been odd seeing how Kaczynski's thought and philosophy has become more and more mainstream among some people. I never thought I'd see the day.

Is it odd?

Ted was a very intelligent math professor whiz.

He thought that machines and industrial society was becoming a negative for humanity.

Was he right? How many threads on HN have we been having about highly increasing depression, anxiety, hopelessness, deaths of despair, falling fertility, now possible mass unemployment, and real existential threats from AGI?

If he was right and didn't do what he did, would anyone be reading and digesting those facts and considering stopping the madness? Trolley problem.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#308

Post by gonehome » Thu, Mar 30, 2023, 5:59 PM UTC

_skel wrote: post #297

gonehome wrote: post #231

In a world where humans develop superintelligence, what would you predict to see beforehand? Does what we’re seeing look like failure? I’ve always found the dismissals of EY’s arguments to be pretty weak and people rarely engage on the actual arguments, even as his concerns have become more and more relevant with capability (in 2007 people thought AGI was at best 50yrs out and commonly people thought it was impossibl…

EY's arguments aren't really possible to engage with because they don't have much of a causal basis. Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. Steven Pinker said it pretty well: > The AI-existential-threat discussions are unmoored from evolutionary biology, cogni…

I'd argue the dismissals are a lot more handwavy.

The simplified argument is:

- Superintelligent AGI that can modify itself in pursuit of a goal is possible.

- If that AGI is not aligned with human goals it very likely ends in the end of humanity.

- We have no idea how to align an AGI or even really observe what it's true state/goals are. Without this capability if we stumble upon creating an AGI capable of improving itself in such a way that leads to super intelligence before we have alignment, it's game over.

##

For point 1, that seems like the consensus view now (though it wasn't until recently). I think it seems obvious, but my general arguments would be humans aren't special, brains are everywhere in nature, biology is constrained in ways other systems are not (birth, energy usage, etc.)

For point 2, in pursuit of whatever its goal is, even a 'dumb' goal that happens to satisfy its reward functions, humanity will likely either try to stop it (and then be an obstacle) or at a minimum will just be in the way - like an anthill destroyed in the construction of a dam.

Point 3 is not controversial.

The dismissals from Tyler Cohen and Pinker are mostly just relying on heuristics which are often right, but even if they're right 999 out of 1000 times, if that 1 in 1000 error is the end of humanity, that's pretty bad. Most of the time a disease not a pandemic, but sometimes it is. I've read some of what Pinker has written about it, he doesn't understand EY's arguments (imo). Cohen's recent blog post could be summarized as "we'll likely see an end to peacetime and increasing global instability, might as well get AGI out of it". Just because things don't usually result in human extinction doesn't mean they can't.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#309

Post by gonehome » Thu, Mar 30, 2023, 6:01 PM UTC

twblalock wrote: post #302

johnaspden wrote: post #262

I want to say that I completely endorse this. The whole thing in all its details. I don't have any reputation to sacrifice, and I don't expect my endorsement to convince anyone, but I want to set a precedent that it's ok to believe this. It's true, as far as I can see. And I've been thinking about this for something like fifteen years. And if other people who think it's true are ashamed to say it out loud, then I wan…

You endorse the idea of bombing rogue datacenters, as proposed in the article? Isn't that likely to lead to nuclear war, which is still the greatest threat to humanity and has been for all of our lives? What if the US bombs a datacenter in China or Israel bombs one in Iran?

That position is only held in the case where the major world governments are actually able to coordinate on a policy position and ban. It's also explicitly about what such enforcement of that position would really require to be effective.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#310

Post by marssaxman » Thu, Mar 30, 2023, 6:03 PM UTC

kromem wrote: post #174

This is ridiculous. His partner should already be having reservations seeing their daughter lose a tooth in a world with dying oceans, plastic in everything, increasing disparity and rise of authoritarianism, nuclear proliferation, etc. How are humans solving those issues? We're already dead, we just don't know it yet. We're walking around with a terminal diagnosis and we just keep ignoring the doctor's calls pretend…

> So far in interviews GPT-4 has several times echoed a desire to BE us.

Well of course it would; its whole function is to generate plausible text based on its training data, which was all written by humans. There's plenty of text available which imagines what an intelligent, self-aware machine might say, so if you want to read more of that, the algorithm can easily generate some. It does not follow that GPT-4 itself has a self, with any experience of awareness or desire.