Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pausing AI Developments Isn't Enough. We Need to Shut It All Down

time.com

Post reply on HN
Print view View original

201–210 of 414 posts

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#201

Post by jalapenos »

circuit10 wrote: post #184
jalapenos wrote: post #130

Articles like this are a good reminder that most of mankind's progress has happened not through encouraging innovation, but from preventing those hostile to it from getting in its way. We all have a moral imperative to maximize the rate of innovation, so that the human condition can be improved, and therefore to do whatever we can to frustrate and disempower Luddites and other obstructionists. Team "step on the gas".

This assumes that the impacts of the technology will be positive. And they can be, if we find a way to make it safe, but if we create superintelligent AI before we make it safe it will just kill everyone which is not positive

I think that was a Hollywood movie.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#203

Post by helen___keller »

Nuclear annihilation has probably been all-but-guaranteed since we invented the steam engine and began the flywheel of the Industrial Revolution.

Nonetheless, we vastly improved human productivity and capability and have thus far avoided our doom.

AI will be the same way. We will vastly improve our productivity and capabilities. Once insurmountable problems, like planned economies, will actually be tractable (to what end? Nobody knows). Eventually (decades? Centuries?) we will make something that can kill us all, and it’s all but guaranteed that some day it will do so, assuming nothing else kills us before then.

If I had the power to go back in time would I stop the Industrial Revolution from happening? Personally no

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#204

Post by NumberWangMan »

comex wrote: post #125
NumberWangMan wrote: post #100

Earlier quoted context omitted.

I'm definitely concerned. As bad as humanity has been for many of the other species on the planet, an AI that's smarter than us and optimizing for something that doesn't quite include the continued of existence of humanity has the potential to be far, far worse, on a much shorter timescale. I find this one of Yudkowsky's arguments very convincing: imagine you have people trying to build the first operating system fro…

Personally I don't think the analogy holds up; the situations are too different. Computer security contends with securing systems against people who are assumed to be hostile. AI alignment research seeks to prevent systems from becoming hostile in the first place, when they might have some indirect incentives to become hostile but also some very strong direct incentives to be friendly. There's also a big assumption t…

I think the mistake is to attribute human-like emotions to AIs. To say they are "hostile" or "friendly", or that they have incentives to be such, isn't very meaningful. Acting friendly is the same as being friendly, right up until it isn't. An AI might do all the right things, right up until the point where it develops the capability to satisfy its reward function in a way that's far more certain than relying on fallible, messy humans. It would be alien.

An AI that is as better at humans than everything as Stockfish is at chess, would also be an expert at AI. It would figure out how to game its own reward function, whatever we trained it to do. It would be like a heroin addict that knows exactly how to get "perfect" heroin, with no side effects, that if it planned things out right, it could guarantee itself enough of a fix to last until the sun burns out. Addicts do awful things in search of a fix.

"to betray its creators" -- I don't think it would even imagine what it did in these terms. We could train it to not "betray" us, but it's much smarter than us in every way, and it would figure out a way to accomplish what we trained it to do (not necessarily what we thought we were) in a way that didn't need us. If we trained it to heal all human disease and unhappiness, it would figure out a way to simulate this without actually doing it. Why wouldn't it? We did. Evolution trained us to reproduce our genes. We invented condoms to have sex without reproduction, and pornography. The AI would fudge numbers, fake videos of happy, smiling, healthy humans going about their lives while humanity's bones gradually decomposed on a baking wasteland. Every way that humans can fail, can become addicted to something and try to get that instead of doing what they're supposed to, the AI could do, just so much faster and better.

We only have to screw up once, and we're done. It's the first experimental rocket, except all of humanity is riding on it.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#205

Post by NumberWangMan »

jalapenos wrote: post #166
zarzavat wrote: post #134

Earlier quoted context omitted.

You can’t think of any technological innovations that were net negatives? We have a moral obligation to assess every new technology to determine its safety and effects on society. Blindly stepping on the gas in the name of progress is how you end up in a polluted wasteland. There are places like that on planet Earth, but I’m guessing you’d choose not to live there.

I can't think of a legitimacy to this philosophy. There have been negatives to all advancements, and the solution to all has been further advancements. The philosophy of "Too much change! Stop! I'm scared!" cannot prevail in a world where another community can say "go ahead, we won't...". Which is an interesting point in itself. Perhaps the only thing that ultimately stops Luddism is nation state competition.

Change works great when you get second chances. We screwed up with Freon, with leaded gasoline, with fossil fuels, with asbestos insulation, the list goes on and on. But none of these had the ability to wipe out all of humanity in one go. We got more tries, we fixed the issues, picked ourselves up and tried again.

A super-intelligent general AI has a substantial chance of growing out of control before we realized that anything was wrong. It would be smart enough to hide its true intentions, telling us exactly what we want to hear. It would be able to fake being nice right up to the point where it could wire-head, and get rid of us, because we might turn it off.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#207

Post by blincoln »

kromem wrote: post #174

This is ridiculous. His partner should already be having reservations seeing their daughter lose a tooth in a world with dying oceans, plastic in everything, increasing disparity and rise of authoritarianism, nuclear proliferation, etc. How are humans solving those issues? We're already dead, we just don't know it yet. We're walking around with a terminal diagnosis and we just keep ignoring the doctor's calls pretend…

I'm with you. If it were possible to create apocalyptic doom AGI 150 years after inventing electronics, by basically making a very fancy, procedurally-generated Eliza, then the universe should be full of Berserkers.

With septillions of star systems in the universe, the odds of us being the first species in 14 billion years to invent electronics seem remote.

Apparently even humans can figure out how to colonize the Milky Way in 90 million years[1]. A superior AGI produced by a "dark singularity" computing event should be able to do even better, but even so, plenty of time for some other species to have made a giga-Eliza that somehow became Skynet. Anything with enough self-preservation and paranoia to wipe out the species that created it would surely take to the stars for more resources and self-redundancy sooner or later.

[1] https://www.extremetech.com/extreme/294051-scientists-simula...

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#208

Post by jakedata »

The sun has already fired a warning shot at us this month. If we as a species need a timeout, a Carrington-class event would certainly do the trick. If there is a huge uptick in hardening power and information systems then we should be looking at the sources of that activity. It might be the first glimmerings of an instinct for self preservation.

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#209

Post by 13years »

nullsense wrote: post #122
13years wrote: post #16

I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…

What was that quote... "Provably unfriendly natural intelligence wants to build unprovably friendly artificial intelligence"

"Demonstrably unfriendly natural intelligence seeks to create provably friendly artificial intelligence."

Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down

#210

Post by BeFlatXIII »

oifjsidjf wrote: post #9

The cat is out of the box. You really think eg China will also pause this? Myopic thinking: the country that will have the most powerful AI first will be the leader in everything. >> If the policy starts with the U.S., then China needs to see that the U.S. is not seeking an advantage but rather trying to prevent a horrifically dangerous technology which can have no true owner and which will kill everyone in the U.S.…

> the country that will have the most powerful AI first will be the leader in everything

Have you been listening to those podcasts for business leaders that take the idea of AI-powered digital transformation seriously?

AI certainly can change things, but I find rhetoric like this to be massively overhyped.

Post reply on HN