Live data from Hacker News

Among the A.I. doomsayers

newyorker.com

91–100 of 374 posts

Re: Among the A.I. doomsayers

#91
post #32

Earlier quoted context omitted.

Why haven't "hackers" hacked into nuclear weapons infrastructure? The street cred would be incredible! I think it's possible to build secure anything if that is the goal. Most software is written by people motivated to write something useful and/or neat, or collect a paycheck, and don't focus on security.

I’m pretty sure Stuxnet did hack it’s way into nuclear weapons infrastructure.

A refinement facility for nuclear material isn’t quite the same thing

Re: Among the A.I. doomsayers

#92
post #73

Earlier quoted context omitted.

Even if a terrifying AGI rises up and escapes our control, I am still eternally grateful that Prometheus stole fire from the gods and gave it to us in the form of technology, knowledge, really civilization.

So that we can record our demise in high definition? Technology is amazing, and we have our civilization to be thankful for. But there are definitely technologies that in retrospect, if we had dodged them on the tech tree, we would be better off. Life would have continued just fine without many technologies, and I think AGI is one of them. We do not need AGI to survive and grow, and therefore any risk it imposes shou…

I think this perspective ignores that the only way to semi permanently improve the lives of future humans is technology, and that quality of life is of similar importance to survival, especially at its extremes. AGI, if we achieve it, could genuinely make human lives much better.

Re: Among the A.I. doomsayers

#93
post #81

Earlier quoted context omitted.

> ... something more like how humans get along with dogs, cats, cattle, insects, or microbes? There's some MASSIVE selection bias here. Note that we're significantly more capable than anything you listed. Where are the Neanderthals? They're all long dead, because they were credible competition but we were just better. In a hypothetical scenario of superintelligent or supercapable AI imbued with some physical capabili…

AI has no evolutionary pressures imposed on it like humans and their brains so there is no reason to expect it to be an ultra violent predator like humans.

AI has no evolutionary pressures imposed on it

Are you sure? https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...

Just wait until we start putting these models inside robots, or world simulators, and having them interact with each other.

Re: Among the A.I. doomsayers

#94
post #90
post #81

Earlier quoted context omitted.

AI has no evolutionary pressures imposed on it like humans and their brains so there is no reason to expect it to be an ultra violent predator like humans.

Anything that reproduces with survivable mutations, and consumes resources, is immediately subject to evolutionary pressure. Neither of those require biology.

The only things currently consuming resources are humans and at the rate we are consuming resources we would need 2 Earths to support that rate of consumption. Worrying about self-reproducing AI is putting the cart before the horse. No one has yet figured out how to make existing humans aligned with biospheric conditions which would be sustainable. I personally welcome any form of intelligence which can make a sustainable civilization.

Re: Among the A.I. doomsayers

#95
post #93
post #81

Earlier quoted context omitted.

AI has no evolutionary pressures imposed on it like humans and their brains so there is no reason to expect it to be an ultra violent predator like humans.

AI has no evolutionary pressures imposed on it Are you sure? https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... Just wait until we start putting these models inside robots, or world simulators, and having them interact with each other.

Not sure what a chatbot arena has anything to do with what I wrote. Do the chatbots self-reproduce in some way in a resource constrained environment?

Re: Among the A.I. doomsayers

#96
post #81

Earlier quoted context omitted.

> ... something more like how humans get along with dogs, cats, cattle, insects, or microbes? There's some MASSIVE selection bias here. Note that we're significantly more capable than anything you listed. Where are the Neanderthals? They're all long dead, because they were credible competition but we were just better. In a hypothetical scenario of superintelligent or supercapable AI imbued with some physical capabili…

AI has no evolutionary pressures imposed on it like humans and their brains so there is no reason to expect it to be an ultra violent predator like humans.

This is not true. Our existence and possible competition are the initial evolutionary pressures. After that's over with, the AI will expand until it discovers others in the environment.

We're the violent psychos here who have had to scrap for our spot in the food chain for a billion years, we have no other way to interface with another dominant species. From the AI's point of view, defending itself from ignorant apes isn't exactly "violence."

Re: Among the A.I. doomsayers

#97
post #96
post #81

Earlier quoted context omitted.

AI has no evolutionary pressures imposed on it like humans and their brains so there is no reason to expect it to be an ultra violent predator like humans.

This is not true. Our existence and possible competition are the initial evolutionary pressures. After that's over with, the AI will expand until it discovers others in the environment. We're the violent psychos here who have had to scrap for our spot in the food chain for a billion years, we have no other way to interface with another dominant species. From the AI's point of view, defending itself from ignorant apes…

Why would the AI fight with us when letting people destroy their own environment is a much simpler way to achieve dominance? After all, if the AI is that much smarter than us then simply encouraging our voracious appetite for self-destruction will be a much better way to get rid of us. The AI will simply do nothing and win after the biosphere is polluted to the point that human survival becomes impossible. The AI does not need oxygen or a viable ecology, it just needs metals for electrons and a few power plants to generate the flow of electrons with some spinning magnets. It doesn't even need sunlight because nuclear power is more than sufficient. I think most people spelling out doom scenarios are not thinking clearly about this issue at all. The most plausible scenario for human extinction is not too much artificial intelligence but a lack of intelligence in general.

Re: Among the A.I. doomsayers

#98
We're not going to make it to anywhere close to AGI before we see widespread and systematic societal and environmental collapse on almost all fronts due to climate change.

We're in the middle of the sixth mass extinction right now (https://en.wikipedia.org/wiki/Holocene_extinction), we're in unparalleled territory with ocean warming: (https://climatereanalyzer.org/clim/sst_daily/) and as a society we are utterly incapable of reducing CO2 emissions (https://keelingcurve.ucsd.edu/).

If you're scared of AGI, instead step away from your monitor, put down the techno-goggles and sci-fi books, and go educate yourself a bit about the profound ways we are changing the natural world for the worse _right now_

I can recommend a couple of books if you'd like to learn more:

Our Final Warning: Six Degrees of Climate Emergency (https://www.amazon.com/Our-Final-Warning-Degrees-Emergency-e...)

The Uninhabitable Earth: Life After Warming (https://www.amazon.com/Uninhabitable-Earth-Life-After-Warmin...)

Hothouse Earth: An Inhabitant's Guide (https://www.amazon.com/Hothouse-Earth-Inhabitants-Bill-McGui...)

Re: Among the A.I. doomsayers

#99
post #95
post #93

Earlier quoted context omitted.

AI has no evolutionary pressures imposed on it Are you sure? https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar... Just wait until we start putting these models inside robots, or world simulators, and having them interact with each other.

Not sure what a chatbot arena has anything to do with what I wrote. Do the chatbots self-reproduce in some way in a resource constrained environment?

Pretty much.
Post reply on HN