Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Great A.I. Awakening

nytimes.com

Post reply on HN
Print view View original

111–120 of 147 posts

Re: The Great A.I. Awakening

#111

Post by nopinsight »

Retric wrote: post #78
nopinsight wrote: post #21

Earlier quoted context omitted.

This is why I doubt if efforts like OpenAI will ultimately turn out to be beneficial or catastrophic. I understand their purpose of democratizing the power and benefits of AI technology broadly. However, if their platform can provide a critical component or developmental path to AGI in the future, the unrestrained distribution of the technology is similar to spreading the blueprint of nuclear warheads. It could resul…

Repeat after me, "AI is not Magic." A super intelligent AI does not change the laws of physics or even get to predict the weather two weeks from now. Sure, they could learn to be a good hacker but the worlds best hacker can't get into air gaped systems. More importantly, flawless security trumps hacking. If there is nothing to exploit there is no way in, which means a lone AI is much more powerful than lot's of AI's…

The power of an AGI goes far beyond intelligence. It could generate millions of its own instances (possibly with variations) and hack on thousands of systems at the same time, actively adapting each one as needed. It could perform social engineering on a large number of people simultaneously and communicate in ultra-high bandwidth and protocols that most humans involved cannot follow.

A likely potent form of gaining power may involve assuming the identities of people one knows, via text messaging or email, and use that to gain access to important information. The aggregation and superhuman-level analysis of private information thousands of times faster than humans would bring enormous financial and social power to the AGI. Given the sad state of security in most systems, and the relative ignorance of information security issues among many political and business leaders, an AGI will not have much problem with this sample plan off the top of my head.

And there are many other more clever plans an AGI could come up with that we have not yet considered...

Re: The Great A.I. Awakening

#112

Post by c22 »

computerphage wrote: post #104
Retric wrote: post #78

Earlier quoted context omitted.

Repeat after me, "AI is not Magic." A super intelligent AI does not change the laws of physics or even get to predict the weather two weeks from now. Sure, they could learn to be a good hacker but the worlds best hacker can't get into air gaped systems. More importantly, flawless security trumps hacking. If there is nothing to exploit there is no way in, which means a lone AI is much more powerful than lot's of AI's…

Airgapped systems have been attacked in the past. Stuxnet is a good example. No magic required.

Just goes to show that flawless hacking trumps security.

Re: The Great A.I. Awakening

#113

Post by tossedaway334 »

visarga wrote: post #85
YeGoblynQueenne wrote: post #61

Earlier quoted context omitted.

>> the unrestrained distribution of the technology is similar to spreading the blueprint of nuclear warheads. There's an episode of Pinky and the Brain where Brain invents a magnification ray and gigantises Pinky and himself - so they're now giant rats [1]. They walk about some human metropolis, looming larger than skyscrapers and terrorising everyone. To cut the story short, Pinky happens and eventually the gun shoo…

We already have 7 billion of these H(uman)GI's walking around the world, with guns and dangerous tech. If the AGI will be much smarter, we shouldn't fear it the same we fear a dictator with nukes.

"Smarter" has no nessisary or consequent intersection with "Shares human values" or even "Values human existence"

https://wiki.lesswrong.com/wiki/Paperclip_maximizer

One of the reasons AGI will be so dangerous is the near certainty it will have utterly alien values and intentions.

It seems exceedingly unlikely that we will be able to program in what consitutes "human values". We can't even begin to remotely agree and explain them fully to fellow humans, let alone what is basicly an alien intelligence.

It is slightly ironic that people bring up Assimov's laws of robotics in this context, because pretty much every story he wrote was about how the laws could be subverted via "loopholing" of sufficently intelligent AI.

If something is very much smarter than humans, we cannot possibly hope to constrain it via some kind of faustian bargin agreement. Because in the end, the devil is always smarter than you are.

If an AI kills you from either malice or indifference, or just that you are composed of plenty of paperclip-capable raw materials, you are dead all the same.

Re: The Great A.I. Awakening

#114

Post by azinman2 »

It'd be really great if people could stop using the term AI by itself when they mean weak AI, or really machine learning. Its ultimately very misleading -- the only "awakening" that's happened is neural nets are popular again and getting good results. If strong AI gets solved... well then that's really when the machine will have awakened!

Re: The Great A.I. Awakening

#115

Post by Retra »

chadgeidel wrote: post #87
Retric wrote: post #78

Earlier quoted context omitted.

Repeat after me, "AI is not Magic." A super intelligent AI does not change the laws of physics or even get to predict the weather two weeks from now. Sure, they could learn to be a good hacker but the worlds best hacker can't get into air gaped systems. More importantly, flawless security trumps hacking. If there is nothing to exploit there is no way in, which means a lone AI is much more powerful than lot's of AI's…

I'll admit this is purely hypothetical , but given that we know many people act on stuff they read on the Internet in an uncritical fashion - we cannot dismiss AI because it "doesn't have access to air gapped systems". A human has access to that system - a superintelligent AI could convince that human to "push the buttons". I don't actually believe this will happen, but I'm not 100% confident it wouldn't. I don't kno…

Then your problem is humans, not AIs. And "pushing a button" is a severe understatement of what an AI would need to have a human do to cause the implied level of damage.

Re: The Great A.I. Awakening

#116

Post by c22 »

zigzigzag wrote: post #43
Fricken wrote: post #15

Earlier quoted context omitted.

You're an iteration of an AGI system that has been improving itself for hundreds of millions of years. The rate at which biological AGIs improve over time is very slow, but it's not like nature has any good reason to be in a hurry. But interesting things happen when you network billions of biological AGIs together, it leads to all sorts of emergent phenomenon, and now the biological AGIs are working on these newfangl…

You're assuming electronic AGI would inherently be superior to biological. Yes machines are "fast" in some senses but are still much less parallel than the brain. Brains can learn something based on only a few examples, even our best deep learning and ML algorithms today require vastly, vastly more data to train on than humans do.

I think the "faster iterations" would come from improved malleability, not necessarily faster "thought".

Re: The Great A.I. Awakening

#117

Post by [deleted] »

Retra wrote: post #115
chadgeidel wrote: post #87

Earlier quoted context omitted.

I'll admit this is purely hypothetical , but given that we know many people act on stuff they read on the Internet in an uncritical fashion - we cannot dismiss AI because it "doesn't have access to air gapped systems". A human has access to that system - a superintelligent AI could convince that human to "push the buttons". I don't actually believe this will happen, but I'm not 100% confident it wouldn't. I don't kno…

Then your problem is humans, not AIs. And "pushing a button" is a severe understatement of what an AI would need to have a human do to cause the implied level of damage.

[deleted]

Re: The Great A.I. Awakening

#118

Post by c22 »

empath75 wrote: post #99
ClassyJacket wrote: post #11

Earlier quoted context omitted.

>By the time AGI is developed, it would be able to improve itself/build improvements, I see no reason that's necessarily true. I'm a general intelligence, and I can't make myself smarter.

We have genetic engineering, and we're general intelligences, but we haven't figured out how to build better humans yet.

But we are also in the position of having to reverse-engineer millions of years of spaghetti code. An artificial general intelligence may just be able to consult its own datasheet and design documentation.
Post reply on HN