Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hawking: AI could end human race

bbc.com

Post reply on HN
Print view View original

71–80 of 118 posts

Re: Hawking: AI could end human race

#71

Post by gus_massa »

jp555 wrote: post #36
gear54rus wrote: post #24

To all those saying it is impossible: The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits. One needs to remember that AI itself may not be evil,…

I understand that a singularity by definition can never be reached. It's like the speed of light or absolute zero; infinity just out of reach no matter how much energy you spend trying to get there.

You are confusing a "mathematical singularity" with "the AI singularity".

In math, a singularity is an unusual point, not sure if it can't be reached. I think it's used in different areas to describe different things. For example, if you model water in a channel, after some time the waves accumulate and you get a vertical water profile, the depth is not more a function, the mathematical model collapse, but in real life the water just make some nice crashing effect.

Re: Hawking: AI could end human race

#72

Post by shittyanalogy »

sailingparrot wrote: post #30
mariusz79 wrote: post #2

I'm tired of this fear mongering. There are hundreds of viruses that could end the human race. There are millions of rocks in space that could end the human race. There are people that could end the human race with push of a button. Human race could be history by the time I stop writing this rant. AI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly…

Nice reasoning. We could die from many other ways, so why should we care about another one? Why should we care about global warming, I mean, it won't stop asteroids from smashing earth. Why should we care about nuclear war head proliferation if we might still die because of a virus? So let's just not give a fuck about anything!

He's saying that regardless of the hypothesized danger, until there is measurable, non sci-fi proof, it's all just fear mongering. Not that we shouldn't "give a fuck about anything."

Re: Hawking: AI could end human race

#73

Post by nadam »

shittyanalogy wrote: post #64
catshirt wrote: post #59

Earlier quoted context omitted.

pretty much all machines are already capable of killing us. they're just not smart enough to be able to want to yet. :)

computer controlled My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?

Smart programs can pay humans to do things. It can send an email to a guy who seems to be weak and desparate (maybe drug addicted) based on their forum contribution. The software can send them money on paypal for the serivces of the guy. The guy's task would be to buy a factory, install this and this software on the factory's computer's, etc... In the mean time the software creates a SaS internet firm (or a software development shop) to earn hundreds of millions of dollars. Intelligence can be the most efficient weapon one can imagine. It can pay, manipulate, divide and conquer humans until humans are needed...

Re: Hawking: AI could end human race

#75

Post by rprospero »

chuckup wrote: post #37

I've always heard the line, Aliens would never travel many light years to destroy/enslave humanity, because we're no threat, and they'd be enlightened enough not to. But the idea that we could stumble upon AI, and it could become hyper intelligent at an astronomical rate ("the singularity") - doesn't this make us a huge potential threat to non-AI life everywhere? We'd have to be quarantined. For our own good, and the…

In the hypothetical, the aliens haven't been wiped out yet by the singularity. As I see it, that leaves two options.

1) The aliens never stumbled upon AI. Considering that they've mastered interstellar travel, then they're likely very far ahead of us technologically. If they haven't stumbled across it, it's pretty unlikely that we will either. It's like quarantining otters because we're worried that they'll accidentally make an anti-matter bomb.

2) The aliens developed the singularity, but defeated it. Perhaps the Gulgaflak theorem provides a logic bomb that stops singularities from occurring. Maybe they tamed the AI into a benevolent force. Whatever it is, they have a way of handling AIs that are, again, more advanced than what we can develop. We don't quarantine otter because our tools are WAY better than theirs.

3) The aliens figured out AI, but decided not to use it. A species wide decision was made to not develop the technology once they figured out exactly how to do it. Considering the short term, local benefits to someone for cheating and doing it anyway, it's essentially the prisoner's dilemma, multiplied by everyone on the entire planet times all the centuries between their discovery of AI and their discovery of life on Earth.

Re: Hawking: AI could end human race

#76

Post by Thimothy »

rbdn wrote: post #53
gear54rus wrote: post #24

To all those saying it is impossible: The point you are making about it being impossible is as moot as the one those that fear AI are making. From logical point of view, singularity (a moment when AI has modified itself to the point beyond our capacity of control or even understanding) seems possible. I don't see a way for us to tell what will happen when it hits. One needs to remember that AI itself may not be evil,…

The message should be: We need more funding and promotion of AI safety research.

He, this remembered me the Portal 2 AI safety tags:

http://io9.com/5897036/the-rise-of-the-machines-as-told-by-w...

Re: Hawking: AI could end human race

#77

Post by shittyanalogy »

nadam wrote: post #73
shittyanalogy wrote: post #64

Earlier quoted context omitted.

computer controlled My hand drill is not capable of killing me with the right software nor is it network capable. And once we know that the larger machines have bad software then what? We keep using them? What's my TV going to do, short circuit the video port?

Smart programs can pay humans to do things. It can send an email to a guy who seems to be weak and desparate (maybe drug addicted) based on their forum contribution. The software can send them money on paypal for the serivces of the guy. The guy's task would be to buy a factory, install this and this software on the factory's computer's, etc... In the mean time the software creates a SaS internet firm (or a software…

Uh huh, and once we realize the networks are compromised in this way we're just going to let it happen. Forever until we're all dead. And the machines are going to trust the dude with enough money to buy a factory. And the dude is just going to buy a factory and infect it for the machines. And then stuff like that is going to happen until we're all dead.

Re: Hawking: AI could end human race

#78

Post by sailingparrot »

shittyanalogy wrote: post #72
sailingparrot wrote: post #30

Earlier quoted context omitted.

Nice reasoning. We could die from many other ways, so why should we care about another one? Why should we care about global warming, I mean, it won't stop asteroids from smashing earth. Why should we care about nuclear war head proliferation if we might still die because of a virus? So let's just not give a fuck about anything!

He's saying that regardless of the hypothesized danger, until there is measurable, non sci-fi proof, it's all just fear mongering. Not that we shouldn't "give a fuck about anything."

You cannot have non sci-fi proof of future events, it will always be unmeasurable theories.

So you can choose to wait and see, and maybe it will be too late or to hard to act when you have sufficient proof.

Or you can choose to start thinking about it now, and see what we can do to to set up safeguards. And perhaps those safeguards will be useless, perhaps not. That's called prevention.

Re: Hawking: AI could end human race

#79

Post by sailingparrot »

shittyanalogy wrote: post #70
airmondii wrote: post #26

Earlier quoted context omitted.

Fear mongering? This isn't Sarah Palin and her death panels. I'm sure Hawking and Musk are smarter and more knowledgeable on the subject than either of us, so maybe it's worth listening to them rather than dismissing out of hand. I don't see what they would have to personally gain by raising the alarm here. Read Superintelligence by Bostrom to help see where they're coming from. It doesn't think AI will be evil eithe…

I'm sure Hawking and Musk are smarter and more knowledgeable on the subject than either of us... That is not a reasonable argument. They have demonstrated nothing but a fear and postulation and we cannot accept authority as evidence. Thus is the nature of science.

Fear is a demonstration of non competency? So there can never be harmful technologies?

If someone more knowledgeable than me on a subject fears it, well I think it's worth at least listening to him. Not dismissing his concerns arrogantly.

And when those person are one of the greatest scientist and one of the greatest entrepreneur of our time, both with some kind of expertise in AI, well...

Re: Hawking: AI could end human race

#80

Post by tomp »

Tloewald wrote: post #55
mariusz79 wrote: post #2

I'm tired of this fear mongering. There are hundreds of viruses that could end the human race. There are millions of rocks in space that could end the human race. There are people that could end the human race with push of a button. Human race could be history by the time I stop writing this rant. AI has a great potential and could help us prevent most of our existential threats. AI is not a god. It will not suddenly…

This isn't fear mongering, it's simply a question of due diligence. AI is a big unknown (unlike viruses, comets, nuclear weapons, incidentally, all of which we have at least some grasp of -- ever seen an AI?)

AI is also, at present (and in the foreseeable future) science fiction.
Post reply on HN