Live data from Hacker News

AI Could Defeat All of Us Combined

cold-takes.com

101–110 of 114 posts

Re: AI Could Defeat All of Us Combined

#101

What a limited imagination of how an AI would take over. The scenarios seem to be centered around an extrapolation of "What if a really smart human were trapped inside a computer?" The amount of anthropomorphism is astounding. The author misses an even scarier prospect - people will want to run such an AI. They will be absolutely giddy at the prospect of running such an AI and it won't be anything like a really smart…

> What a limited imagination of how an AI would take over.

I think you've missed the point of the article. It's not intended to be a picture of how AIs would actually take over, but rather a lower bound -- a demonstration that AIs could take over even if they are restricted in this way.

Re: AI Could Defeat All of Us Combined

#102

Why could you not just unplug it? Any AI is going to need to interface with the external world. It's probably going to be quite large as well and require lots of power + networking. That leaves it quite vulnerable to having its "supply"-lines cut. AI-doomsayers should spend more time engaging with military history and theory.

It could divide itself into smaller pieces and hack million of devices/iot worldwide and live on.

Re: AI Could Defeat All of Us Combined

#103
The funny thing is that we think of each human as a general intelligence, but really, the only interesting intelligence is humanity as a collective. An isolated human will never learn speech, reading, writing, or math. We are just neurons in a swarm that managed to figure out how to rapidly train new neurons as they are produced. On each generation, the swarm learns more about the universe and expands into it.

I think computer AI makes as much sense as all our books and tweets and talking producing an aggregate intelligence "one level" above us. If our consciousness is formed from countless neurons propagating signals, for all we know, all our propagating signals to each other will form a consciousness above us. One that can't communicate with us any more than we can with our neurons. One that can't read or write English any more than we can read neuron propagation signals. For all we know, maybe our society is already conscious the same way we are, it just thinks slower.

Re: AI Could Defeat All of Us Combined

#104

Earlier quoted context omitted.

Why does the latest IPCC climate report scare you so much? While we're still not on a great path the worst case scenario has gotten better.

It’s been a while since I looked at IPCC report. Is the worst case still apocalyptically bad? We’re nearing 1.5 degrees above average and the results are bad enough already. (Not looking forward to another summer of fires like Australia had a couple of years ago.)

We're still not only increasing emissions, but accelerating the increasing emissions. I don't think we'll avoid a rise of 4 degrees over the average.

On the bright side we may have killed the cycles of glaciation that would have been awkward in the future, to put it mildly. The cost is going to be so heavy though.

Re: AI Could Defeat All of Us Combined

#105
post #102

Why could you not just unplug it? Any AI is going to need to interface with the external world. It's probably going to be quite large as well and require lots of power + networking. That leaves it quite vulnerable to having its "supply"-lines cut. AI-doomsayers should spend more time engaging with military history and theory.

It could divide itself into smaller pieces and hack million of devices/iot worldwide and live on.

Why would it continue to function if divided up? The latency between components has now increased dramatically and you've also introduced new attack surfaces (those devices get their power from some electricity substation which can be shut down... same for their wireless networking).

Re: AI Could Defeat All of Us Combined

#106

If there is such thing as General Intelligence, we don't know what it is. And I believe that it could well be an empty abstraction, an idea, not unlike the idea of God. What we call Human Intelligence is an aggregate of many skills, built on top of almost hardwired foundations, which is the product of natural evolution over millions of years. Our kind of intelligence seems only general to us, because we all share the…

> And I believe that it could well be an empty abstraction, an idea, not unlike the idea of God.

Huh? God is not an "empty abstraction" nor an "idea".

Re: AI Could Defeat All of Us Combined

#107

What a limited imagination of how an AI would take over. The scenarios seem to be centered around an extrapolation of "What if a really smart human were trapped inside a computer?" The amount of anthropomorphism is astounding. The author misses an even scarier prospect - people will want to run such an AI. They will be absolutely giddy at the prospect of running such an AI and it won't be anything like a really smart…

> What a limited imagination of how an AI would take over. I think you've missed the point of the article. It's not intended to be a picture of how AIs would actually take over, but rather a lower bound -- a demonstration that AIs could take over even if they are restricted in this way.

Right, and my point is that this framing is itself dangerously naive.

Re: AI Could Defeat All of Us Combined

#108

Earlier quoted context omitted.

> What a limited imagination of how an AI would take over. I think you've missed the point of the article. It's not intended to be a picture of how AIs would actually take over, but rather a lower bound -- a demonstration that AIs could take over even if they are restricted in this way.

Right, and my point is that this framing is itself dangerously naive.

Again, if you are saying "actually it would be much worse than that", then you aren't disagreeing with the article, you are expanding on it, because the article is deliberately only saying "it would be at least this bad". In no place does it make any claim about an upper bound, so you are not contradicting it. It's confusing to the reader when you frame expanding on someone else's point as if you were instead disagreeing with it.

Perhaps you are thinking that a person who doesn't read carefully would get the mistaken impression that the article is claiming "things would be about this bad", instead of the intended "things would be at least this bad (which likely means far worse)"? If so, then you should make that explicit, because then you are once again not contradicting the article, but rather a sloppy thinker's mistaken impression of it.

(If I write, for instance, "The dinosaurs lived over a million years ago", the correct response is not, "That is wrong, they lived over 65 million years ago", it is "In fact, more than that is true; they lived over 65 million years ago". The difference is important for not wasting time on imaginary disagreements!)

Stating a lower bound does not in any way imply that one believes that lower bound to be anywhere near tight!

Especially I don't know why you've inferred that Karnofsky cannot imagine worse scenarios. Given the circles he's in, I would be very surprised if he cannot. He's not including worse scenarios because they're not relevant to the point he's making, which is that this is a lower bound. In fact, we can say more than that -- he's not including worse scenarios because he wants to make the argument as airtight as possible for the skeptical. He wants to use arguments that are as strong as possible to establish a lower bound, not establish a position that is as strong as possible, because that would require using arguments that many would find less acceptable.

(This by the way is an illustration by the way of the more general principle of why trying to talk about the author, rather than the article, is a mistake. It's a lot easier to make mistaken inferences about authors! Stick to discussing the actual ideas and you don't have to worry about those mistakes.)

Re: AI Could Defeat All of Us Combined

#109

Earlier quoted context omitted.

Right, and my point is that this framing is itself dangerously naive.

Again, if you are saying "actually it would be much worse than that", then you aren't disagreeing with the article, you are expanding on it, because the article is deliberately only saying "it would be at least this bad". In no place does it make any claim about an upper bound, so you are not contradicting it. It's confusing to the reader when you frame expanding on someone else's point as if you were instead disagre…

[deleted]

Re: AI Could Defeat All of Us Combined

#110
post #2

Another idea too dangerous to leave unchecked, like Nuclear weapons or Biological warfare. I think most people will agree that a GAI can't be bargained with, tempted, bought or otherwise contained - We will be at its complete mercy regardless of any constraints we might think up. What I would like to discuss, is how we can get humanity to a point where we can responsibly wield weapons that powerful without risking th…

Maybe the answer is distributed and redundant human civilizations? So even if one blows itself up, others survive.

Great idea, but it will require one planet per partition
Post reply on HN