Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

831–840 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#831

Earlier quoted context omitted.

I don't think I was doing either? Assume they are widely used. Does every human die?

Oh sorry! I meant that the people to whom you referred above focus on the "one true catastrophe" (whatever that is) without truly thinking what it would mean to have a few dozen nuclear weapons used. It's not just the immediate effect of the explosions. It's also the destruction of power distribution, industrial capacity, food and medicine production, and other things that millions of people depend on.

Does that lead to human extinction?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#832
post #482

Earlier quoted context omitted.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

If humans go extinct it sure as hell won't be AI that does it, but rather good old human ignorance and aggression.

Why are you so sure it can't be AI? Do you think making something smarter than humans is impossible?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#833
post #785
post #773

Earlier quoted context omitted.

If we had to have global policing of every large computing cluster and everything that runs there, that does not feel to me like a huge overreach or something that could easily slide into totalitarianism. It would be better if we didn't need to do this, but I don't see a less intrusive way. Of course, if you don't think there is a real existential threat then it's pointless and bad to do this, it all depends on that…

Intelligence is a real potential existential threat, just ask every large mammal as of about 10k years ago, save those in Africa. The question is, is an authoritarian nightmare that much better than getting tossed into the meat grinder?

> The question is, is an authoritarian nightmare that much better than getting tossed into the meat grinder?

If the choice was between those two extremes, I'd take the authoritarian nightmare.

Extinction is forever, dictatorships end.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#834
post #704

Earlier quoted context omitted.

There's also the bomb every datacenter doing AI thing that LeCun has proposed, which is a lot more practical than this. Or perhaps a self replicating work that destroys AIs.

This was why I pointed out that if I was a nation state worried about this, I'd host my AI training at the cloud providers of the country making those threats... Let them bomb themselves. And once they've then destroyed their own tech industry, you can still buy a bunch of computational resources elsewhere and just not put them in easily identifiable data centres.

We put our servers in datacenters for a real reason, not just for fun though. Good luck training an AI when your intra node communication is over the internet (this is obviously a technical problem, but the truth is we haven't solved it).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#835

Earlier quoted context omitted.

Kubrick created a memorable character and a plot twist, but the movie itself wasn't meant to be a warning about AGI/ASI. "${my favorite work of fiction} mentioned/alluded to it too!" does not make that work of fiction equivalent to a serious take on the topic in real-world context.

The gp was edited. It originally credited Mr. Yudkowsky with being the first person to imagine ai as a serious threat to humans, without mentioning that the trope existed in sci-fi, or mentioning sci-fi at all

I see, thanks for clarification.

The pre-edit text must've sounded funny, as from what I remember of Yudkowsky's writings, he frequently referenced old sci-fi tropes about or related to AI, commenting on their relevance and applicability in the real world.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#836

Earlier quoted context omitted.

The joke will be on us all when your open-source powered mechwarriors start destroying dams and power plants and even take a stab at launching some nukes, because the models figured this is the most effective way to "get them", and weren't aligned enough to see why they should take a different approach.

While my tongue was firmly in my cheek, and still is somewhat, whats to stop (for example) Boston Dynamics and OpenAI collaborating and beating us to your nightmare scenario first? And is that a situation you would prefer?

Of course not. I'm leaning towards Yudkowsky's recommendation, which is to put a pause on all of it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#837

Earlier quoted context omitted.

While my tongue was firmly in my cheek, and still is somewhat, whats to stop (for example) Boston Dynamics and OpenAI collaborating and beating us to your nightmare scenario first? And is that a situation you would prefer?

Of course not. I'm leaning towards Yudkowsky's recommendation, which is to put a pause on all of it.

I have a bridge to sell Yudkowsky and everyone else who believes that an "AI pause" is actually something that can happen.

Global treaties pretty much all occur with implied threats of violence, and we don't have enough guns to force this across India, China, the middle east, the EU, and the US. Major AI development happens in all of those places.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#838
post #712

Earlier quoted context omitted.

Not only is such an entity possible, it's already here, the only difference is clock speed. Corportations (really any large organization devoted to some kind of intellectual work, or you could say human society as a whole) can be thought of as a superintelligent entity composed of biological processors running in parallel. They that can store, process, and create information at superhuman speeds compared to the indiv…

The parent was positing, at a minimum, a 30 million fold increase in clock speed. This entails a proportional increase in energy consumption, which would likely destroy any thermal envelope the size of a brain. The only reason current processors run that fast is that there are very few of them: millions of synaptic calculations therefore have to be multiplexed into each, leading to an effective clock rate that's far…

I can think of a counterpoint or a workaround (or at least a sketch of either) for each of your objections, despite being moderately intelligent for a human. A superintelligence will think of many, many more (if it couldn't, it wouldn't be much of an intelligence in the first place).

For example:

> The parent was positing, at a minimum, a 30 million fold increase in clock speed. This entails a proportional increase in energy consumption, which would likely destroy any thermal envelope the size of a brain.

No reason to restrict the size of an artificial mind to that of a human brain. With no need to design for being part of a mobile, independently-operating body, the design space grows substantially. Even on existing hardware, you can boost the serial processing speed by a whole order of magnitude if you submerge it in liquid nitrogen. I imagine you could get much greater speeds still on hardware designed specifically for cryogenic cooling with cryo liquids at much lower temperatures. Maybe not 7 orders of magnitude difference, but 3 seem plausible with custom hardware.

Corporations are a good example of what's effectively AIs living among us today, but their performance is indeed bottlenecked by many things. A lot of them don't seem fundamental, but rather a result of the "corporate mind" growing organically on top of the runtime of bureaucracy - it's evolved software, as opposed to designed one. E.g.:

> there are many situations where the steps to design a solution are sequential and a hundred people won't get you there any faster than two

That's IMO because managers and executives, like everyone else, don't see employees and teams as a computational process, and don't try to optimize them as one. You can get a boost on sequential problems with extra parallelism, but it would look weird if done with human teams. You could absolutely do pipelining and branch prediction with teams and departments, but good luck explaining why this works to your shareholders...

The point being, while corporations themselves won't become superintelligences, due to bottlenecks you mention, those limits don't apply to AIs inhabiting proper computers. 7 orders of magnitude advantage over humans may be way too much, but I don't see anything prohibiting 2-3, maybe even 4 orders, and honestly, even 1 order of magnitude difference is unimaginably large.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#839
post #735

Earlier quoted context omitted.

We don't have an answer yet to what the form factor of AGI/ASI will be, but if it's anything like current trends the idea of 'killed' is laughable. You can be killed because your storage medium and execution medium are inseparable. Destroy the brain and you're gone, and you don't even get to copy yourself. With AGI/ASI if we can boot it up from any copy on a disk if we have the right hardware then at the end of the d…

No power and it its already dead. Destroy copies, dead. Really not complicated at all.

Ah yes, the "all you gotta do is push a button" argument.

Recall the timeless counterpoint:

https://www.youtube.com/watch?v=ld-AKg9-xpM&t=30s

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#840
post #765

Earlier quoted context omitted.

What additional information would understanding AI actually bring to the table? Worst case you just assume the AI can become human-like. Certainly all the vocal AI experts can come up with is things that humans already do to each other already, only noting that AI enables it at larger scale. Clearly there is no benefit in understanding the AI technology with respect to this matter when the only thing you can point to…

Worst case you can expect AI to become alien, becoming human like is probably one of the better outcomes. We as humans only have human intelligence to reference what intelligence is, and in doing so we commonly cut off other branches of non human intelligence in our thinking. Human intelligence has increased greatly as our sensor systems have increased their ability to gather data and 'dumb' it down to our innate sen…

No, if one can conceive of such aliens, that would be the best case. The worst case is falling back to the understanding of the human, for the reasons you describe.

However, as you point out, one does not need to have any understanding of AI to conceive of what AI can become as natural creatures can theoretically evolve into the same thing. Experts will have already been thinking about what problems might arise if alien life came emerged earth long before AI even existed. AI doesn't change anything.

Post reply on HN