Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

671–680 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#671

Post by daveguy »

bondarchuk wrote: post #632
bnralt wrote: post #561

Earlier quoted context omitted.

Every time I see Yudkowsky’s writing I wonder why people take him seriously. Just a couple of months back he was duped by a fake story about an airforce AI deciding to kill it’s operators in a simulation. It turned out the whole thing was thought experiment. Asking Yudkowsky what we should do about AI is like asking Green Peace what we should do about nuclear power.

> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Because he identified the potential risk of superhuman AI 20 years before almost everyone else. Sure, science fiction also identified that risk (as people here seem eager to point out), but he identified it as an actual real-world risk and has been trying to take real-world action on it for 20 years. No matter what else you think about him…

A poorly premised fantasy fiction horror story about what might happen during the interaction between a super intelligence and its creator makes for a great story, but it's not a blueprint for the future, much less a pressing concern. Misuse of current AI is far more concerning than anything dreamt up about AGI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#672

Post by breuleux »

JoshTriplett wrote: post #523
RandomLensman wrote: post #401

Earlier quoted context omitted.

I find his arguments unconvincing. Humans can and have contained human intelligence, for example (look at the Khmer Rouge, for example). Also, right now there is nothing to contain. The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false. I can "create" any hypothetical risk by using that technique.

> Humans can and have contained human intelligence ASI is not human intelligence. As a lower bound on what "superintelligence" means, consider something that 1) thinks much faster, years or centuries every second, and 2) thinks in parallel, as though millions of people are thinking centuries every second. That's not even accounting for getting qualitatively better at thinking , such as learning to reliably make the n…

> As a lower bound on what "superintelligence" means, consider something that 1) thinks much faster, years or centuries every second, and 2) thinks in parallel, as though millions of people are thinking centuries every second.

I'm fairly certain that what you describe is physically impossible. Organic brains may not be fully optimal, but they are not that many orders of magnitude off.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#673

Post by BobaFloutist »

Terretta wrote: post #653
tensor wrote: post #482

Earlier quoted context omitted.

If humans go extinct it sure as hell won't be AI that does it, but rather good old human ignorance and aggression.

> If humans go extinct it sure as hell won't be AI that does it, but rather good old human ignorance and aggression. And the shift to kakistocracy . https://en.wikipedia.org/wiki/Kakistocracy

Haha poop government.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#674

Post by philipkglass »

zarzavat wrote: post #374
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack Chemical weapons are absolutely terrifying , especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings). If AI makes this stuff easier to carry out then we are completely fucked…

I take the opposite lesson from that incident: attempting exotic attacks with chemical weapons is very expensive and not very effective. The UN estimated that the lab where the cult made chemical weapons had a value of 30 million dollars, and with that investment they killed 22 people (including 13 in the subway attack). A crazed individual can kill that many people in a single attack with a truck or a gun. There are numerous examples from the past decade.

It doesn't matter much if the AI can give perfect "explain like I'm 5" instructions for making VX. The people who carry out those instructions are still risking their lives before claiming a single victim. They also need to spend a lot of money amount on acquiring laboratory equipment and chemicals that are far enough down the synthesis chain to avoid tipping off governments in advance.

The one big risk I can see, eventually, is if really capable AIs get connected to really capable robots. They would be "clanking replicators" capable of making anything at all, including VX or nuclear weapons. But that seems a long way off from where we are now. The people trumpeting X-Risk now don't think that the AIs need to be embodied to be an existential risk. I disagree with that for reasons that are too lengthy to elaborate here. But it's easy to see how robots that can make anything (including copies of themselves) would be the very sharpest of two-edged swords.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#675

Post by pixl97 »

dragonwriter wrote: post #580
lettergram wrote: post #425

Earlier quoted context omitted.

zero chance climate change (1-2 degree increase) would end humanity. 5-10 probably wouldn’t end humanity lol Nuclear war would probably kill 80-90% but even then wouldn’t kill humanity. Projections I’ve seen are only like 40-50% of the countries hit. AI is scary if they tie it into everything and people slowly stop being capable. Then something happens, and we can’t boil water. Beyond that scenario I don’t see the ri…

> zero chance climate change (1-2 degree increase) would end humanity “Humanity” is not the same thing as “human civilization”. But, yes, its unlikely that it will be extinct in a century, even more unlikely that it will be from climate change. ...and yet climate change is still more likely to do it in that time than AI.

This the the difficulty in assigning probability to something that is distinctly possible but not measured.

Your guess on what AI will do in the future is based on how AI has performed in the past. At the end of the day we have no ability to know if the function it will follow or not. But on 1900 and one second I can pretty much promise you that you would not have said that nuclear war or climate change would be your best guess on what would cause the collapse of humanity. Forward looking statements that far in the future don't work well these days.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#676

Post by danenania »

randomdata wrote: post #658
danenania wrote: post #656

Earlier quoted context omitted.

I guess my experience is different. I've heard plenty about realistic scenarios. It's out there if you look for it, or even if you just spend time thinking it through. Identity theft is far from the biggest danger even with current capabilities. Though to your point, I think part of the issue is that people who study this stuff are often hesitant to give too much detail in public because they don't want to give ideas…

> I've heard plenty about realistic scenarios. Of course. Everyone has an opinion. Some of those opinions will end up be quite realistic, even if just by random chance. You don't have to be an expert to come up with right ideas sometimes. Hinton's vision of AI being used to steal identities is quite realistic. But that doesn't make him an expert. His opinion carries no more weight than any other random hobo on the st…

Oh, I've heard plenty of discussion of positive scenarios as well, including post-scarcity.

If you just look for a list of all the current AI tools and startups that are being built, you can get a pretty good sense of the potential across almost every economic/industrial sphere. Of course, many of these won't work out, but some will and it can give you an idea of what some of the specific benefits could be in the next 5-10 years.

I'd say post-scarcity is generally a longer-term possibility unless you believe in a super-fast singularity (which I'm personally skeptical about). But many of the high risk uses are already possible or will become possible soon, so they are more front-of-mind I suppose.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#677

Post by randomdata »

danenania wrote: post #676
randomdata wrote: post #658

Earlier quoted context omitted.

> I've heard plenty about realistic scenarios. Of course. Everyone has an opinion. Some of those opinions will end up be quite realistic, even if just by random chance. You don't have to be an expert to come up with right ideas sometimes. Hinton's vision of AI being used to steal identities is quite realistic. But that doesn't make him an expert. His opinion carries no more weight than any other random hobo on the st…

Oh, I've heard plenty of discussion of positive scenarios as well, including post-scarcity. If you just look for a list of all the current AI tools and startups that are being built, you can get a pretty good sense of the potential across almost every economic/industrial sphere. Of course, many of these won't work out, but some will and it can give you an idea of what some of the specific benefits could be in the nex…

> I've heard plenty of discussion of positive scenarios as well, including post-scarcity.

Everyone has an opinion, but who are the experts (in the subject matter, not some other thing) discussing it?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#678

Post by varelse »

bondarchuk wrote: post #662
dghlsakjg wrote: post #642

Earlier quoted context omitted.

Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.

I addressed science fiction in my comment. The crucial difference is taking it seriously and actually trying to do something tangible about slowing/stopping AI progress, which is not something Stanley Kubrick did.

[dead]

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#679

Post by hackinthebochs »

breuleux wrote: post #667
hackinthebochs wrote: post #595

Earlier quoted context omitted.

Its interesting seeing the vast range of claims people confidently use to discount the dangers of AI. Individual humans are limited by biology, an AGI will not be similarly limited. Due to horizontal scaling, an AGI will perhaps be more like a million individuals all perfectly aligned towards the same goal. There's also the case that an AGI can leverage the complete sum of human knowledge, and can self-direct towards…

I would say you are also overconfident in your own statements. > Individual humans are limited by biology, an AGI will not be similarly limited. On the other hand, individual humans are not limited by silicon and global supply chains, nor bottlenecked by robotics. The perceived superiority of computer hardware on organic brains has never been conclusively demonstrated: it is plausible that in the areas that brains ha…

Yes, every point in favor of the possibility of AGI comes with an asterisk. That's not all that interesting. We need to be competent at reasoning under uncertainty, something few people seem to be capable of. When the utility of a runaway AGI is infinitely negative, while the possibility of that outcome is substantially non-zero, rationality demands we act to prevent that outcome.

>How would they force perfect alignment, though? In order to be effective, each of these individuals will need to work on different problems and focus on different information, which means they will start diverging

I disagree that independence is required for effectiveness. Independence is useful, but it also comes with an inordinate coordination cost. Lack of independence implies low coordination costs, and the features of an artificial intelligence implies the ability to maximally utilize the abilities of the sub-components. Consider the 'thousand brains' hypothesis, that human intelligence is essentially the coordination of thousands of mini-brains. It stands to reason that the more powerful the mini-brains, along with the efficiency of coordination, implies a much more capable unified intelligence. Of course all that remains to be seen.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#680

Post by RandomLensman »

ben_w wrote: post #666
RandomLensman wrote: post #401

Earlier quoted context omitted.

I find his arguments unconvincing. Humans can and have contained human intelligence, for example (look at the Khmer Rouge, for example). Also, right now there is nothing to contain. The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false. I can "create" any hypothetical risk by using that technique.

First, how is the Khmer Rouge an example of containment, given that regime fell? Second, even if your argument is "genocide of anyone who sounds too smart was the right approach and they just weren't trying hard enough ", that only really fits into "neither alignment nor containment, just don't have the AGI at all". Containment, for humans, would look like a prison from which there is no escape; but if this is suppos…

It means intelligence can be killed (and the Khmer were brought down externally).

We might not even need to contain, all hypothetical.

Post reply on HN