Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

661–670 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#661

Post by pixl97 »

tomp wrote: post #585
jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

You're arguing against your previous point. I contemplated the same before. How can I cause maximum panic (not even death!) that would result in economic damages and/or anti-freedom policy changes, for least amount of money/resources/risk of getting caught. Yet here we are, peaceful and law-abiding citizens, building instead of destroying. The ultimate truth is, if you don't like "The System", destroying it won't mak…

The vast majority of people don't want to destroy the system, they want to replace it with their own self serving system. Of course this isn't really different than saying "most large asteroids miss the Earth". The one that sneaks up on you and whollops you can put you in a world of hurt.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#662

Post by bondarchuk »

dghlsakjg wrote: post #642
bondarchuk wrote: post #632

Earlier quoted context omitted.

> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Because he identified the potential risk of superhuman AI 20 years before almost everyone else. Sure, science fiction also identified that risk (as people here seem eager to point out), but he identified it as an actual real-world risk and has been trying to take real-world action on it for 20 years. No matter what else you think about him…

Hal 9000, the malevolent AI in 2001: a space Odyssey, was in one of the most respected films of all time three years before Yudkowsky was born. Plenty of people have been beating that drum for years.

I addressed science fiction in my comment. The crucial difference is taking it seriously and actually trying to do something tangible about slowing/stopping AI progress, which is not something Stanley Kubrick did.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#663

Post by gmd63 »

Workaccount2 wrote: post #474
patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

I fully recognize the danger AI presents, and believe that it will mostly likely end up being terrible for humanity. But thanks to my own internal analysis ability and the anonymity of the internet, I am also willing to speak candidly. And I think I speak for many people in the tech community, whether they realize it or not. So here we go: My objective judgement of the situation is heavily adulterated by my incredibl…

> how can you spend your life working to get to the garden of Eden, and then not eat the fruit? Even just a taste.

Because it's insect-brain level behavior to surrender in faith to some abstract achievement without understanding its viability or actual desirability.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#664

Post by dragon96 »

homefree wrote: post #70

I think Yann is probably wrong. He refuses to engage earnestly with the “doomer” arguments. The same type of motivated reasoning could also be attributed to himself and Meta’s financial goals - it’s not a persuasive framing. The attempts I’ve seen from him to discuss the issue that aren’t just name calling are things like saying he knows smart people and they aren’t president - or even that his cat is pretty smart an…

> He refuses to engage earnestly with the “doomer” arguments. The same type of motivated reasoning could also be attributed to himself and Meta’s financial goals - it’s not a persuasive framing.

Exactly my thoughts too.

I don't agree with the Eliezer doomsday scenario either, but it's hard to be convinced by a scientist who refuses to engage in discussion about the flagged risks and instead panders to the public's fear of fear-mongering and power-seizing.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#665

Post by civilitty »

hodgesrm wrote: post #629
cjbprime wrote: post #600

Earlier quoted context omitted.

(Neither nuclear war nor climate change are considered as this-century extinction risks by the singularity-minded side of the existential risk community, just because there would necessarily be pockets of unaffected people.)

I'm always a bit surprised at how people seem to wish away the destructiveness of nuclear weapons or the likelihood of their use. Another commenter alluded to a "failure of imagination" which seems very applicable here. You would think just watching videos of above-ground tests would cure that but apparently not. [0] [0] https://www.youtube.com/watch?v=gZuyPn0_KRE

The threat of nuclear weapons is massively overblown (pun intended) thanks to a bunch of Cold War fearmongering.

The Earth land surface area is 149 million sq km and there are only about 12,500 nuclear weapons in the world. Even if they were 10 megatons each* and were all detonated, with a severe damage blast radius of 20km (~1250 sq km), it'd cover about a tenth of the land available.

Since the vast majority are designed to maximize the explosive yield, it wouldn't cause the kind of fallout clouds that people imagine, nor would it cause nuclear winter. It'd be brutal to live through (i.e. life expectancy of animals in Chernobyl is 30% lower) but nuclear weapons simple can't cause a human extinction. Not by a longshot.

* As far as I know, no one has any operational nuclear weapons over 2 megatons and the vast majority are in the 10s and 100s of kiloton range, so my back of the napkin math is guaranteed to be 10-100x too high.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#666

Post by ben_w »

RandomLensman wrote: post #401
TeMPOraL wrote: post #390

Earlier quoted context omitted.

Since we're in Yudkowsky subthread, I'll remind here that he spent some 20 odd years explaining why containment won't work, period . You can't both contain and put to work something that's smarter than you. Another reason containment won't work is, we now know we won't even try it . Look at what happened with LLMs. We've seen the same people muse at the possibility of it showing sparks of sentience, think about the d…

I find his arguments unconvincing. Humans can and have contained human intelligence, for example (look at the Khmer Rouge, for example). Also, right now there is nothing to contain. The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false. I can "create" any hypothetical risk by using that technique.

First, how is the Khmer Rouge an example of containment, given that regime fell?

Second, even if your argument is "genocide of anyone who sounds too smart was the right approach and they just weren't trying hard enough", that only really fits into "neither alignment nor containment, just don't have the AGI at all".

Containment, for humans, would look like a prison from which there is no escape; but if this is supposed to represent an AI that you want to use to solve problems, this prison with no escape needs a high-bandwidth internet connection with the outside world… and somehow zero opportunity for anyone outside to become convinced they need to rescue the "people"[0] inside like last year: https://en.wikipedia.org/wiki/LaMDA#Sentience_claims

[0] or AI who are good at pretending to be people, distinction without a difference in this case

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#667

Post by breuleux »

hackinthebochs wrote: post #595
Chathamization wrote: post #572

Earlier quoted context omitted.

Whenever I see people jump to alignment they invariably have jumped over the _much_more questionable assumption that AGI’s will be godlike. This doesn’t even match what we observe in reality - drop a human into the middle of a jungle, and it doesn’t simply become a god just because it has an intelligence that’s orders of magnitude greater than the animals around it. In fact, most people wouldn’t even survive. Further…

Its interesting seeing the vast range of claims people confidently use to discount the dangers of AI. Individual humans are limited by biology, an AGI will not be similarly limited. Due to horizontal scaling, an AGI will perhaps be more like a million individuals all perfectly aligned towards the same goal. There's also the case that an AGI can leverage the complete sum of human knowledge, and can self-direct towards…

I would say you are also overconfident in your own statements.

> Individual humans are limited by biology, an AGI will not be similarly limited.

On the other hand, individual humans are not limited by silicon and global supply chains, nor bottlenecked by robotics. The perceived superiority of computer hardware on organic brains has never been conclusively demonstrated: it is plausible that in the areas that brains have actually been optimized for, our technology hits a wall before it reaches parity. It is also plausible that solving robotics is a significantly harder problem than intelligence, leaving AI at a disadvantage for a while.

> Due to horizontal scaling, an AGI will perhaps be more like a million individuals all perfectly aligned towards the same goal.

How would they force perfect alignment, though? In order to be effective, each of these individuals will need to work on different problems and focus on different information, which means they will start diverging. Basically, in order for an AI to force global coordination of its objective among millions of clones, it first has to solve the alignment problem. It's a difficult problem. You cannot simply assume it will have less trouble with it than we do.

> There's also the case that an AGI can leverage the complete sum of human knowledge

But it cannot leverage the information that billions of years of evolution has encoded in our genome. It is an open question whether the sum of human knowledge is of any use without that implicit basis.

> and can self-direct towards a single goal for an arbitrary amount of time

Consistent goal-directed behavior is part of the alignment problem: it requires proving the stability of your goal system under all possible sequences of inputs and AGI will not necessarily be capable of it. There is also nothing intrinsic about the notion of AGI that suggests it would be better than humans at this kind of thing.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#668

Post by ben_w »

shafyy wrote: post #467
TeMPOraL wrote: post #327

Earlier quoted context omitted.

Note that ways ASI can make us extinct includes all the ways we could do it to ourselves. One of the possible scenario is tricking humans into starting WWIII, perhaps entirely by accident. Another is that even being benignly applied to optimize economic activity in broad terms might have the AI strengthen the very failure modes that accelerate climate change. Point being, all the current global risks won't disappear…

> One of the possible scenario is tricking humans into starting WWIII This sounds more like a plot of a third-grade movie than something that could happen in reality.

Both the USA and the USSR have come close due to GOFAI early warning systems:

https://en.wikipedia.org/wiki/Thule_Site_J#RCA_operations

https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...

Post-collapse Russia also came close one time because someone lost the paperwork: https://en.wikipedia.org/wiki/Norwegian_rocket_incident

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#669

Post by bondarchuk »

throwaway9274 wrote: post #650
bondarchuk wrote: post #632

Earlier quoted context omitted.

> Every time I see Yudkowsky’s writing I wonder why people take him seriously. Because he identified the potential risk of superhuman AI 20 years before almost everyone else. Sure, science fiction also identified that risk (as people here seem eager to point out), but he identified it as an actual real-world risk and has been trying to take real-world action on it for 20 years. No matter what else you think about him…

Twenty years ago was 2003. There are dozens of examples of AI x-risk thought experiments dating back to 1960. As far as I can tell, all he did was open a forum for people to write fanfic about these earlier ideas.

I opened the lesswrong homepage through archive.org on an arbitrary date in 2010, where I found a post about raising $100000 for a grant program. Most of that went towards papers and fellowships at the singularity institute AFAICT.

https://web.archive.org/web/20100304171507/http://lesswrong....

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#670

Post by 4bpp »

ImPostingOnHN wrote: post #633
4bpp wrote: post #501

Earlier quoted context omitted.

To continue contributing to climate change takes very little: you need some guy willing to spin a stick for long enough to start a fire, or to feed and protect a flock of cows. To continue contributing to AI, you need to maintain a global multi-billion supply chain with cutting edge technology that might have a four-digit bus factor. The mechanisms that advance climate change are also grandfathered in to the point th…

to continue contributing to the harm caused by AI only requires that 1 person use an existing model running off their laptop to foment real-world violence or spread disinformation en masse on social media, or use it on a cheap swarm of weaponized/suicide drones, for a few examples

There's a qualitative barrier there. The AI risk people in the know are afraid of is not a flood of AI-generated articles, but something that probably can't be achieved yet with current levels of AI (and more slop generated from current-day AI won't hasten the advent of). On the other hand, modern-day greenhouse gases are exactly the greenhouse gases that climate activists are afraid of in the limit.
Post reply on HN