Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

381–390 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#381

Post by itWontBeTatBad »

richardw wrote: post #177
brutusborn wrote: post #83

Earlier quoted context omitted.

Who do you find persuasive and what material should I read/watch to understand their POV? So far I’ve read a bunch of lesswrong posts and listened to some Eliezer talks but still can’t understand the basis of their arguments or when I do, they seem very vague. The only suggestion that makes sense to me is from the FEP crowd. Essentially if someone sets up at AI with an autopoietic mechanism then it would be able to t…

I think the two sides have such different perspectives. Some are optimistic builders and see only opportunity. Others are "safety oriented engineers" whose job and mindset is to build guarantees into systems, or secure countries from external dangers. The latter has a very hard time with the lack of guarantees with a system that is only ever going to increase in capability. Choose any limit. Any. AI will be smart but…

I agree with your counterpoint: the best artificially intelligent super agent wakes up with zero desire to eliminate humanity.

On the other hand, we will breed such systems to be cooperative and constructive.

This whole notion that AI is going to destroy the economy (or even humanity!) is ridiculous.

Even if malicious humans create malicious AI, it'll be fought by the good guys with their AI. Business as usual, except now we have talking machines!

War never changes.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#382

Post by gooseus »

homefree wrote: post #87
brutusborn wrote: post #83

Earlier quoted context omitted.

Who do you find persuasive and what material should I read/watch to understand their POV? So far I’ve read a bunch of lesswrong posts and listened to some Eliezer talks but still can’t understand the basis of their arguments or when I do, they seem very vague. The only suggestion that makes sense to me is from the FEP crowd. Essentially if someone sets up at AI with an autopoietic mechanism then it would be able to t…

There’s a decent podcast interview between Sam Harris and Eliezer Yudkowsky (on Sam’s pod), I think that’s a decent introduction and they break down the ideas in a way that’s more approachable for someone curious about it. For my personal quick summary I have earlier comments: https://news.ycombinator.com/item?id=36104090

Oh man, this podcast, I still remember walking down the street and having to take a break multiple times because I would just start exclaiming out loud (JFC!) like a lunatic listening to Eliezer talk about the AI Box Experiment as evidence of something.

If you look up the real results of this AI Box "experiment" that Eliezer claims to have won 3 of 5 times, you find that there aren't isn't any actual data or results to review because it wasn't conducted in any real experimental setting. Personally, I think the way these people talk about what a potential AGI would do reveals a lot about more about how they see the world (and humanity) than how any AGI would see it.

For my part, I think any sufficiently advanced AI (which I doubt is possibly anytime soon) would leave Earth ASAP to expand into the vast Universe of uncontested resources (a niche their non-biology is better suited to) rather than risk trying to wrestle control of Earth from the humans who are quite dug in and would almost certainly destroy everything (including the AI) rather than give up their planet.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#383

Post by vidarh »

TeMPOraL wrote: post #359
vidarh wrote: post #258

Earlier quoted context omitted.

It'd be more than a substantial regulatory burden, it's impossible. At most they'd be able to drive up the cost by forcing users to obscure it by not doing full sets of calculations at a single provider. Any attempt even remotely close to extreme enough to have any hope of being effective would require such invasive monitoring of computations run that it'd kill their entirely cloud industry. Basically, you'll only su…

Even if it's this costly, it's still possible, and much cheaper than the consequences of an unaligned AGI coming into existence.

Sure, you can insist on monitoring which software can be run on every cloud service, every managed server, on the servers in every colo, or on every computer owned by everyone (of any kind, including basic routers or phones, or even toys, - anything with a CPU that can be bought in sufficient bulk and used for computations), and subject every piece of code allowed to run at any kind of scale to a formal review by government censors.

At which point you've destroyed your economy and created a regime so oppressive it makes China seem libertarian.

Now you need to repeat that for every country in the world, including ones which are intensely hostile to you, while everyone will be witnessing your economic collapse.

It may be theoretically possible to put in place a sufficiently oppressive regime, but it is not remotely plausible.

Even if a country somehow did this to themselves, the rest of the world would rightly see such a regime as an existential threat if they tried enforcing it on the rest of the world.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#384

Post by patcon »

I need to resist all this regulatory resistance here, spoken amongst tech people.

No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant

This is the only territory where humanity is mounting a conversation about a REAL response that is appropriately cautious. This x-risk convo (again, appropriately CAUTIOUS) and our rapid response to Covid are the only things that makes me hopeful that humanity is even capable of not obliterating itself.

And I'd say the same thing EVEN IF this AI x-risk thing could later be 100% proven to be UNFOUNDED.

Humanity has so very little demonstrated skills of pumping the brakes as a collective, and even a simple exercise of "can we do this" is valuable. This is sorely needed, and I'm glad for it

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#385

Post by lucia-wermer »

koheripbal wrote: post #380
lucia-wermer wrote: post #371

[flagged]

I can't tell if this comment is just naive or sarcastic antisemitism. Can you elaborate?

???? My admiration and praise for Jewish scientists and businessmen in the field of AI is sincere.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#386

Post by TeMPOraL »

api wrote: post #349
jacquesm wrote: post #339

Earlier quoted context omitted.

There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

> The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups.

You're forgetting about governments and militaries of major powers. It's not that they want to burn the world for no reason - but they still end up seeking capability to do doomsday levels of harm, by continuously seeking to have marginally more capability than their rivals, who in turn do the same.

Or put another way: please look into all the insane ideas the US was deploying, developing, or researching at the peak of Cold War. Plenty of those hit doomsday level, and we only avoided them seeing the light of day because USSR collapsed before they could, ending the cold war and taking both motivation and funding from all those projects.

Looking at that, one can't possibly believe the words you wrote above.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#387

Post by RandomLensman »

TeMPOraL wrote: post #367
RandomLensman wrote: post #140

Earlier quoted context omitted.

At least nuclear weapons pose an actual existential risk as opposed to AI - and even then we don't just go to war.

Nukes don't have a mind of their own. They're operated by people, who fortunately turned out sane enough that they can successfully threaten each other into a stable state (MAD doctrine). Still, adding more groups to the mix increases risk, which is why non-proliferation treaties are a thing, and are taken seriously. Powerful enough AI creates whole new classes of risks, but it also magnifies all the current ones. E.…

AI could pose those risks, but it also could not - that is the difference to nuclear weapons.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#388

Post by dotnet00 »

jeffparsons wrote: post #41
dotnet00 wrote: post #36

Earlier quoted context omitted.

Or it would create an incentive for reducing the cost of the hardware needed to run beefier AI models and passing that reduction down to consumers.

Which would still mean that those who can afford more/better hardware can therefore afford more/better AI. I don't think a solution lies down this path...

Is that really an issue? Eg it's the same with games, the better your hardware, the more likely you can play a new game at higher settings with better performance, with most cheaper devices outright unable to handle anything recent.

But if you wait a GPU generation or two, the performance has improved enough that you can do that with a much cheaper GPU.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#389

Post by ben_w »

sgt101 wrote: post #314
ben_w wrote: post #283

Earlier quoted context omitted.

Only in exactly the same way that the police have the same agenda as the mafia. International bans need treaties, with agreed standards for intervention in order to not escalate to war. The argument is that if you're not willing to go all the way with enforcement, then you were never serious in the first place. Saying you won't go that far even if necessary to enforce the treaty is analogous to "sure murder is illega…

I don't agree. The police enforce civil order by consent expressed through democracy. There is no analogy in international affairs. Who is it that is going to get bombed? I am thinking it will not be the NSA data centre's in Utah, or any data centres owned by nuclear armed states.

All policing is to a certain degree by consent, even in dictatorships which only use democratic as a fig-leaf. International law likewise.

Just as the criminal justice system is a deterrent against crime despite imperfections, so are treaties, international courts, sanctions, and warfare.

> I am thinking it will not be the NSA data centre's in Utah, or any data centres owned by nuclear armed states.

For now, this would indeed seem unlikely. But so did the fall of the British Empire and later the Soviet Union before they happened.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#390

Post by TeMPOraL »

RandomLensman wrote: post #317
concordDance wrote: post #304

Earlier quoted context omitted.

To me it seems that the odds of us making something(s) "smarter" (better planning and modelling) than a human in the next 50 years is a coin toss. The odds of us being able to ensure its desires/goals align with what we would like are probably around 30%. And the odds of a "smarter than human" things ending up conquering or killing the humans if they desire to do so being around 90%. This naturally gives a probably o…

I don't even think we need alignment, just containment. The probabilities are really just too shaky for me to estimate. Not sure I would put a high probability of intelligent thing being dangerous by itself, for example.

Since we're in Yudkowsky subthread, I'll remind here that he spent some 20 odd years explaining why containment won't work, period. You can't both contain and put to work something that's smarter than you.

Another reason containment won't work is, we now know we won't even try it. Look at what happened with LLMs. We've seen the same people muse at the possibility of it showing sparks of sentience, think about the danger of X-risk, and then rush to give it full Internet access and ability to autonomously execute code on networked machines, racing to figure out ways to loop it on itself or otherwise bootstrap an intelligent autonomous agent.

Seriously, forget about containment working. If someone makes an AGI and somehow manages to box it, someone else will unbox it for shits and giggles.

Post reply on HN