Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

741–750 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#741

Post by degrews »

upupupandaway wrote: post #182
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Obviously no one is talking about "stopping matrix multiplication globally". But setting limits on the supply of and access to hardware, in a way that would meaningfully impede AI development, does seem feasible if we were serious about it.

Also, Eliezer is not claiming this will definitely work. He thinks it ultimately probably won't. The claim is just that this is the best option we have available.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#742

Post by pixl97 »

lewhoo wrote: post #295
dnissley wrote: post #291

Earlier quoted context omitted.

The best argument against human cloning/gmo is simply that we can't do this well on other animal species just yet, so probably we should be hold off on the humans. It's hyper pragmatic from an ethics perspective, not a trolley problem or stack of hypotheses at all.

I'm no expert but I actually think we are pretty good at genetically modifying organisms and that manifests itself in crops for instance with enough research to support the benefits.

I think much of this is, because Monsanto aside, no one has had a nice big genetic accident or intentional 'oh no' yet. Yea, we've had issues but not at the people are starving.

If we start thinking of plant based biological weapons, just imagine modifying an existing grass to produce tons of pollen and to cross pollinate wheat but not produce seed in the wheat.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#743

Post by pixl97 »

veidr wrote: post #510
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

I don't want to be rude; I understand where you are coming from (at least in terms of single-minded risk mitigation from a theoretical and unconstrained viewpoint), but this reads like Mike Huckabee's 'just keep an aspirin pill pinned between your knees' advice to teenage girls on how not to get pregnant. There is obviously (!!!!!!!!!) no way in this — or any imaginable — timeline that we are going to just "stop". (A…

Butlerian Jihad level revolt by the average person?

No reason to see that happening these days, and by the time we're willing to it may already be too late.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#744

Post by librexpr »

bnralt wrote: post #561
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Every time I see Yudkowsky’s writing I wonder why people take him seriously. Just a couple of months back he was duped by a fake story about an airforce AI deciding to kill it’s operators in a simulation. It turned out the whole thing was thought experiment. Asking Yudkowsky what we should do about AI is like asking Green Peace what we should do about nuclear power.

Yudkowsky was not fooled. He made three tweets on this subject in that time frame, which can all be seen using this link:

https://nitter.net/ESYudkowsky/search?f=tweets&since=2023-05...

First tweet:

> ...can we get confirmation on this being real?

https://nitter.net/ESYudkowsky/status/1664313290317795330#m

Second tweet, which is a reply:

> Good they tested in sim, bad they didn't see it coming given how much the entire alignment field plus the previous fifty years of SF were warning in advance about this exact case

https://nitter.net/ESYudkowsky/status/1664357633762140160#m

Third tweet:

> Disconfirmed.

https://nitter.net/ESYudkowsky/status/1664639807002214401#m

The second tweet does not explicitly say "conditional on this turning out to be real", but given that the immediately preceding tweet was expressing doubt, it is implicit from the context that that is what he meant.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#745

Post by jcims »

jacquesm wrote: post #499
jcims wrote: post #455

Earlier quoted context omitted.

>The stories from that night continue to haunt me today. These conversations are oddly precious and intimate. It's so difficult to find someone that is willing to even 'go there' with you let alone someone that is capable of creatively and fearlessly advancing the conversation.

Let's just say I'm happy they're my friends and not my enemies :) It's pretty sobering to realize how intellect applied to bad stuff can lead to advancing the 'state of the art' relatively quickly once you drop the usual constraints of ethics and morality. To make a Nazi parallel: someone had to design the gas chambers, someone had to convince themselves that this was all ok and then go home to their wives and kids t…

My favorite part of those conversations is when you decide to stop googling hahaha.

>Technology acts as a force multiplier though

It really does, and to a point you made elsewhere infrastructure is effectively a multiplication of technology, so you wind up with ways to compound the asymmetric effect in powerful ways.

>Without technology that kind of reach would be impossible.

I worked for a bug bounty for a while and this was one of my takeaways. You have young kids with meager resources in challenging environments making meaningful contributions to the security of a Silicon Valley juggernaut.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#746

Post by pixl97 »

candiodari wrote: post #357
dnissley wrote: post #299

Earlier quoted context omitted.

People seem to have very small imaginations when it comes to thinking about how they might circumvent the restrictions they are proposing. I don't know of it's just wishful thinking, desperation, or what. For example if OpenAI et al were reduced to using consumer gpus to run their training loads it would increase costs, but not by an order of magnitude. It would just be one more hurdle to overcome. And there are stil…

As someone who works in medical research, I'm pretty sure it's possible to create a GM bacterial version of poppy's, just like we've got that for insulin, that would end the drug wars, by enabling hundreds of grams of the stuff to be produced in a single bottle that can be kept virtually anywhere (it has to be kept heated, and shaken from time to time). Such a bottle's contents can be trivially duplicated and the who…

The drug wars are not because we can't produce enough drugs. I'm pretty sure we can grow enough poppies to kill all of humanity a few times over and at an insanely cheap price.

Carfentanil is a decent example of this. It's insanely potent and even a small amount smuggled in goes a long way.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#748

Post by RandomLensman »

cma wrote: post #736
RandomLensman wrote: post #619

Earlier quoted context omitted.

I don't agree with the uniqueness of AI risk. A large asteroid impacting earth is a non-hypothetical existential risk presently out of our control. We do not currently plan on spending trillions to have a comprehensive early detection system and equipment to control that risk. The differentiating thing here is that blocking hypothetical AI risk is cheap, while mitigating real risks is expensive.

We have a decent grasp of the odds of an asteroid extinction event in a way that we don't for AI.

And that makes taking that extinction risk everyday better?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#749

Post by pixl97 »

RandomLensman wrote: post #397
TeMPOraL wrote: post #395

Earlier quoted context omitted.

AI does pose all those risks, unless we solve alignment first. Which is the whole problem.

I does not, because that kind of AI doesn't exist at the moment and the malevolence etc. are bunch of hypotheses about a hypothetical AI, not facts.

You keep repeating this tired argument in this thread, so just subtract the artificial element from it.

Instead imagine a non-human intelligence. Maybe its alien carbon organic. Maybe it's silicon based life. Maybe it's based on electrons and circuits.

In this situation, what are the rules of intelligence outside of the container it executes in?

Also, every military in the world wargames on hypotheticals because making your damned war plan after the enemy attacks is a great way to wear your enemies flag.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#750

Post by RandomLensman »

pixl97 wrote: post #728
RandomLensman wrote: post #574

Earlier quoted context omitted.

Your existentially risky AI is imaginary, it might not exist. Who would check an imaginary gun on a table?

Lets go back in time instead... it's 1400 and you're a native American. I am a fortune teller and I say people in boats bearing shiny metal sticks will eradicate us soon. To the natives that gun would have also been imaginary if you would have brought up the situation to them. I would say that it would believable if they thought the worst possible outcome they could ever have was another tribe of close to the same ca…

What we also have is viruses and bacteria killing people irrespective of those having zero intelligence. We also have smart people being killed by dumb people. And people with sticks killing people with guns. My point is, these stories don't mean anything in relation to AI.

Btw., the conquistadors relied heavily on exploiting local politics and locals for conquest, it wasn't just some "magic tech" thing, but old fashioned coalition building with enemies of enemies.

Post reply on HN