Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

851–860 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#851

Post by TeMPOraL »

pclmulqdq wrote: post #850
TeMPOraL wrote: post #848

Earlier quoted context omitted.

You're reading this wrong. Yudkowsky isn't saying that this will happen; in fact, it's apparent to him as much as to everyone else how unlikely is that. Yudkowsky's point is that this is the minimum and least difficult thing we need to do. LeCun and others on both sides of the "AI regulation" argument are busy arguing the color of the carpet, while the whole house is on fire.

I think we all also fundamentally disagree about whether the house is on fire at all. History is littered with examples of Luddites yelling about how the bottom of a technological S-curve is actually an exponential and how we Must Do Something to prevent a catastrophe. With possibly one exception (biological weapons, research on which seems to have few positive externalities) they have always been wrong. I did not me…

> History is littered with examples of Luddites yelling about how the bottom of a technological S-curve is actually an exponential and how we Must Do Something to prevent a catastrophe.

Can't think of any such examples, do you have some?

Of the two that come to my mind:

- The only thing the Luddites were yelling about is having their livelihoods pulled out from under them by greedy factory owners who aggressively applied automation instead of trying to soften the blow. They weren't against the technological progress; the infamous destruction of looms wasn't a protest against progress, but a protest against treatment of laboring class.

- The "Limits to Growth" crowd, I don't think they were wrong at all. Their predictions didn't materialize on schedule because we hit a couple unexpected technological miracles, the Haber–Bosch process being the most prominent one, but the underlying reasoning looks sound, and there is no reason to expect we'll keep lucking into more technological miracles.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#852

Post by dragonwriter »

TeMPOraL wrote: post #836
nineteen999 wrote: post #781

Earlier quoted context omitted.

While my tongue was firmly in my cheek, and still is somewhat, whats to stop (for example) Boston Dynamics and OpenAI collaborating and beating us to your nightmare scenario first? And is that a situation you would prefer?

Of course not. I'm leaning towards Yudkowsky's recommendation, which is to put a pause on all of it.

Lots of people recommend a pause, Yudkowsky explicitly argues that a pause is entirely inadequate and recommends an extreme anti-AI-research global crusade, so its kind of odd to point to him the way you have as a pause-recommender.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#853

Post by veidr »

ben_w wrote: post #635
veidr wrote: post #599

Earlier quoted context omitted.

My recollection is that Russian troops/mercenaries/gunmen/whatever stole washing machines and dishwashers, but it was (presumably) just for regular theft/looting/pillagine purposes. Separately, during the pandemic, various companies bought various products, including washing machines, to extract the chips within, because the chips were the supply chain bottleneck, so it made economic sense to purchase them at even 10…

That's the first I've heard of it being done during the pandemic. I would also be surprised that this would be only x10-x20 the price, rather than x100.

For sure; I mean, mine (a fairly modern, drum washer/dryer thing) cost like $2000 and the extremely basic functionality suggests the chip/SoC should be $20 or less.

But, I assume any company doing that in any kind of serious volume has some strategy for monetizing the "washing machine with chip excised" (?).

Also, my bike is VanMoof and they just went bankrupt, and it seems like "having inventory of almost-finished hipster e-bikes — EXCEPT those 1 or 2 key parts" was part of their fatal situation.

So I can imagine scenarios where you buy washing machines to extract chips to lose less money than you would otherwise.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#855

Post by kylebenzle »

marvin wrote: post #500
kylebenzle wrote: post #462

Earlier quoted context omitted.

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

Humans are likely far less intelligent/capable than systems that will soon be practically feasible to engineer, and we will lose to a generally superhuman AI that has sufficiently different objectives than we do. So an accident with a runaway a system like that would be catastrophic. Then Yudkowsky spins a gauntlet of no-evidence hypotheses for why such an accident is inevitable and leads to the death of literally al…

But can ANYONE bridge the gap from LLMs to an AGI?

I've been programming since highschool, 5 years doing data science and I am at a total loss as to what people that the threat it.

Like, ACTUALLY, what do you think is the fear?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#856

Post by kylebenzle »

berniedurfee wrote: post #515
kylebenzle wrote: post #462

Earlier quoted context omitted.

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

I don’t think we’re close, not even within 50 years of real AGI. Which may or may not decide to wipe us out. However, even given the current state of “AI”, I think there are countless dangers and risks. The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society. Even just the transition of spam and phishing from poorly worded blasts with grammatical err…

100 years ago anyone could write a book and put someone else's name as the author.

So we can't trust video or audio of famous people anymore, again what is the fear?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#857

Post by kylebyte »

ethanbond wrote: post #343
kylebyte wrote: post #292

Earlier quoted context omitted.

I'm pretty sure the worst case of your nuclear war scenario is actually the worst case of the climate change one. Chunks of the world suddenly becoming unlivable and resources getting more scarce sounds like a recipe for escalation into war to me.

The nuclear war scenario, according to all known information about nuclear war policies, is not “chunks of the world becoming unlivable.” The principals in a nuclear conflict do not appear to even have a method to launch just a few nukes in response to a nuclear attacks: they will launch thousands of warheads at hundreds of cities.

I was unclear, I meant chunks of the world becoming unlivable from climate change could easily be the catalyst for a nuclear war. So the worst case for both is the same in my mind.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#858

Post by Veedrac »

brutusborn wrote: post #846
temphypercube wrote: post #842

Earlier quoted context omitted.

The argument is that you don't have to explicitly make a system self-interested, but that self-preservation follows as an implied subgoal of almost any goal. Whatever it is your system actually 'wants', it can't make it happen if it doesn't exist. The obvious rejoinder is 'just make the system want to do what you want it to do', which does fix this problem! But the biggest problem is that we don't know how to do this…

I think I understand the general premise, just not how it would follow specifically. Say you have an AI that is setup as an agent that can give tasks to members of a company to maximise company performance measured by financials and employee wellbeing. To accomplish this goal, the AI develops the instrumental goal of not being deactivated. If your AI is only allowed to give tasks to employees, how would this instrume…

Note that an AI system being put in a situation intended to maximize some metric like company finances is not the same as that AI system directly or ultimately optimizing on those metrics, any more than the goal of a random McDonalds worker is necessarily to make McDonalds wealthier. There's agreement here only as long as whatever inner optimizer that AI system is using finds the situation it's in is most concords with what it's optimizing for, and what it's optimizing for is probably some much more naturalistic, unchosen characteristic of how it was trained and instantiated, modulated by selection pressures that state that grabby preferences last longer and have greater impact than benign ones.

Those preferences need not exist because anything wanted them there; they just need enough input entropy to show up, and enough competitive advantage to stay around. Nobody decided that prokaryotic microbes should exist and have the downstream impact of all of the biological world, just as nobody needs to decide that a system that is capable of robustly replicating against adversarial pressure should therefore robustly replicate against adversarial pressure in actuality. The problem is ultimately that the existence of those capabilities puts you very close to a cliff-edge where those capabilities are exercised in some way that gets selected for.

> If your AI is only allowed to give tasks to employees, how would this instrumental goal turn malicious? And how would this maliciousness cause harm if the only messages sent from the AI are tasks?

It's not to hard to think of concrete answers to this question even restricting oneself to acknowledging capabilities we see in actual humans of normal intelligence and human throughput, but the more important point is simply: Yes, limiting the ways weak unaligned AGI can interact with the world can in fact mitigate harm, and this is in fact a good reason for leading-edge AI development to happen in a way where it's possible at all even in theory for AGI to have limitations on how it interacts with the world.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#859

Post by tome »

upbeat_general wrote: post #790
tome wrote: post #212

Earlier quoted context omitted.

Could I ask what exactly you get from 4 that you don't get from 3.5?

Personally it’s not one specific function but the answer I’m looking for [e.g, code that just works] is significantly better from GPT-4 imo. The better the output is, the less time I spend and the more likely I am to default to using the model in the first place.

I see. So even though 3.5 would attempt to answer your question, the results from 4 are significantly better?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#860

Post by gooseus »

homefree wrote: post #847
gooseus wrote: post #382

Earlier quoted context omitted.

Oh man, this podcast, I still remember walking down the street and having to take a break multiple times because I would just start exclaiming out loud (JFC!) like a lunatic listening to Eliezer talk about the AI Box Experiment as evidence of something. If you look up the real results of this AI Box "experiment" that Eliezer claims to have won 3 of 5 times, you find that there aren't isn't any actual data or results…

The box experiment was just an example that people could be persuaded to let it out even if the AI was initially constrained. Basically that people are imperfectly secure. It’s also not that important given it’s unlikely to be put in a box in the first place. Your latter point about AGI exploring the universe makes a lot of implicit assumptions about its reasoning. The point of the paperclip maximizer example and the…

> Your latter point about AGI exploring the universe makes a lot of implicit assumptions about its reasoning.

Absolutely, that's actually kind of my point... Anyone who tries to predict how some AGI will behave will be making a lot of implicit assumptions about how that AGI will see the world. This is why I said:

> Personally, I think the way these people talk about what a potential AGI would do reveals a lot about more about how they see the world (and humanity) than how any AGI would see it.

Since AGI could conceivably take any form, these discussions end up being a kind of Rorschach test that allow people to tell stories based strongly on their own personal fears and desires.

People who say that AGI will look at humans like we look at ants or apes, and exterminate us if we get in their way are saying a lot about how they view ants and/or apes. I doubt you'd find myrmecologists (or anyone dedicating their lives to studying less complex life) assuming a highly intelligent AGI would want to exterminate lesser life forms.

With regards to the paper-clip maximizer, I think a run-away dumb AI is more likely, but not as risky since they aren't considered to be so intelligent that humans can't figure out how to stop them. You just need to include some kind of regulating function in with your utility function, it seems equivalent to making sure you don't accidentally turn all of your iron plates into iron sticks in Factorio. Def possible, and certainly sucks, but it's not the end of the world.

I have a hard time conceiving of an AI that is smart enough to manipulate people, break out of every containment system, be unstoppable by humanity... be can't be reasoned with because it's entire goal is to just maximize paperclips. I honestly don't think an entity can possess the ability to defeat the collective intelligence of humanity, yet lack the ability to understand the universe in a similar way; for instance, if it is incapable of altering it's goal from "Maximizing Paperclips", then that fact would be a likely path to a vulnerability we could use to stop it.

Post reply on HN