Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario • Hacker News Board

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#501

Post by 4bpp » Thu, Nov 02, 2023, 2:37 PM UTC

ImPostingOnHN wrote: post #469

TeMPOraL wrote: post #321

Earlier quoted context omitted.

No, I'm proposing replacing a high risk scenario (I disagree with Hinton here) with a low risk scenario, which has low chance to escalate to any serious fighting. It seems people have missed what the proposed alternative is, so let me spell it out: it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world. It's not fundamentally more dang…

> No, I'm proposing replacing a high risk scenario (I disagree with Hinton here) I disagree with you here, I think the risk is low, not high. > it's about getting governments to SIGN AN INTERNATIONAL TREATY. It is NOT about having US (or anyone else) policing the rest of the world We haven't been able to do that for climate change. When we do, then I'll be convinced enough that it would be feasible for AI. Until then…

To continue contributing to climate change takes very little: you need some guy willing to spin a stick for long enough to start a fire, or to feed and protect a flock of cows. To continue contributing to AI, you need to maintain a global multi-billion supply chain with cutting edge technology that might have a four-digit bus factor.

The mechanisms that advance climate change are also grandfathered in to the point that we are struggling to conceive of a society that does not use them, which makes "stop doing all of that" a hard sell. On the other hand, every society at least has cultural memory of living without several necessary ingredients of AI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#502

Post by jprete » Thu, Nov 02, 2023, 2:38 PM UTC

PoignardAzur wrote: post #370

persnickety wrote: post #149

> Every new technology is developed and deployed the same way: > You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely. This makes an assumption: that the problematic technology is an intentional development rather than an emergent feature of an intentionally developed technology. We already have a name for such emergent features: "bugs…

> > You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely. That argument in particular seems like wishful thinking. What in the last 10 years of our tech industry makes him think "move slowly and deploy carefully" is going to be the strategy they adopt now ? And that's without considering the arguments in "We have no moat and neither do…

Open-source models are deployed all over the place _now_ with no concern for safety. Fortunately nobody has yet managed to make a mutatable reproducing system out of them.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#503

Post by oceanplexian » Thu, Nov 02, 2023, 2:41 PM UTC

hollerith wrote: post #460

johnthewise wrote: post #259

Earlier quoted context omitted.

He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. His risk calculations derive primarily from the AI itself, not weaponization of AI by others. Rich Sutton seems to agree with this take and embraces our extinction: https://www.youtube.com/watch?v=NgHFMolXs3U

>He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. That's a little bit inaccurate: he believes that it is humanly possible to acquire a body of knowledge sufficient to align an AI (i.e., to aim it at basically any goal the creators decide to aim it at), but that it is extremely unlikely that any group of humans will manage to do before unaligned AI ki…

I wonder if I, too, can simply rehash the plot of Terminator 2: Judgment Day and then publish research papers on how we will use AI’s to battle it out with other AI’s.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#504

Post by bumby » Thu, Nov 02, 2023, 2:43 PM UTC

JackFr wrote: post #464

jacquesm wrote: post #363

Earlier quoted context omitted.

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

> Case in point: the Kennedy assassination changed the world Really? In what substantive way?

The counterfactuals are impossible to assess, but there are a lot of theories that Kennedy wanted to dismantle the intelligence agencies. These are the same organizations many point to as driving force behind many arguable policy failures, from supporting/disposing international leaders to the drug war and even hot wars.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#505

Post by carapace » Thu, Nov 02, 2023, 2:43 PM UTC

api wrote: post #481

germandiago wrote: post #472

Earlier quoted context omitted.

True. We receive warnings all the time but I think what these people are afraid of is actually losing their power.

Until these same people start being concerned about DIY CRISPR kits, I do not take them seriously: https://www.the-odin.com/diy-crispr-kit/ This is dangerous in ways that require no leaps of logic or hypothetical sci-fi scenarios to imagine. I studied biology as an undergrad and can think of several insanely evil things you could do with this plus some undergrad-level knowledge of genetics and... not gonna elaborate.…

Holy shit. That's legitimately terrifying.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#506

Post by Gravityloss » Thu, Nov 02, 2023, 2:44 PM UTC

jacquesm wrote: post #363

api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

Bill Joy contemplated this in his famous essay from 2000 "Why the Future Doesn't Need Us" https://www.wired.com/2000/04/joy-2/

Technology makes certain kind of bad acts more possible.

I think I was a bit shocked by that article in the day.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#507

Post by hutzlibu » Thu, Nov 02, 2023, 2:47 PM UTC

itsoktocry wrote: post #421

zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

> If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. If we go extinct in the next 100 years, it's not going to be from climate change. How would that even work?

Basically, climate change would creates lots of problems, lots of hungry immigrants. And right now we are already close to a nuclear war, so climate change would not directly wipe out humanity.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#508

Post by Zigurd » Thu, Nov 02, 2023, 2:48 PM UTC

When AGIs allocate capital far better than the best human minds in free markets can, shouldn't the AGIs be the billionaires? Would they not deserve it? And, if not the AGIs, then who?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#509

Post by losteric » Thu, Nov 02, 2023, 2:49 PM UTC

logicchains wrote: post #50

PeterStuer wrote: post #45

Earlier quoted context omitted.

We have been in a state of abundance for a long time. We just choose not to distribute the abundance.

The global average PPP GDP per capita is 18k USD/year. 18k USD/year is not remotely close to abundance.

Median incomes might be a better figure. Per-capita is skewed by outliers.

Look at the US: https://en.m.wikipedia.org/wiki/File:US_GDP_per_capita_vs_me... - huge disparity between mostly flat income versus rapidly growing per-capita

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#510

Post by veidr » Thu, Nov 02, 2023, 2:53 PM UTC

JoshTriplett wrote: post #52

lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

I don't want to be rude; I understand where you are coming from (at least in terms of single-minded risk mitigation from a theoretical and unconstrained viewpoint), but this reads like Mike Huckabee's 'just keep an aspirin pill pinned between your knees' advice to teenage girls on how not to get pregnant.

There is obviously (!!!!!!!!!) no way in this — or any imaginable — timeline that we are going to just "stop".

(And it's hard to imagine any mechanism that could enforce "stop" that isn't even worse than the scenarios we're already worrying about. How does that work? Elon or Xi-Poohbear have to personally sign off on installations of more than 1000 cores?)