Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario • Hacker News Board

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#701

Post by bbor » Thu, Nov 02, 2023, 6:25 PM UTC

tivert wrote: post #615

api wrote: post #470

Earlier quoted context omitted.

The most likely outcome of this exercise in pumping the brakes is the monopolization of high-end AI by a small number of very rich or powerful people. They will then be able to leverage it as an intelligence amplifier for themselves while renting out access to inferior versions of its capabilities to the rest of humanity and becoming even more extraordinarily wealthy. If AI really is going to take off as much as some…

> The most plausible of the "foom" scenarios is one in which a tiny group of humans leverage superintelligence to become living gods. I consider "foom" very speculative, but this version is plausible because we already know humans will act like that while AI so far has no self-awareness or independence to speak of. One of my hopes is that "superintelligence" turns out to be an impossible geekish fantasy. It seems pla…

I do wonder how long this bias towards the status quo will survive human history. This is the first big shakeup (what if we’re not the smartest beings?), and I think the next is prolly gonna be long lives (what will the shape of my life be?). From there it’s full on sci-fi speculation, but I definitely am putting money that humans will incrementally trade comfort and progress for… I guess existential stability.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#702

Post by jrflowers » Thu, Nov 02, 2023, 6:27 PM UTC

FeepingCreature wrote: post #173

jrflowers wrote: post #109

Earlier quoted context omitted.

Out of curiosity can you point to anyone that makes a “doomer” argument that isn’t Eliezer Yudkowsky or restating his specific points? Refuting the cult of personality counterpoint with “no really this one guy can see the future” is

It turns out if you're thinking about a problem twenty years before everyone else you tend to be the first person to make lots of arguments. So I don't see how "without restating his specific points" is supposed to be feasible. If Eliezer has already made all the strongest points, are people supposed to invent new ones?

> If Eliezer has already made all the strongest points, are people supposed to invent new ones?

I like this post because the rebuttal to “there’s a kind of cult of personality around this guy (1) where everybody just sort of agrees that he’s categorically correct to the extent that world leaders are stupid or dangerous not to defer to the one guy on policy” is “Yeah he really can see the future. The proof of that is he’s been blogging about a future that’s never come to pass for the longest time”

1 https://www.lesswrong.com/posts/Ndtb22KYBxpBsagpj/eliezer-yu...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#703

Post by vidarh » Thu, Nov 02, 2023, 6:28 PM UTC

hackinthebochs wrote: post #551

vidarh wrote: post #454

Earlier quoted context omitted.

I don't think you understand how little it takes to have Turing complete computation, and how hard it is to stop even end-users from accessing it, much less companies or a motivated nation state. The "war on general-purpose computation" is more like a "war on convenient general-purpose computation for end users who aren't motivated or skilled enough to work around it". Are you going to ban all spreadsheets? The abili…

>I don't think you understand how little it takes to have Turing complete computation This is a dumb take. No one's calculator is going implement an AGI. It will only happen in a datacenter with an ocean of H100 GPUs. This computing power does not materialize out of thin air. It can be monitored and restricted.

No one's calculator needs to. The point was to reply to the notion that there is any possible way the "war on general-purpose computation" actually has any shot of actually stopping general-purpose computation to make a point about how hard limiting computation is in general.

> It will only happen in a datacenter with an ocean of H100 GPUs. This computing power does not materialize out of thin air. It can be monitored and restricted.

Access to H100 could perhaps be restricted. That will drive up the cost temporarily, that's all. It would not stop a nation state actor that wanted to from finding alternatives.

The computation cost required to train models of a given quality keeps dropping, and there's no reason to believe that won't continue for a long time.

But the notion you couldn't also sneak training past monitoring is based on the same flawed notion of being able to restrict general purpose computation:

It rests on beliefs about being able to recognise what can be used to do computation you do not want. And we consistently keep failing to do that for even the very simplest of cases.

The notion that you will be able to monitor which set of operations are "legitimate" and which involves someone smuggling parts of some AI training effort past you as part of, say, a complex shader is as ludicrous as the notion you will be able to stop general purpose computation.

You can drive up the cost, that is all. But if you try to do so you will kill your ability to compete at the same time.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#704

Post by vidarh » Thu, Nov 02, 2023, 6:30 PM UTC

carbotaniuman wrote: post #693

vidarh wrote: post #434

Earlier quoted context omitted.

We're nowhere near there . Even on the most locked down devices we have, getting computation past the gatekeepers doesn't require reviews remotely detailed enough to catch attempts at sneaking these kinds of computation past, or even making them expensive, and at no point in history have even individuals been able to rent as much general purpose computational capacity for as little money. Even almost all the most loc…

There's also the bomb every datacenter doing AI thing that LeCun has proposed, which is a lot more practical than this. Or perhaps a self replicating work that destroys AIs.

This was why I pointed out that if I was a nation state worried about this, I'd host my AI training at the cloud providers of the country making those threats... Let them bomb themselves. And once they've then destroyed their own tech industry, you can still buy a bunch of computational resources elsewhere and just not put them in easily identifiable data centres.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#705

Post by jebarker » Thu, Nov 02, 2023, 6:31 PM UTC

pixl97 wrote: post #654

jebarker wrote: post #539

Earlier quoted context omitted.

This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.

The calculation of considering something is a credible threat directly corresponds with the ease of implementation. If I said "I am going to drop a nuke on jebarker" that's not a credible threat unless I happen to be a nation state. Now, if I said "I'm going to come to jebarkers house and blast him with a 12 gauge" and I'm from the US, that is a credible threat in most cases due to the ease of which I can get a weapo…

The problem with this analysis is that we have complete uncertainty about the probability that AGI/ASI will be developed in any particular time frame. Anyone that says otherwise is lying or deluded. So the risk equation is impossible to calculate for AGI/ASI wiping out humanity. And since you appear to be evaluating the damage as essentially infinite, i.e. existential risk, you're advocating that as long as there's a greater than zero probability of someone using that capability then the risk is infinite. Which is not useful for deciding a course of action.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#706

Post by sdwr » Thu, Nov 02, 2023, 6:33 PM UTC

bondarchuk wrote: post #689

jonathankoren wrote: post #640

Earlier quoted context omitted.

But it’s not a real world risk. Arguably, it’s even less of a real world risk because it’s a common sci-fi trope. The whole concept is farcical.

> Arguably, it’s even less of a real world risk because it’s a common sci-fi trope. It would be nice if the world worked that way. Then we could diminish any risk just by writing a lot of scifi about it :)

That's exactly how it works. Invest thought into it before it happens, then when it becomes an issue, there's already awareness and possibilities have been mapped out.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#707

Post by nyc_data_geek1 » Thu, Nov 02, 2023, 6:34 PM UTC

bnralt wrote: post #561

upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Every time I see Yudkowsky’s writing I wonder why people take him seriously. Just a couple of months back he was duped by a fake story about an airforce AI deciding to kill it’s operators in a simulation. It turned out the whole thing was thought experiment. Asking Yudkowsky what we should do about AI is like asking Green Peace what we should do about nuclear power.

Anybody subscribing to twitter is immediately disqualified as a credible, rational actor in my book.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#708

Post by jebarker » Thu, Nov 02, 2023, 6:34 PM UTC

jefftk wrote: post #547

jebarker wrote: post #539

Earlier quoted context omitted.

This doesn't really answer my question. These are arguments for why threat actors might hold the belief that "humanity is cancer", but it doesn't actually provide evidence that this is a credible threat in the real world.

The two people I cited at the end are examples of philosophers who think a lifeless world is preferable to the status quo and to futures we are likely to get. These aren't arguments for why someone might hold the belief, but evidence that some people do seriously hold the belief. (Another example here would be https://en.m.wikipedia.org/wiki/Aum_Shinrikyo )

Fair enough. I don't doubt there are people in the world that hold this view. I just don't know how credible they are as a threat to humanity.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#709

Post by godelski » Thu, Nov 02, 2023, 6:35 PM UTC

lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

Importantly locking away model checkpoints makes AI safety research, at best, exceptionally difficult or impossible. We already don't know all the training methods nor do we know the data used for the models. AI safety is absolutely contingent on works being __more__ open source, not less. But there's less profit in that for these companies (potentially more profit for the country as a whole though, as we see new businesses form and sectors develop, but this is more spread out and far harder to calculate the value of so let's ensure we recognize this as speculative).

We also need to be keenly aware that xrisk is not the major issue here. We do not need AGI to create massive amounts of harm, nor do we need AGI to create a post-scarce world (which the transition to can also generate massive harm if we don't do it carefully. Despite generating a post-scarce world -- or at least getting ever closer to that -- is, imo, one of the most important endeavors we can work on). Current ML systems can already do massive amounts of destruction, but so can a screwdriver.

My big fear is that AI will go the way of nuclear. A technology is not good or bad, it is like a coin with a value. You can spend that coin on something harmful or something beneficial, but the coin is just a coin at the end of the day. That's what regulation is supposed to be about, expenditure, not minting.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#710

Post by nonbirithm » Thu, Nov 02, 2023, 6:37 PM UTC

Workaccount2 wrote: post #474

patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

I fully recognize the danger AI presents, and believe that it will mostly likely end up being terrible for humanity. But thanks to my own internal analysis ability and the anonymity of the internet, I am also willing to speak candidly. And I think I speak for many people in the tech community, whether they realize it or not. So here we go: My objective judgement of the situation is heavily adulterated by my incredibl…

Exactly! You hit on something I have been thinking about for a long time, but phrased better than I could have. We need to start saying this out loud more and more.

There is a problem that lies above the development of AI or advanced technology. It is the zeitgeist that caused humanity to end up in this situation to begin with, questioning the effects of AI in the future. It's a product of humans surrendering to a neverending primal urge at all costs. Advancing technology is just the means by which the urge is satiated.

I believe the only way we can survive this is if we can suppress that urge for our own self-preservation. But I don't think it's feasible at this stage. We may have to begin questioning the merit of parts of the human condition and society very soon.

Given the choice, I think a lot of people today would love to play God if only the technology was in their hands right this minute. Where does that urge arise from? It deserves to be put in the spotlight.