Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario • Hacker News Board

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#451

Post by morsch » Thu, Nov 02, 2023, 1:37 PM UTC

logicchains wrote: post #398

morsch wrote: post #189

Earlier quoted context omitted.

Oh, I don't think my post was advocating anybody give anything away. But if we just arrived there, magically, it would be pretty great, from a utilitarian point of view. But thanks for the knee-jerk anti-communist rant, it's really great that we're back to that. Life was great in the 50s.

>Life was great in the 50s. Americans in the 50s were a lot happier than Americans in the 2020s, very little mental illness or obesity, housing was affordable, people weren't at each others' throats like nowadays.

Also: much lower income inequality. Strange, huh. And the average family income was 4400 USD, ie. 50000 USD in 2023, so pretty much the 18000 USD per capita we started with.

Shame about the institutional racism, mysoginy, homophobia, etc though.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#452

Post by ImPostingOnHN » Thu, Nov 02, 2023, 1:38 PM UTC

ethanbond wrote: post #343

kylebyte wrote: post #292

Earlier quoted context omitted.

I'm pretty sure the worst case of your nuclear war scenario is actually the worst case of the climate change one. Chunks of the world suddenly becoming unlivable and resources getting more scarce sounds like a recipe for escalation into war to me.

The nuclear war scenario, according to all known information about nuclear war policies, is not “chunks of the world becoming unlivable.” The principals in a nuclear conflict do not appear to even have a method to launch just a few nukes in response to a nuclear attacks: they will launch thousands of warheads at hundreds of cities.

I suppose thousands of warheads could be launched at hundreds of cities, and that could make chunks of the world uninhabitable and kill millions

climate change will happen, and it will do that, left unchecked

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#453

Post by melagonster » Thu, Nov 02, 2023, 1:41 PM UTC

upupupandaway wrote: post #182

JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

we had seen the result of endeavor to stop global warming. but how can we do...

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#454

Post by vidarh » Thu, Nov 02, 2023, 1:42 PM UTC

TeMPOraL wrote: post #430

vidarh wrote: post #404

Earlier quoted context omitted.

How do you intend to prevent people from carrying out general purpose computation? Because that is what you'd need to do. You'd need to prevent the availability of any device where users can either directly run software that has not been reviewed, or that can be cheaply enough stripped of CPU's or GPU's that are capable of running un-reviewed software. That review would need to include reviewing all software for "com…

> How do you intend to prevent people from carrying out general purpose computation? Ask MAFIAA, and Intel, and AMD, and Google, and other major tech companies, and don't forget the banks too. We are already well on our way to that future. Remember Cory Doctorow's "War on general-purpose computation"? It's here, it's happening, and we're losing it. Therefore, out of many possible objections, this one I wouldn't put m…

I don't think you understand how little it takes to have Turing complete computation, and how hard it is to stop even end-users from accessing it, much less companies or a motivated nation state. The "war on general-purpose computation" is more like a "war on convenient general-purpose computation for end users who aren't motivated or skilled enough to work around it".

Are you going to ban all spreadsheets? The ability to run SQL queries? The ability to do simple regexp based search replace? The ability for users to template mail responses and set up mail filters? All of those allows general purpose computation either directly or as part of a system where each part may seem innocuous (e.g. the simple ability to repeatedly trigger the same operation is enough to make regexp based search and replace Turing complete; the ability to pass messages between a templated mailing list system and mail filters can be Turing complete even if neither the template system and filter in isolation is)

The ability for developers to test their own code without having it reviewed and signed off by someone trustworthy before each and every run?

Let one little mechanism through and the whole thing is moot.

EDIT: As an illustration, here's a Turing machine using only Notepad++'s find/replace: https://github.com/0xdanelia/regex_turing_machine

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#455

Post by jcims » Thu, Nov 02, 2023, 1:44 PM UTC

jacquesm wrote: post #363

api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

>The stories from that night continue to haunt me today.

These conversations are oddly precious and intimate. It's so difficult to find someone that is willing to even 'go there' with you let alone someone that is capable of creatively and fearlessly advancing the conversation.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#456

Post by varelse » Thu, Nov 02, 2023, 1:44 PM UTC

jeffparsons wrote: post #194

upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

[dead]

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#457

Post by ImPostingOnHN » Thu, Nov 02, 2023, 1:46 PM UTC

reducesuffering wrote: post #247

jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

How are you anti-doomers arguments still this naive years later? No one is saying global coordination is in anyway remotely easy at all. They’re saying that it doesn’t matter what the difficulty is, the other option is fucking catastrophic

> They’re saying that it doesn’t matter what the difficulty is, the other option is fucking catastrophic

That's precisely what people have been saying about climate change. Some progress been made there, but if we can't solve that problem collectively (and we haven't been able to), we aren't going to be able to solve the "gain central control over AI" problem

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#458

Post by ryanklee » Thu, Nov 02, 2023, 1:47 PM UTC

RandomLensman wrote: post #450

ryanklee wrote: post #437

Earlier quoted context omitted.

I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior. Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI. It's not going beyond,…

I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories. For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.

Once again, you are raising the bar artificially high by pointing to examples where we fail to have reasonable casual accounts. But you are conveniently ignoring the mountain of reasonable accounts that we employ all day long simply to navigate our human world.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#459

Post by danaris » Thu, Nov 02, 2023, 1:50 PM UTC

lampington wrote: post #319

zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

> If they are Am I the only one slightly disturbed by the use of "they" in that sentence? I know that the HN commentariat is broad-based, but I didn't realise we already had non-human members ;-)

It's perfectly reasonable to refer to "human civilizations" as a plural, since that phrase can very commonly be used to refer to different cultures.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#460

Post by hollerith » Thu, Nov 02, 2023, 1:50 PM UTC

johnthewise wrote: post #259

jeffparsons wrote: post #210

Earlier quoted context omitted.

It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless. Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabo…

He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. His risk calculations derive primarily from the AI itself, not weaponization of AI by others. Rich Sutton seems to agree with this take and embraces our extinction: https://www.youtube.com/watch?v=NgHFMolXs3U

>He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity.

That's a little bit inaccurate: he believes that it is humanly possible to acquire a body of knowledge sufficient to align an AI (i.e., to aim it at basically any goal the creators decide to aim it at), but that it is extremely unlikely that any group of humans will manage to do before unaligned AI kills us all. There is simply not enough time because (starting from the state of human knowledge we have now) it is much easier to create an unaligned AI capable enough that we would be defenseless against it than it is to create an aligned AI capable enough to prevent the creation of the former.

Yudkowsky and his team have been working the alignment problem for 20 years (though 20 years ago they were calling it Friendliness, not alignment). Starting around 2003, his team's plan was to create an aligned AI to prevent the creation of dangerously-capable unaligned AIs. He is so pessimistic and so unimpressed with his team's current plan (ie., to lobby for governments to stop or at least slow down frontier AI research to give humanity more time for some unforeseen miracle to rescue us) that he only started executing on it about 2 years ago even though he had mostly given up on his previous plan by 2015.