Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

241–250 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#241
post #156

Earlier quoted context omitted.

If tomorrow TSMC, NVIDIA and ASML get destroyed in the next 9/11, it's "stopped". It's completely possible.

Fantasies, bordering on threats.

Of course it's a threat. Civilians doing that would be bad. But governments do much the same via military intervention on a regular basis, hence Yudkowsky talking about airstrikes that one time.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#242

Earlier quoted context omitted.

AI not being allowed in high-risk applications is a bad idea, same for explicable in human terms (cannot do that now for human decisions anyway). The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words.

The «high risk applications» mentioned here seem to be of social risk.

And AI fear cares no social risk? Some people are talking about internment, war, etc. to prevent certain forms of AI - not sure the right risks are in focus here.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#244
post #77

Earlier quoted context omitted.

And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

> With threat of military intervention.

I assume you are american, right? Cause that sounds pretty american. Although I suppose you might also be chinese or russian, they also like that solution a lot.

Whichever of those you are, I'm sure your side is definitely the one with all the right answers, and will be a valiant and correct ruler of the world, blessing us all with your great values.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#245

Earlier quoted context omitted.

> The degree of global policing and surveillance necessary to fully prevent secret labs from working on AI would be mind-boggling. It's not that bad given the compute requirements for training even the basic LLMs we have today. But yes, it's a long shot.

Training a SOTA model is expensive, but you only need to do it once, and fine-tune a thousand times for various purposes. And it's not even that expensive when compared to the cost of building other large scale projects. How much is a dam, or a subway station? There are also corporations who would profit from making models widely available, such as chip makers, they would commoditise the complement. Once you have you…

> Once you have your very capable, open sourced model, that runs on phones and laptops locally, then fine-tuning is almost free.

Hence the idea to ban development of more capable models.

(We're really pretty lucky that LLM based AGI might be the first type of AGI made, it seems much lower risk and lower power than some of the other possibilties)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#246

Earlier quoted context omitted.

And yet, it doesn't seem to be that easy with chips.

Yet China and Russia are still getting their chips, by smuggling if need be. So much for the power of blocking access.

China sure, to a degree (though they also make a lot of the phones and therefore just get shipped tons of chips), but wasn't Russia so short they stole washing machines for the chips? Or was that just war propaganda?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#247

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

How are you anti-doomers arguments still this naive years later? No one is saying global coordination is in anyway remotely easy at all. They’re saying that it doesn’t matter what the difficulty is, the other option is fucking catastrophic

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#248

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

There is no existential risk now and there might never be. So "stop", especially with the use of violence, is a pretty odd position to take to say it mildly.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#249

Earlier quoted context omitted.

This is the type of weak dismissal I’m talking about.

A weak dismissal for a weak argument. The strong dismissal is that many processes, even relatively simple ones, are formally chaotic in the https://en.wikipedia.org/wiki/Chaos_theory sense, meaning that predicting how they evolve over time becomes exponentially more difficult the further ahead in time we try to predict. Meaning we quickly get to the point where all the energy in the known universe wouldn't be suffici…

They don't have to perfectly predict the future, they just have to run faster than we do, unconstrained by a brain that needs to fit in a birth canal or neurons that need chemicals. We had a natural limit to intelligence, for anything more we need to collaborate, often across time and space and while dealing with people who can't stop warring with each other for any number of reasons. It doesn't take much imagination to see how that is beaten. I'd guess a good percentage would be worshiping at the foot af the AI within a few months, so we'd be competing with ourselves in no time.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#250

Earlier quoted context omitted.

Sure, it's not likely to work, but Yud doesn't know of an idea with a better chance of working. If you agree with him on the likelihood of superintelligence in the next few decades and the difficulty of value alignment, what course of action would you suggest?

I think the core is not to agree with the existential risk scenario in the first place. Once you believe some malevolent god will appear and doom us all, anything can be justified (and people have done so in the past - no malevolent god appeared then, btw.).

What's your view on where the X risk idea breaks down?

Is it that it's impossible to make something smarter than a human? Or that such a thing wouldn't have goals/plans that require resources us humans want/need? Or that such a thing wouldn't be particularly dangerously powerful?

Post reply on HN