Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

611–620 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#611

Post by [deleted] »

jacquesm wrote: post #363
api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…

[deleted]

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#612

Post by nyssos »

shoubidouwah wrote: post #478
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

One good way of staying to a prudent course / halting haphazard - winner takes all progress is to use social tech for it. In particular, there is an interesting possibility: the top researchers who drive the AI scenario all kind of know each other, and can self-organize in an international union, with pooled resources for strikes. AI revolution is dependent on simultaneous progress and cross capitalization of good id…

> the top researchers who drive the AI scenario all kind of know each other, and can self-organize in an international union, with pooled resources for strikes.

This is skipping to the end in the same way that "governments should just agree to work together" is. The hard part is building an effective coordination mechanism in the first place, not using it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#614

Post by danenania »

randomdata wrote: post #570
concordDance wrote: post #213

Earlier quoted context omitted.

> The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. How sure are you that they don't think the risk is real? I have only third hand accounts, but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously, its not…

> but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously Everyone has an opinion, but the experts seem strangely silent on this matter. I mean experts in human risk, not experts in AI. The latter love to ramble on, but are unlikely to know anything about social and humanitarian consequences – at least no more than the stock boy at Walmart. Dedicating one's life…

Conversely, why should we expect an expert in "human risk" with no understanding of AI to have anything productive to add to the conversation? We should be looking for people who are well-grounded in both areas, if any exist.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#615

Post by tivert »

api wrote: post #470
patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

The most likely outcome of this exercise in pumping the brakes is the monopolization of high-end AI by a small number of very rich or powerful people. They will then be able to leverage it as an intelligence amplifier for themselves while renting out access to inferior versions of its capabilities to the rest of humanity and becoming even more extraordinarily wealthy. If AI really is going to take off as much as some…

> The most plausible of the "foom" scenarios is one in which a tiny group of humans leverage superintelligence to become living gods. I consider "foom" very speculative, but this version is plausible because we already know humans will act like that while AI so far has no self-awareness or independence to speak of.

One of my hopes is that "superintelligence" turns out to be an impossible geekish fantasy. It seems plausible, because the most intelligent people are frequently not actually very successful.

But if it is possible, I think a full-scale nuclear war might be a preferable outcome to living with such "living gods."

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#616

Post by SilasX »

jeffparsons wrote: post #194
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

But it will work fine for limiting GHG or CFC emissions?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#617

Post by aeternum »

RandomLensman wrote: post #345
rpigab wrote: post #294

Earlier quoted context omitted.

Or decrease, because who would go to the casino if everyone knew the outcome? Only the owners and staff. "Useless" companies are part of the total market value. If they never get any funding, that's less value overall. Even if that translates to more value for "useful" companies. Also, if you know with absolute certainty from the beginning that Apple Inc is going to be worth X billion dollars, then you never get to b…

You could calculate the value of things under the discounting changes. The idea of perfect foresight of the future is kind of insane anyway. Not sure why all of sudden we would go back to believing in a deterministic universe.

The universe can be both deterministic and mostly unpredictable. In fact it most likely is both of those things.

Entropy bounds computation and prediction of chaotic systems requires extreme amounts of computation.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#618

Post by NoGravitas »

devnonymous wrote: post #432
toth wrote: post #418

Earlier quoted context omitted.

I am not saying it's not a huge problem. Hundreds of millions dying would probably make it the worst thing that ever happened. I am just saying it's not an extinction risk.

...tbh, imo, the impact of this is already visible in the world (eg: it can be argued that the conflict in Ukraine is directly linked to climate change) whereas ASI is still fantasy that exists only in the minds of the AI 1% , to use the terminology of the OP.

I believe you are correct on both points. I believe the first major war to be caused substantially by climate change was the Syrian Civil War, followed by the Yemeni Civil War.

The main AI threats for the foreseeable future are its energy usage contributing to climate change, and the use of generative AI to produce and enhance political unrest, as well as its general use in accelerating other political and economic crises. AGI isn't even on the radar, much less ASI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#619

Post by RandomLensman »

hackinthebochs wrote: post #598
RandomLensman wrote: post #593

Earlier quoted context omitted.

There is an infinity of hypothetical dangers. Then at the very least there should be a clear and public discussion as to why the AI danger is more relevant than others than we do not do much about (some not even being hypothetical). That is not happening.

I'm in favor of public discussions. It's tragic that there is a segment of the relevant powers that are doing their best to shut down any of these kinds of discussions. Thankfully, they seem to be failing. But to answer the question, the danger of AI is uniquely relevant because it is the only danger that may end up being totally out of our control. Nukes, pandemics, climate change, etc. all represent x-risk to vario…

I don't agree with the uniqueness of AI risk. A large asteroid impacting earth is a non-hypothetical existential risk presently out of our control. We do not currently plan on spending trillions to have a comprehensive early detection system and equipment to control that risk.

The differentiating thing here is that blocking hypothetical AI risk is cheap, while mitigating real risks is expensive.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#620

Post by hollerith »

Workaccount2 wrote: post #474
patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

I fully recognize the danger AI presents, and believe that it will mostly likely end up being terrible for humanity. But thanks to my own internal analysis ability and the anonymity of the internet, I am also willing to speak candidly. And I think I speak for many people in the tech community, whether they realize it or not. So here we go: My objective judgement of the situation is heavily adulterated by my incredibl…

Do you regret choosing AI development as a career?
Post reply on HN