Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

341–350 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#341

Post by toth »

toth wrote: post #337
M95D wrote: post #315

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. > There is no scenario where climate change leads to human extinction. He was talking about the human CIVILIZATION...

You are right, but "human civilization will be extinct" is an awkward phrase. Human extinction via climate change is off the table. The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's t…

Sorry, I meant IPCC (International Panel on Climate Change).

I don't think IUPAC (International Union of Pure and Applied Chemistry) has come out with views on impact of global warming :)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#342

Post by mangosteenjuice »

toth wrote: post #337
M95D wrote: post #315

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. > There is no scenario where climate change leads to human extinction. He was talking about the human CIVILIZATION...

You are right, but "human civilization will be extinct" is an awkward phrase. Human extinction via climate change is off the table. The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's t…

Right, the collapse of our current civilization is inevitable. That doesn't mean it won't be succeeded by more advanced ones in the future

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#343

Post by ethanbond »

kylebyte wrote: post #292
toth wrote: post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

I'm pretty sure the worst case of your nuclear war scenario is actually the worst case of the climate change one. Chunks of the world suddenly becoming unlivable and resources getting more scarce sounds like a recipe for escalation into war to me.

The nuclear war scenario, according to all known information about nuclear war policies, is not “chunks of the world becoming unlivable.”

The principals in a nuclear conflict do not appear to even have a method to launch just a few nukes in response to a nuclear attacks: they will launch thousands of warheads at hundreds of cities.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#344

Post by ryanklee »

RandomLensman wrote: post #160
troupo wrote: post #153

Earlier quoted context omitted.

> AI not being allowed in high-risk applications is a bad idea wat > same for explicable in human terms (cannot do that now for human decisions anyway). What? Human decisions can be explained and appealed. Good luck proving something when a blackbox finds you guilty of anything.

You can have ex-post narratives for what humans do, but that is not the same as an actual explanation. Actual human cognition is a black box as of now. The issue of appeals processes is totally separate. That doesn't mean we handover everything to AI, but the idea that we understand human decisions cannot be the reason for that. For some high risk things like controlling certain complex machinery or processes AI migh…

That's a bit of an exaggeration. We understand high level motivations as being highly correlated to certain outcomes. Hunger, eat. Horny, sex. Wronged, vengeance. Money, directed. You don't necessarily need to know the underlying processes to make reliable predictions or valid causal explanations about human behavior.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#345

Post by RandomLensman »

rpigab wrote: post #294
RandomLensman wrote: post #148

Earlier quoted context omitted.

If every investment became risk free because of perfect information about the future, it would likely increase the total market value by quite a lot.

Or decrease, because who would go to the casino if everyone knew the outcome? Only the owners and staff. "Useless" companies are part of the total market value. If they never get any funding, that's less value overall. Even if that translates to more value for "useful" companies. Also, if you know with absolute certainty from the beginning that Apple Inc is going to be worth X billion dollars, then you never get to b…

You could calculate the value of things under the discounting changes.

The idea of perfect foresight of the future is kind of insane anyway. Not sure why all of sudden we would go back to believing in a deterministic universe.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#346

Post by ryanklee »

brutusborn wrote: post #89
PeterStuer wrote: post #48

People focus on LLM's and diffusion models because they are so omnipresent now. But an AI for stock picking and prediction that would be realy next level and outcompete the current ones cosistently would siphon of so much wealth it would basically own society if the operators were clever enough not to get so gready short term that the system would litterally collapse overnight.

I have wondered if this is the risk that Elon is worried about. But he can’t discuss it explicitly because he is working on such a system. If he communicates its existence, then he reveals a huge attack surface and makes his life (and his teams) much more difficult. So he is limited to publically saying that AI is dangerous but not revealing the true failure mode.

Musk is a buffoon. He doesn't know anything more than he's told the whole world. He knows less.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#347

Post by ethanbond »

PeterStuer wrote: post #85
samus wrote: post #73

Earlier quoted context omitted.

This device only works if these are two completely equal services served by two equally capable persons with the same investment of time and resources. And it assumes that we mutually trust each other to actually do the thing. Also, bringing money into the game makes the responsibility stronger. Especially if those two persons are not living in a vacuum, but have other bills to pay [edit: i.e., to make other people d…

One could argue GDP measures the degree of financialization of a society more than it measures the presence or absense of abundance.

In fact, Kusnetz the actual inventor of the modern GDP measure said something similar:

“ Economic welfare cannot be adequately measured unless the personal distribution of income is known. And no income measurement undertakes to estimate the reverse side of income, that is, the intensity and unpleasantness of effort going into the earning of income. The welfare of a nation can, therefore, scarcely be inferred from a measurement of national income as defined above.”

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#348

Post by TeMPOraL »

varjag wrote: post #318
TeMPOraL wrote: post #287

Earlier quoted context omitted.

> Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabotage competing efforts. I'd reconsider your revision of your estimate of Yudkowsky, as you seem to be dropping it for not proposing the very ideas he spent the last 20+…

Threat of war is always the final argument in international affairs, and you can tell how serious a treaty is by how directly it acknowledges that fact. Montreal Protocol have worked despite no threats of violence, on a not entirely dissimilar problem. Tho I share the skepticism on solution to alignment problem.

That's a great and very relevant case study, thanks for bringing it up!

The way I understand, it worked because alternatives to the ozone-destroying chemicals were known to be possible, and the costs of getting manufacturers to switch, as well as further R&D, weren't that big. I bucket it as a particularly high-profile example of the same class as most other international treaties: agreements that aren't too painful to follow.

Now in contrast to that, climate agreements are extremely painful to follow - and right now countries choose to make a fake effort without actually following. With a hypothetical AI agreement, the potential upsides of going full-steam ahead are significant, there are no known non-dangerous alternatives, so it won't be followed unless it comes with hard, painful consequences. Both climate change and AI risk are more similar to nuclear proliferation issue.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#349

Post by api »

jacquesm wrote: post #339
api wrote: post #330

Earlier quoted context omitted.

So ask it how to stop someone from doing this. Ask it how to augment the human immune system. Train AIs on the structures of antibiotics and antiviral drugs and have them predict novel ones. Create an AI that can design vaccines against novel agents. Those things could stop our terrorist and could also stop the next breakout zoonotic virus or “oops!” in a lab somewhere. Intelligence amplification works for everyone.…

There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups.

There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now.

Even without AI our continued technological advancement will keep giving us more and more power, as individuals and especially groups. If we don’t think we can climb this mountain without destroying ourselves then it means the entire scientific and industrial endeavor was when we signed our own death warrant, AI or not. You can order CRISPR kits.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#350

Post by actualwitch »

lampington wrote: post #319
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

> If they are Am I the only one slightly disturbed by the use of "they" in that sentence? I know that the HN commentariat is broad-based, but I didn't realise we already had non-human members ;-)

Beep boop
Post reply on HN