Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

431–440 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#431

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

This is more or less exactly where I land on this. The extinction risk argument is somewhat plausible in the long term[0], but it doesn't seem that we are especially close to developing that class of problem. Meanwhile, there are very real and immediate risks to this type of technology -- in various forms of giving some humans more power over other humans. The attacks proposed by the extinction risk people actively m…

>The extinction risk argument is somewhat plausible in the long term[0], but it doesn't seem that we are especially close to developing that class of problem.

The parable at the beginning of Superintelligence relates to this; the problem looks far away, but we don't know where, precisely it will arise, or if it will be too late when we realize it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#432
post #418

Earlier quoted context omitted.

Climate change has compounding effects (eg: water scarcity, leading to political conflict, leading to displacement, leading to pandemics ...and that's just one vector). Real world is real real messy. COViD should have taught people that. Unfortunately, it didn't.

I am not saying it's not a huge problem. Hundreds of millions dying would probably make it the worst thing that ever happened. I am just saying it's not an extinction risk.

...tbh, imo, the impact of this is already visible in the world (eg: it can be argued that the conflict in Ukraine is directly linked to climate change) whereas ASI is still fantasy that exists only in the minds of the AI 1% , to use the terminology of the OP.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#433

Earlier quoted context omitted.

AI not being allowed in high-risk applications is a bad idea, same for explicable in human terms (cannot do that now for human decisions anyway). The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words.

> The latter would kind of rule out any quantum computing, too, as the only real explanation there is through maths, not words. Maths are human terms.

If the level of explanation is something that the person suffering from the decision need not necessarily understand, fair enough. But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#434
post #383

Earlier quoted context omitted.

Sure, you can insist on monitoring which software can be run on every cloud service, every managed server, on the servers in every colo, or on every computer owned by everyone (of any kind, including basic routers or phones, or even toys, - anything with a CPU that can be bought in sufficient bulk and used for computations), and subject every piece of code allowed to run at any kind of scale to a formal review by gov…

It all sounds very bad, but the truth is, we're going to get there anyway - in fact, we're half-way there, thanks to the influence of media conglomerates pushing DRM, adtech living off surveillance economy, all the other major tech vendors seeking control of end-user devices, and security industry that provides them all with means to achieve their goals. Somehow, no one is worried about the economy. Perhaps because i…

We're nowhere near there. Even on the most locked down devices we have, getting computation past the gatekeepers doesn't require reviews remotely detailed enough to catch attempts at sneaking these kinds of computation past, or even making them expensive, and at no point in history have even individuals been able to rent as much general purpose computational capacity for as little money.

Even almost all the most locked down devices available on the market today can still either be coaxed to do general purpose calculation one way or another or mined for GPU's or CPU's.

No one is worried about the economy because none of the restrictions being pushed are even close to the level that'd be needed to stop even individuals from continuing general purpose computation, much less a nation state actor.

This is without considering the really expensive ways: Almost every application you use can either directly or in combinations become a general purpose computational device. You'd need to ensure e.g. that there'd be no way to do sufficiently fast batch processing of anything that can be coaxed into suitable calculations. Any automation of spreadsheets or databases, for example. Even just general purpose querying of data. A lot of image processing. Bulk mail filtering where there's any way of conditionally triggering and controlling responses (you'd need multiple addresses, but even a filtering system like e.g. Sieve that in isolation is not Turing complete becomes trivially Turing complete once you allow multiple filters to pass messages between multiple accounts).

Even regexp driven search and replace, which is not in itself Turing complete becomes Turing complete with the simple addition of a mechanism for repeatedly executing the same search and replace on its own output. Say a text editor with a "repeat last command" macro key.

And your reviewers would only need to slip up once and let something like that slip past once with some way of making it fast enough to be cheap enough (say, couple the innocuous search-and-replace and the "repeat last command" with a an option to run macros in a batch mode) before an adversary has a signed exploitable program to use to run computations.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#435

Earlier quoted context omitted.

Climate change has compounding effects (eg: water scarcity, leading to political conflict, leading to displacement, leading to pandemics ...and that's just one vector). Real world is real real messy. COViD should have taught people that. Unfortunately, it didn't.

> Climate change has compounding effects Why is the default to assume that every change will be negative?

Perhaps because we aren't seeing evidence to the contrary?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#436
post #276

Earlier quoted context omitted.

There is no scenario where climate change leads to human extinction. Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad. Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors. Unaligned ASI though, could actually make us…

Climate change and nuclear war are not orthogonal risks. Climate change leads to conflict. For example, the Syria drought of 2006-2010. More climate change leads to larger conflicts, and large conflicts can lead to nuclear exchanges. Think about what happens if India and Pakistan (both nuclear powers) get into a major conflict over water again.

India and Pakistan didn’t nuke each other last time(s)… even though they had nuclear weapons.

The assumption we’d need to use nukes is insane. For the same cost (and energy requirements) for a set of nukes we can filter salt from ocean water or collect rain.

I agree famine and drought can cause conflict. But we are no where need that. If you read the climate change predictions (from the UN), they actually suspect more rain (and flooding) in many regions.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#437

Earlier quoted context omitted.

I didn't say it was fully explainable. And you don't need to settle debates about free will to offer valid causal explanations for human behaviors. Are you saying we don't know at all why anyone does anything they ever do? That every action is totally unpredictable and after the fact that there is no way for us to offer more or less plausible explanations?

We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not eve…

I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior.

Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI.

It's not going beyond, it would be achieving parity.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#438
post #412

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

None of the world-leading scientists that co-authored this paper: https://managing-ai-risks.com/ Work for big labs? How is that you think the proposed conspiracy works exactly? They're paying off Yoshua Bengio? (disclaimer: I work on safety for a large lab. But none of these proponents do!)

And how many of them have been invited to the White House to share their concerns? https://www.whitehouse.gov/briefing-room/statements-releases... I don't see any of their names in this meeting's attendees, for example. Sam Altman's there, Satya Nadella's there, Sundar Pichai's there... No Yoshua Bengio, though.

What the person you replied to is saying is that commercial developers of AI have a significant financial incentive to be the ones defining what risky AI looks like.

If they can guide the regulatory process to only hinder development that they didn't want to do anyway (like develop AI powered nuclear missile launch systems) then it opens the gates for them to develop AI which could replace 27% of jobs currently done by people (https://www.reuters.com/technology/27-jobs-high-risk-ai-revo...) and become the richest enterprises in history.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#439

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

> If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. If we go extinct in the next 100 years, it's not going to be from climate change. How would that even work?

Agricultural pressure is one of the strongest predictors of unrest. Imagine that but not localized.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#440
post #325

Earlier quoted context omitted.

AI can make a lot of other risks worse. For example, say someone asks GPT5 (or decensored Llama-5) how to kill everyone and it patiently and expertly walks them through how to create a global pandemic.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

Which is very worrying! That's part of why I left big tech to work on detecting novel pandemics. [1] Luckily, the people capable of killing everyone seem not to have wanted to so far.

But if the bar keeps lowering, at some point it will be accessible to "humanity is a cancer" folks and I do think we'll see serious efforts to wipe out humanity.

[1] https://www.jefftk.com/p/computational-approaches-to-pathoge...

Post reply on HN