Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

261–270 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#261

Post by jeffparsons »

concordDance wrote: post #255
jeffparsons wrote: post #210

Earlier quoted context omitted.

It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless. Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabo…

That depends on making powerful AIs and being sure they are well aligned being as easy as making powerful AIs which look aligned to cursory inspection. The former seems much harder, though I'd say talk to Robert E. Miles for a more detailed look.

It depends on there being _some chance_ that it can be achieved.

The benchmark strategy we are comparing it against is tantamount to licking a wall repeatedly and expecting that to achieve something. So even a strategy that may well fail but has _some_ chance of success is a lot better by default.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#262

Post by RandomLensman »

concordDance wrote: post #250
RandomLensman wrote: post #207

Earlier quoted context omitted.

I think the core is not to agree with the existential risk scenario in the first place. Once you believe some malevolent god will appear and doom us all, anything can be justified (and people have done so in the past - no malevolent god appeared then, btw.).

What's your view on where the X risk idea breaks down? Is it that it's impossible to make something smarter than a human? Or that such a thing wouldn't have goals/plans that require resources us humans want/need? Or that such a thing wouldn't be particularly dangerously powerful?

It could break down on any of those, for example (with the first one not forever, but could take much longer than what is mainstream thinking now).

A lot of existential risk involves bold hypotheses about machines being able to "solve" a lot of things humans cannot, but we don't know how much is actually just solvable by superior intelligence vs. what just doesn't work in that way. Human collective intelligence failed on a lot of things so far. Even the basic idea of exponential scaling intelligence is a hypothesis which might not hold true.

Also, some existential risk ideas involve hypotheses around evolution and on how species dominate - again, might not be right.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#263

Post by jeffparsons »

johnthewise wrote: post #259
jeffparsons wrote: post #210

Earlier quoted context omitted.

It can't be the only card left to play. There are so many others. They may be weak, but they still have to be better than one that is guaranteed to be absolutely useless. Example of a weak, but at least _better_ option: convince _one_ government (e.g. USA) that it is in their interest to massively fund an effort to (1) develop an AI that is compliant to our wishes and can dominate all other AIs, and (2) actively sabo…

He is not really convinced you can align an AI that is intelligent past some threshold with aims of a single entity. His risk calculations derive primarily from the AI itself, not weaponization of AI by others. Rich Sutton seems to agree with this take and embraces our extinction: https://www.youtube.com/watch?v=NgHFMolXs3U

You don't have to be sure that it's possible, because the alternative we're comparing it to is absolutely useless. You only need to not be sure that it's impossible. Which is where most of us, him included, are today.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#264

Post by ben_w »

peterth3 wrote: post #124
jamilton wrote: post #115

Earlier quoted context omitted.

They're saying don't do licensing, just make it illegal.

Sorry, who’s “they” here? The incumbents OpenAI / DeepMind / Anthropic?

Yudkowsky and similar.

(I'm a lot more optimistic than such doomers, but I can at least see their position is coherent; LeCun has the opposite position, but unfortunately, while I want that position to be correct, I find his position to be the same kind of wishful thinking as a factory boss who thinks "common sense" beats health and safety regulations).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#265

Post by redox99 »

peterth3 wrote: post #119

Can we stop labeling prominent AI researchers as “AI Godfather”? It’s so silly and barely truthful. The concept of AI has been around since Turing and if anyone deserves a title like “Father of AI” it’s him. LeCun is Chief AI Scientist at Meta. They can just leave it at that.

LeCun is not called AI godfather because of his job at meta, but becase of pioneering CNNs in 1989.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#266

Post by RandomLensman »

brutusborn wrote: post #253
RandomLensman wrote: post #196

Earlier quoted context omitted.

Because everyone getting richer from removing uncertainty isn't a risk to start with.

What about a single actor getting rich and leaving everyone else’s certainty unchanged?

Only works with fairly limited riches, otherwise what is happening leaks out (kind of playing out that way already).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#267

Post by JoshTriplett »

peterth3 wrote: post #124
jamilton wrote: post #115

Earlier quoted context omitted.

They're saying don't do licensing, just make it illegal.

Sorry, who’s “they” here? The incumbents OpenAI / DeepMind / Anthropic?

No, the incumbents think it should continue. The xrisk folks are not the AI companies. The AI companies pay lip service to xrisk and try to misdirect towards regulation they think they can work with. Judging by this thread, the misdirection is working.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#268

Post by johnthewise »

jeffparsons wrote: post #229
YetAnotherNick wrote: post #222

Earlier quoted context omitted.

Global coordination using coercion has worked for nukes. It's very hard but not impossible to ban NVidia or any competitor and create spy network in all large companies and if any engineer is found to create AI, mark them as terrorist and give them POW treatment. And monitor chip fab like nuclear enrichment facility and launch war with country found to create chip fab, which I belive is hard to do it in complete secr…

Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough. One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a…

yeah, he proposed bombing rogue data centers by enforcing countries to be a viable option. he proposes this so we have a chance of solving alignment first, if that's even possible, not eliminating this risk once and for all. In the limit, if alignment is not solved, we are probably doomed.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#269

Post by ben_w »

raverbashing wrote: post #51
lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

Correct. The risk that's the real ploy is much higher and much more impactful than the actual risk of Rogue AI (as much as the risk might exist, though probably tiny)

"Rogue" is just a type of bug.

Buggy old-fashioned AI (i.e. a bunch of ifs and loops) has, more than once, almost started WW3.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#270

Post by RandomLensman »

ben_w wrote: post #257
RandomLensman wrote: post #140

Earlier quoted context omitted.

At least nuclear weapons pose an actual existential risk as opposed to AI - and even then we don't just go to war.

Nukes being a risk was suggested as a reason why the US was willing to invade Iraq for trying to get one but not North Korea for succeeding. It's almost certainly more complex of course, but the UK called it's arsenal a "deterrent" before I left, and I've heard the same for why China stopped in the hundreds of warheads.

Even if that were true for Iraq, which I doubt, it would have been the odd one out.

Btw., China is increasing its arsenal at the moment.

Post reply on HN