Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario • Hacker News Board

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#441

Post by nerdponx » Thu, Nov 02, 2023, 1:25 PM UTC

concordDance wrote: post #213

lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. How sure are you that they don't think the risk is real? I have only third hand accounts, but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously, its not…

The best lies contain a kernel of truth.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#442

Post by samus » Thu, Nov 02, 2023, 1:26 PM UTC

Geee wrote: post #108

samus wrote: post #101

Earlier quoted context omitted.

> I don't see bots as a huge problem, because it's somewhat easy to verify that someone is a real person, and it would be quite obvious if someone is using lots of bots for their cause. That only works on platforms designed to match user profiles with real-world persons. This platform definitely isn't.

You can read their profile or read their comment history to verify their humanness. Faking everything wouldn't be impossible, but certainly difficult & take time.

Falling profiles on scale might be hard, but so is verifying wherever they are real. If you have only even 20 commenters replying to one of your comments, then you're gonna spend the rest of the hour digging through them to figure out which are bots and which aren't.

The comment history is not visible on every relevant platform, and there are good reasons for people to disable it where possible. Furthermore, many people keep their profiles minimal out of principle. Hard to distinguish those from bots.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#443

Post by digdugdirk » Thu, Nov 02, 2023, 1:26 PM UTC

lettergram wrote: post #425

zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

zero chance climate change (1-2 degree increase) would end humanity. 5-10 probably wouldn’t end humanity lol Nuclear war would probably kill 80-90% but even then wouldn’t kill humanity. Projections I’ve seen are only like 40-50% of the countries hit. AI is scary if they tie it into everything and people slowly stop being capable. Then something happens, and we can’t boil water. Beyond that scenario I don’t see the ri…

That's an incredibly confident statement that very much misses the point. 50% loss of the global human population means we don't come back to the current level of technological development. 80-90% means we go back to the stone age.

Photosynthesis stops working at temperatures very close to our current high temps. Pollen dies at temperatures well within those limits - we're just lucky we haven't had massive heatwaves at those times in the year.

People need to understand that climate change isn't a uniform bumping up of the thermostat - it's extra thermal energy that exists in a closes system. That extra energy does what it wants, when it wants, in ways we can't currently accurately predict. It could be heatwaves that kill off massive swaths of global food production. Or hurricanes that can spin up and hit any coastline at any time of year. Or a polar vortex that gets knocked off the pole and sits over an area of the globe where the plant and animal life has zero ability to survive the frigid temperatures.

It's not a matter of getting to wear shorts more often. We're actively playing russian roulette with the future of humanity.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#444

Post by The_Colonel » Thu, Nov 02, 2023, 1:26 PM UTC

api wrote: post #349

jacquesm wrote: post #339

Earlier quoted context omitted.

There is no symmetry between destruction and creation/prevention of destruction, destruction is far easier.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

Another thing which is asymmetrical is the level of control given to AI. The good actors are likely going to be very careful about what AI can do and can't do. The bad guys don't have much to lose and allow their AIs to do anything. That will significantly cripple the good guys.

As an example, the good guys will always require to have human in the loop in the weapon systems, but that will increase latency at minimum. The bad guy weapons will be completely AI controlled, having an edge (or at least equalizing) over the good guys.

> The number of people who want to do doomsday levels of harm is small

And that's a big limiting factor in what the bad actors can do today. AI to a large degree removes this scaling limitation since one bad person with some resources can scale "evil AI" almost without limit.

"Hey AI, could you create me a design for a simple self-replicating robot I can drop into the ocean and step-by-step instructions on what you need to bootstrap the first one? Also, figure out what would be the easiest produced poison which would kill all life in the oceans. It should start with that after reach 50th generation."

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#445

Post by concordDance » Thu, Nov 02, 2023, 1:27 PM UTC

logicchains wrote: post #402

concordDance wrote: post #221

Earlier quoted context omitted.

I'm confused. We know persuasive humans exist, we employ them as politicians or sales people. There are also many unpersuasive humans. Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human? Though even if that WAS the limit I'd still be somewhat worried due to the possibility of that persuasiveness…

>Given we seem to have a decent range of persuasiveness even amongst these very very similar minds, why do you think the upper limit for persuasiveness is a charismatic human Because there's a hard limit on how much people can be made to act against their own self interest.

Is there really? Some religious orgs have gotten quite good at getting people to blow themselves up for the promise of 77 virgins after death.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#446

Post by hollerith » Thu, Nov 02, 2023, 1:30 PM UTC

upupupandaway wrote: post #182

JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Yudkowsky does not expect a stop to the development of AI globally. He, and thousands of others (including me) are merely saying that if there is not a stop, then we are all dead.

Yudkowsky thinks we are all dead with probability .999. I don't know the subject nearly as well as he does, which makes it impossible for me to have much certainty, so my probability is only .9.

Also, it is only the development of ever-more-capable AIs that is the danger. If our society wants to integrate language models approximately as capable as GPT-4 thoroughly into the economy, there is very little extinction danger in that (because if GPT-4 were capable enough to be dangerous, we'd already all be dead by now).

Also, similar to how even the creators of GPT-4 failed to predict that GPT-4 would turn out capable enough to score in the 92th percentile of the Bar Exam, almost none of the people who think we are all dead claim to know when exactly we are all dead except to say that it will probably be some time in the next 30 years or so.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#447

Post by meowface » Thu, Nov 02, 2023, 1:33 PM UTC

concordDance wrote: post #199

jeffparsons wrote: post #194

Earlier quoted context omitted.

My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.

He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general.

(I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#448

Post by btbuildem » Thu, Nov 02, 2023, 1:35 PM UTC

patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

On one hand, we have some pie-in-the-sky, pop-culture-inspired "risk".

On the other, we have the ugliness of human nature, demonstrated repeatedly countless times over millennia.

These are people with immense resources, hell-bent on acquiring even more riches and power. Like always in history, every single time. They're manipulating the discourse to distract from the real present danger (them) and shift the focus to some imaginary danger (oh no, the Terminator is coming).

The Terminators of most works of fiction were cooked up in govt / big corp labs, precisely by the likes of Altman and co. It's _always_ some billionaire villain or faceless org that the scrappy rebels are fighting against.

You want to protect the future of humanity from AI? Focus on the human incumbents, they're the bad actors in this scenario. They've always been.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#449

Post by ImPostingOnHN » Thu, Nov 02, 2023, 1:35 PM UTC

toth wrote: post #426

wcoenen wrote: post #352

Earlier quoted context omitted.

Climate change and nuclear war are not orthogonal risks. Climate change leads to conflict. For example, the Syria drought of 2006-2010. More climate change leads to larger conflicts, and large conflicts can lead to nuclear exchanges. Think about what happens if India and Pakistan (both nuclear powers) get into a major conflict over water again.

I don't disagree with that, but then the risk is nuclear war. It could be triggered by global warming conflicts or by something else. Still, like I said somewhere else, even all out nuclear war is unlikely to lead to human extinction . It could push us back to the neolithic in some scenarios, but even then there is some disagreement about what would happen. Of course, even in the most optimistic scenario it would be…

Then, actually, the risk is the root cause: climate change, not nuclear war. The global war could be conventional, too. Or chemical, or biological, or genetically engineered. No matter the tool used, the risk is why they would use it vs. not using it now.

In any case, besides addressing the root cause vs. proximal cause, you couldn't even address the proximal cause anyways: it's more likely that the world could do something about climate change than about war in general.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#450

Post by RandomLensman » Thu, Nov 02, 2023, 1:36 PM UTC

ryanklee wrote: post #437

RandomLensman wrote: post #427

Earlier quoted context omitted.

We are getting off track in our discussion. For non-trivial decisions we know we can have ex-post justifications from humans but at the same time we know that those aren't necessarily complete or even true at times: A credit officer not liking someone will have a justification for refusal that does not include that and we might never know - from the outside - the true explanation and that credit officer might not eve…

I don't think we're off track at all. You made the claim that human cognition is a black box. That's not true. We have valid causal explanations for human cognition /and/ human behavior. Just because we don't have explanations at every level of abstraction does not prevent us from having them at some levels of abstraction. We very well may find ourselves in the same situation with regard to AI. It's not going beyond,…

I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories.

For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.