Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

511–520 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#511

Post by ethanbond »

ImPostingOnHN wrote: post #452
ethanbond wrote: post #343

Earlier quoted context omitted.

The nuclear war scenario, according to all known information about nuclear war policies, is not “chunks of the world becoming unlivable.” The principals in a nuclear conflict do not appear to even have a method to launch just a few nukes in response to a nuclear attacks: they will launch thousands of warheads at hundreds of cities.

I suppose thousands of warheads could be launched at hundreds of cities, and that could make chunks of the world uninhabitable and kill millions climate change will happen, and it will do that, left unchecked

I'm not arguing whatsoever against action on climate change, I'm just articulating actually how bad a nuclear exchange would be. It's far, far, far worse than most people imagine because they (understandably) couldn't fathom how monstrous the actual war plans were (and are, as far as anyone knows).

Daniel Ellsberg, the top nuclear war planner at RAND during the Cold War, claims that the Joint Chiefs gave an estimate to Kennedy in 1961 that they'd expect 600,000,000 deaths from the US's war plan alone.

That's 600 million people:

1. At 1960s levels of urban populations (especially in China, things have changed quite a lot -- and yes the plan was to attack China... every moderately large city in China, in fact!)

2. Using 1960s nuclear weapons

3. Not including deaths from the Russian response (and now Chinese), at the time estimated to be 50 - 90 million Americans

4. Before the global nuclear winter

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#512

Post by wcoenen »

lettergram wrote: post #436
wcoenen wrote: post #352

Earlier quoted context omitted.

Climate change and nuclear war are not orthogonal risks. Climate change leads to conflict. For example, the Syria drought of 2006-2010. More climate change leads to larger conflicts, and large conflicts can lead to nuclear exchanges. Think about what happens if India and Pakistan (both nuclear powers) get into a major conflict over water again.

India and Pakistan didn’t nuke each other last time(s)… even though they had nuclear weapons. The assumption we’d need to use nukes is insane. For the same cost (and energy requirements) for a set of nukes we can filter salt from ocean water or collect rain. I agree famine and drought can cause conflict. But we are no where need that. If you read the climate change predictions (from the UN), they actually suspect mor…

> India and Pakistan didn’t nuke each other last time(s)… even though they had nuclear weapons.

I was referring to the Indo-Pakistani war of 1947-1948 where the conflict focused on water rights. Nuclear weapons did not enter the picture until much later.

Earlier this year, those old tensions about water rights resurfaced:

https://www.usip.org/publications/2023/02/india-and-pakistan...

> For the same cost (and energy requirements) for a set of nukes we can filter salt from ocean water or collect rain.

The fight would be over the water flowing through the Indus, which is orders of magnitudes more than all Indian and Pakistani desalination projects combined.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#513

Post by hollerith »

edanm wrote: post #179
jrflowers wrote: post #109

Earlier quoted context omitted.

Out of curiosity can you point to anyone that makes a “doomer” argument that isn’t Eliezer Yudkowsky or restating his specific points? Refuting the cult of personality counterpoint with “no really this one guy can see the future” is

Sure. You can read the book "Superintelligence" by Nick Bostrom. You can also just ready things that Geoffrey Hinton has said, or Stuart Russel (both prominent AI researchers). Not everyone makes exactly the same argument, of course, and Eliezer Yudkowsky is both one of the first to make AI safety arguments, and also one of the biggest "Doomers". But at this point he's very far from the only one. (I happen to mostly…

It was plain to many people that AI research was dangerous before Eliezer started say so publicly in 2003. Mostly these people kept silent though and chose not to enter (or not to persist in) the AI field.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#514

Post by j45 »

JoshTriplett wrote: post #52
lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

Except while saying stop, and saying they will, none have stopped because apparently others won’t.

Since developing the danger first to help regulate out everyone who doesn’t cross the line first.. open source alternatives are increasingly important.

The regulations announced in the US in the past week effectively make it for the few and not the many.

There’s been a few rumblings that OpenAI has achieved AGI recently as well.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#515

Post by berniedurfee »

kylebenzle wrote: post #462
meowface wrote: post #447

Earlier quoted context omitted.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

I don’t think we’re close, not even within 50 years of real AGI. Which may or may not decide to wipe us out.

However, even given the current state of “AI”, I think there are countless dangers and risks.

The ability to fabricate audio and video that’s indistinguishable from the real thing can absolutely wreck havoc on society.

Even just the transition of spam and phishing from poorly worded blasts with grammatical errors to very eloquent and specific messages will exponentially increase the effectiveness of attacks.

LLM generated content which is a mix of fact and fantasy will soon far outweigh all the written and spoken content created by humans in all of history. That’s going to put humans in a place we’ve never been.

Current “AI” is a tool that can allow a layman to very quickly build all sorts of weapons to be used to meet their ends.

It’s scary because “AI” is a very powerful tool that individuals and states will weaponize and turn against their enemies.

It will be nice to finally have a good spellchecker though.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#516

Post by hackinthebochs »

kylebenzle wrote: post #462
meowface wrote: post #447

Earlier quoted context omitted.

He's actually suggested it may happen within 10 years. Which makes it hard to take him seriously in general. (I do personally take his overall concerns seriously, but I don't trust his timeline predictions at all, and many of his other statements.)

Can anyone enumerate what exactly your concerns are cause I don't get it. AI I'd a great spell checker, a REALLY good spell checker, and people are losing their minds over it, I don't get it.

To be clear, no one is afraid of these text-completion models. We're afraid of what might be around the corner. The issue is that small misalignments in an optimizer's objectives can have outsized real-world effects. General intelligence allows an optimizer to find efficient solutions to a wide range of computational problems, thus maximizing the utility of available computational resources. The rules constrain its behavior such that on net it ideally provides sufficient value to us above what it destroys (all action destroys value e.g. energy). But misalignment in objectives provides an avenue by which the AGI can on net destroy value despite our best efforts. Can we be sure we can provide loophole-free objectives that ensures only value-producing behavior from the human perspective? Can we prove that the ratio of value created to value lost due to misalignment is always above some suitable threshold? Can we prove that the range of value destruction is bounded so that if it does go off the rails, its damage is limited? Until we do, x-risk should be the default assumption.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#517

Post by ryanklee »

troupo wrote: post #486
RandomLensman wrote: post #450

Earlier quoted context omitted.

I disagree: for anything beyond basal stuff, we have hypotheses for causal explanations of human cognition, not full theories. For example, we cannot explain the mental process how someone came up with a new chess move and check the validity in that case. We can have some ideas how it might happen and that person might also have some ideas how, but then we back to hypotheses.

All this is demagoguery. If a bank denies you credit, it has to explain why. Not "AI told us so". If police arrests you, they have to explain why, not "AI told us so". If your job fires you, it has to explain why, not "AI told us so". etc.

I'm totally lost as to how any of that is relevant to anything I've said. The claim I am rebutting is that we can't expect to say anything about AI causality because we can't even say anything about human causality.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#518

Post by m3kw9 »

patcon wrote: post #384

I need to resist all this regulatory resistance here, spoken amongst tech people. No, Yann. I am FULLY in support of drastic measures to mitigate and control AI research for the time being. I have no vested stake in any of the companies. I lived for a year in a building that hosted several AI events per week. I'm not ignorant This is the only territory where humanity is mounting a conversation about a REAL response t…

These doomers saying there is risk of death is like saying there is risk of death if you are exposed to the outside world. Of course there is risk, but you still have to go out because of risk vs reward. Same with AI the risk is always death, but the rewards are way bigger. Everybody who has unnaturally low risk tolerance is showing up in these conversations.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#519

Post by ipaddr »

JoshTriplett wrote: post #497
itsoktocry wrote: post #423

Earlier quoted context omitted.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders". The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people w…

But someone could ask the AI how to stop it.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#520

Post by tppiotrowski »

wetmore wrote: post #465
tppiotrowski wrote: post #420

Earlier quoted context omitted.

The people who can do doomsday level harm need to complete a level of education that makes them smart enough and gives them time to consider the implications of their actions. They have a certain level of maturity before they can strike and this maturity makes them understand the gravity of their actions. Also, by this point they are probably set financially due to their education and not upset with the world (possib…

Counterpoint: the Unabomber

I don't know enough about the unabomber but from what I heard he wasn't trying doomsday stuff. Seemed like more targeted assassinations at certain individuals. Feel free to enlighten me though...
Post reply on HN