Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

781–790 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#781

Post by nineteen999 »

TeMPOraL wrote: post #775
nineteen999 wrote: post #774

Earlier quoted context omitted.

The joke will be on them when our open-source powered mechwarriors made of bulldozer parts come for them.

The joke will be on us all when your open-source powered mechwarriors start destroying dams and power plants and even take a stab at launching some nukes, because the models figured this is the most effective way to "get them", and weren't aligned enough to see why they should take a different approach.

While my tongue was firmly in my cheek, and still is somewhat, whats to stop (for example) Boston Dynamics and OpenAI collaborating and beating us to your nightmare scenario first? And is that a situation you would prefer?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#782

Post by OkayPhysicist »

TeMPOraL wrote: post #386
api wrote: post #349

Earlier quoted context omitted.

The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…

> The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. You're forgetting about governments and militaries of major powers. It's not that they want to burn the world for no reason - but they still end up seeking capability to do doomsday levels of harm, by continuously seeking to have margin…

We avoided having them see the light of day because MAD is the single most effective peacekeeping tool ever developed. The fact that both sides had doomsday devices all but guaranteed that they would never be used. People can be evil, but they're selfishly evil.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#783

Post by OkayPhysicist »

ethanbond wrote: post #511
ImPostingOnHN wrote: post #452

Earlier quoted context omitted.

I suppose thousands of warheads could be launched at hundreds of cities, and that could make chunks of the world uninhabitable and kill millions climate change will happen, and it will do that, left unchecked

I'm not arguing whatsoever against action on climate change, I'm just articulating actually how bad a nuclear exchange would be. It's far, far, far worse than most people imagine because they (understandably) couldn't fathom how monstrous the actual war plans were (and are , as far as anyone knows). Daniel Ellsberg, the top nuclear war planner at RAND during the Cold War, claims that the Joint Chiefs gave an estimate…

That's not extinction, that's not even close. Maybe, just maybe, it'd be the end of both western and eastern civilization, but it's nowhere near wiping out all life on Earth.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#784

Post by jamilton »

peterth3 wrote: post #124
jamilton wrote: post #115

Earlier quoted context omitted.

They're saying don't do licensing, just make it illegal.

Sorry, who’s “they” here? The incumbents OpenAI / DeepMind / Anthropic?

In this case I meant the GP comment from JoshTriplett.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#785

Post by pixl97 »

toth wrote: post #773
pixl97 wrote: post #737

Earlier quoted context omitted.

What is the difference between a weather model, a nuclear bomb simulation, and training a LLM? At the end of the day all these are just calculations, of which you need thousands of processors to do with high accuracy. Pretty much what you're suggesting is a global police force to ensure we're performing the 'correct calculations'. Having to form an authoritarian world computer police isn't much of a solution to the p…

If we had to have global policing of every large computing cluster and everything that runs there, that does not feel to me like a huge overreach or something that could easily slide into totalitarianism. It would be better if we didn't need to do this, but I don't see a less intrusive way. Of course, if you don't think there is a real existential threat then it's pointless and bad to do this, it all depends on that…

Intelligence is a real potential existential threat, just ask every large mammal as of about 10k years ago, save those in Africa.

The question is, is an authoritarian nightmare that much better than getting tossed into the meat grinder?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#786

Post by pixl97 »

RandomLensman wrote: post #751
pixl97 wrote: post #735

Earlier quoted context omitted.

We don't have an answer yet to what the form factor of AGI/ASI will be, but if it's anything like current trends the idea of 'killed' is laughable. You can be killed because your storage medium and execution medium are inseparable. Destroy the brain and you're gone, and you don't even get to copy yourself. With AGI/ASI if we can boot it up from any copy on a disk if we have the right hardware then at the end of the d…

No power and it its already dead. Destroy copies, dead. Really not complicated at all.

You talk with these interesting certainties. Like if a human dies and we have their body and DNA we can be pretty assured they are dead.

With something like an AI you can never be sure as you must follow the entire chain of thermodynamic evidence of the past to when the AI was created that no copy of it was ever made. Not just by you, by any potential intruder in the system.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#787

Post by OkayPhysicist »

bbor wrote: post #690
l33tman wrote: post #400

Earlier quoted context omitted.

The issue is not that caution is warranted or not, it's that it's cynically used by Sam Altman to increase the value of his company. I don't trust a silicon valley rich guy with this more than I trust anybody else. Why should he sit and decide what the rest of us can't do, while he's getting richer? That is what the article is about..

Getting angry about one guy getting rich is silly in comparison to the serious threats presented by sudden AGI. Did all the other guys in Silicon Valley (or Wall Street, or Monaco, or Dubai) get rich in a justifiable fair way? I agree that big companies/capitalists using their power to suppress the rest of us sucks, but that’s a systemic political battle that should be considered separately here.

It's a systemic political battle, and the only reason anyone's talking about banning AI. They're only luddites when it benefits them, personally, in the here and now. Parroting their silly fearmongering tactics "What if THE MATRIX, guys?!" is just giving them more power.

Ludditry is just cowardice, specifically focused on the fear of change. Want to know a realistic worse case scenario for AI development? The same rich assholes that are rich today invest in developing AI tech to the point that the market value for human labor plummets to an unlivable point, and class mobility subsequently goes to zero. THAT's a doomsday scenario, and it's made ever more likely by stupid claims that only our Silicon Valley Immortal Philosopher-Kings are worthy of control over AI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#788

Post by ahamm »

hm-nah wrote: post #727
ahamm wrote: post #492

Earlier quoted context omitted.

I used to think that this fear was driven by rational minds until I read Michael Lewis' "Going Infinite" and learned more about effective altruism [EA]. Michael Lewis writes that what is remarkable about these guys [i.e. EAs] is that they're willing to follow their beliefs to their logical conclusion (paraphrasing) without regard to social cost/consequences or just inconvenience of it all. In fact, in my estimation t…

My comment above, which I think you’re replying to, is not based in fear. It’s based on first-hand experience experimenting with the likes of AutoExpert, AutoGen and ChatDev. The tools underlying these projects are quite close to doing, in a very short amount of time and cost, what it takes human knowledge workers a long time (and hence cost) to do. I think it’s as close as Summer 24’ that we get simplified GenAI gro…

I was responding to the initial question actually (https://news.ycombinator.com/item?id=38113521). I appreciate the insight though - looking forward to checking out the Gibson book.

Still, I'm not sure that I see the 'AI jumping out of the box scenario' any time soon (or ever). Chat apps finding missile codes, or convincing your smart toaster to jump in your bubble bath while you're inside seem only as real as a man in a box with intentions is (and unfortunately humans can kill from inside and out of boxes).

I'm definitely concerned about the implications to social welfare, the social bias involved in systems that make big decisions about an individuals freedom, etc, but these leaps from super impressive automated text analysis to humanity doomsday scenario seem like fear mongering to me mostly because these are scenarios that already exist today (along with massive problems in general social welfare).

The scenarios that don't exist (like Nick Bostrom's "An objective function that turns the world into a paper clip factory in "Superintelligence: Paths, Dangers, Strategies") strike me as easy to fix by just standing by the power outlet so things don't get out of hand. There are a lot of risks that don't exist yet. Alien contact is one of them - never happened, but it could happen, and if it does it could wipe us out - so be afraid and do something noble for the cause. This to me feels like a very rational response to what is essentially a 'magical' prior. We're scared now because we 'feel' close to General AI, but we really have no way of quantifying how close we are to it, and how dangerous (if at all) it would actually be. I'm definitely open to being wrong, but its hard not to agree with LeCun on a level.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#789

Post by hanspeter »

SilasX wrote: post #767
hanspeter wrote: post #732

Earlier quoted context omitted.

It will not. The difference is that reducing emissions just 10% is still better than 0% while stopping 90% from doing AI advancements is not better than 0% (it might actually be worse).

That's a different argument than the parent, who claimed that global restriction isn't even possible.

I don't see that being different.

I would also claim that global restriction is not possible and even in the unlikely scenario that we're able to restrict 90% of world, we're still not better off than if we did nothing.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#790

Post by upbeat_general »

tome wrote: post #212
Palmik wrote: post #79

Earlier quoted context omitted.

The gap between the value added by GPT 3.5 and GPT 4 to my every day tasks is immense. Most people, outside of these circles, don't realize that such a large gap exists.

Could I ask what exactly you get from 4 that you don't get from 3.5?

Personally it’s not one specific function but the answer I’m looking for [e.g, code that just works] is significantly better from GPT-4 imo. The better the output is, the less time I spend and the more likely I am to default to using the model in the first place.
Post reply on HN