Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

571–580 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#571

Post by sva_ »

discmonkey wrote: post #485
veidr wrote: post #409

"AI Godfather" is... pretty fucking ludicrous. Prominent figure, notable researcher? Sure. But it's like calling Neil deGrasse Tyson (who I enjoy a lot; Astrophysics for People In A Hurry was great) the "Godfather of Physics". Both "there's not one" and "anyway, if there were, he wouldn't be it" apply.

Idk inventing convolutions and a bunch of other stuff we use everyday is somewhat significant. Machine learning, if we call it a field, is a whole lot newer than physics, and so a few people have contributed quite a bit in recent memory.

He didn't really invent them but he made them work and showed their feasibility in neural networks.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#572

Post by Chathamization »

hackinthebochs wrote: post #537
RandomLensman wrote: post #391

Earlier quoted context omitted.

No, I meant it as in: there is no reason to agree with that scenario. Stacking a lot of hypotheses on top of each other as to get to something dangerous isn't necessarily something that is convincing.

There's also no reason to agree that AI will be aligned and everything will be fine. The question is what should our default stance be until proven otherwise? I submit it is not to continue building the potentially world-ending technology. When you see a gun on the table, what do you do? You assume its loaded until proven otherwise. For some reason, those who imagine AI will usher in some tech-utopia not only assume…

Whenever I see people jump to alignment they invariably have jumped over the _much_more questionable assumption that AGI’s will be godlike. This doesn’t even match what we observe in reality - drop a human into the middle of a jungle, and it doesn’t simply become a god just because it has an intelligence that’s orders of magnitude greater than the animals around it. In fact, most people wouldn’t even survive.

Further, our success as a species doesn’t come from lone geniuses, but from large organizations that are able to harness the capabilities of thousands/millions of individual intelligences. Assuming an AGI that’s better than an individual human is going to automatically be better than millions of humans - and so much better that it’s godlike - is disconnected from what we see in reality.

It actually seems to be a reflection of the LessWrong crowd, who (in my experience) greatly overemphasize the role of lone geniuses and end up struggling when it comes social aspects of our society.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#573

Post by RandomLensman »

JoshTriplett wrote: post #523
RandomLensman wrote: post #401

Earlier quoted context omitted.

I find his arguments unconvincing. Humans can and have contained human intelligence, for example (look at the Khmer Rouge, for example). Also, right now there is nothing to contain. The idea of existential risk relies on a lot of stacked-up hypotheses that all could be false. I can "create" any hypothetical risk by using that technique.

> Humans can and have contained human intelligence ASI is not human intelligence. As a lower bound on what "superintelligence" means, consider something that 1) thinks much faster, years or centuries every second, and 2) thinks in parallel, as though millions of people are thinking centuries every second. That's not even accounting for getting qualitatively better at thinking , such as learning to reliably make the n…

Even for your last example, two hypotheses need to be true: (1) such information exists, and (2) the AI has access to such information/can generate it. EDIT: actually at least three: (3) the human and/or the AI can apply that information.

It also unclear to what extent thinking alone can solve a lot of problems. Similar, it is unclear if humans could not contain superhuman intelligence. Pretty unintelligent humans can contain very smart humans. Is there an upper limit on intelligence differential for containment?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#574

Post by RandomLensman »

hackinthebochs wrote: post #537
RandomLensman wrote: post #391

Earlier quoted context omitted.

No, I meant it as in: there is no reason to agree with that scenario. Stacking a lot of hypotheses on top of each other as to get to something dangerous isn't necessarily something that is convincing.

There's also no reason to agree that AI will be aligned and everything will be fine. The question is what should our default stance be until proven otherwise? I submit it is not to continue building the potentially world-ending technology. When you see a gun on the table, what do you do? You assume its loaded until proven otherwise. For some reason, those who imagine AI will usher in some tech-utopia not only assume…

Your existentially risky AI is imaginary, it might not exist. Who would check an imaginary gun on a table?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#575

Post by dragonwriter »

JoshTriplett wrote: post #52
lazzlazzlazz wrote: post #35

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

Yudkowsky’s call for declaring a global AI halt and engaging in unlimited warfare with anyone who defies it poses a greater and more immediate existential risk to humanity than AI ever could; it is largely mitigated (though still a bigger X-Risk than AI) because no one with the capacity to implement it in any substantive way takes him seriously at the moment.

But the people selling AI X-Risk that anyone with any policy influence is listening to are AI sellers using it to push regulatory capture, so those are the people being talked about when people are discussing X-Riskers as a substantive force in policy discussions, not the Yudkowsky cult, which, to the extent it is relevant, mostly provides some background fear noise which the regulatory capture crew leverages.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#576

Post by sterlind »

JoshTriplett wrote: post #497
itsoktocry wrote: post #423

Earlier quoted context omitted.

> how to kill everyone and it patiently and expertly walks them through how to create a global pandemic. There are thousands of people around the globe working in labs every day that can do this.

There is a massive difference between "there are a few thousand people in the world who might have the specialized knowledge to kill billions" and "anyone with a handful of GPUs or access to a handful of cloud systems could have a conversation with LatestAIUnlimited-900T and make a few specialized mail orders". The Venn diagram of "people who could arrange to kill most of humanity without being stopped" and "people w…

there's a much easier way to create a global pandemic: wait for the next Ebola outbreak, hop on a plane, get infected, fly to NYC and spread your bodily fluids everywhere. I was (pleasantly) surprised that ISIL didn't try this.

and making a pandemic virus is considerably more involved than making "a few specialized mail orders." and if it becomes easier in the future, far better to lock down the mail orders than the knowledge, no?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#577

Post by bumby »

edgyquant wrote: post #535
bumby wrote: post #494

Earlier quoted context omitted.

How does that number compare to the number of people (experts and laypeople) who have access to the mentioned AI models? Risk = severity x likelihood I think the OP's point was that AI increases the likelihood by dramatically increasing the access to that level of knowledge.

So it’s open and shared knowledge you think is bad? Good to know.

Please go review the HN guidelines: https://news.ycombinator.com/newsguidelines.html

And to address the snarky mischaracterization, like with most things in life "it depends." As a general rule, I'm in favor of democratizing most resources, to include information. But there are caveats. I don't think, for example, non-anonymized health information should be open knowledge. Also, from the simple risk equation above, as the severity of consequence or the likelihood of misuse go up, there should probably be additional mitigations in place if we care about managing risk.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#578

Post by ativzzz »

rpigab wrote: post #294
RandomLensman wrote: post #148

Earlier quoted context omitted.

If every investment became risk free because of perfect information about the future, it would likely increase the total market value by quite a lot.

Or decrease, because who would go to the casino if everyone knew the outcome? Only the owners and staff. "Useless" companies are part of the total market value. If they never get any funding, that's less value overall. Even if that translates to more value for "useful" companies. Also, if you know with absolute certainty from the beginning that Apple Inc is going to be worth X billion dollars, then you never get to b…

> because who would go to the casino if everyone knew the outcome

Everyone knows the outcome of casinos - the house wins in the long run. People still go because they think they have a shot of wining in the short run. People like gambling

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#579

Post by RandomLensman »

troupo wrote: post #526
RandomLensman wrote: post #433

Earlier quoted context omitted.

If the level of explanation is something that the person suffering from the decision need not necessarily understand, fair enough. But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention.

> But then is the limit that one human on earth can understand it? They way I understand the thrust of the regulation that isn't the intention. Yes, that is the intention when it comes to humans interacting with systems. https://news.ycombinator.com/edit?id=38113944

From my discussions with regulators, I would not necessarily subscribe to that view.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#580

Post by dragonwriter »

lettergram wrote: post #425
zimpenfish wrote: post #260

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.

zero chance climate change (1-2 degree increase) would end humanity. 5-10 probably wouldn’t end humanity lol Nuclear war would probably kill 80-90% but even then wouldn’t kill humanity. Projections I’ve seen are only like 40-50% of the countries hit. AI is scary if they tie it into everything and people slowly stop being capable. Then something happens, and we can’t boil water. Beyond that scenario I don’t see the ri…

> zero chance climate change (1-2 degree increase) would end humanity

“Humanity” is not the same thing as “human civilization”.

But, yes, its unlikely that it will be extinct in a century, even more unlikely that it will be from climate change.

...and yet climate change is still more likely to do it in that time than AI.

Post reply on HN