Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

821–830 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#821

Post by brutusborn »

RandomLensman wrote: post #624
brutusborn wrote: post #592

Earlier quoted context omitted.

How would it leak out?

Transactions and positions are not all necessary super secret. People and other machines notice things. You need to take a lot of care to be unnoticed unless tiny and once you move to much it is basically impossible (might not know who you are, but might notice your actions and the impact).

Having info on someone’s trades doesn’t mean you know the basis for their trades. And by the time they have executed the trade any leaked info becomes less valuable.

If that were the case then adversarial HFT algorithms wouldn’t work.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#822

Post by brutusborn »

Veedrac wrote: post #638
brutusborn wrote: post #83

Earlier quoted context omitted.

Who do you find persuasive and what material should I read/watch to understand their POV? So far I’ve read a bunch of lesswrong posts and listened to some Eliezer talks but still can’t understand the basis of their arguments or when I do, they seem very vague. The only suggestion that makes sense to me is from the FEP crowd. Essentially if someone sets up at AI with an autopoietic mechanism then it would be able to t…

> still can’t understand the basis of their arguments I've given what I consider the basic outline and best first introduction here. https://news.ycombinator.com/item?id=36124905 If you have a specific point of divergence, it would help to highlight it. > But there don’t seem to be any incentives for a big player to dedicate resources to [self-replication abilities], so it doesn’t seem very likely. If you have a gene…

I agree the potential exists, but what would incentivise someone to create a self-replicating system that is driven by its own self-interest and not the creators?

The only reason I can think of is a mad-science type who just wants to watch the world burn and hates humanity. Anyone with the sophistication to build a self-replicating system would build it such that it acts towards the goal of replication of its creator rather than itself. I.e. an AI that is self-interested is not useful for the creator and thus wouldn’t be built.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#823

Post by strken »

logicchains wrote: post #406
strken wrote: post #322

Earlier quoted context omitted.

Why would you need to predict the future to be convincing? Charismatic people aren't acting like oracles, they're just...being charming. They frame actions in ways that make them look good, do little favours for you, stroke your ego, and otherwise have a good grasp of human social dynamics. Sure, there's an aspect of understanding the other person, but chaos theory doesn't stop politicians from obtaining and keeping…

>Why would you need to predict the future to be convincing? You need to be able to predict how people will react to your words.

Sure, but that's not predicting the future to a high degree of certainty. People predict how others will react all the time. The most convincing humans are already very good at it, so we know it's possible to some degree. While I agree chaos theory puts an upper limit on how well someone can predict the future, there's no particular evidence that human ability is close to the limit.

I think you're imagining some mastermind that has to get everything right or it dies, instead of a master politician or businessperson who's a few percentage points better than the existing competition and for whom the advantage compounds. Yes, Less Wrong is way too hysterical about everything, but charisma that's better than human is a problem. If you're convincing enough that all your business deals, legal arguments, political and ad campaigns, lobbying attempts, etc. are slightly better than the competition, then money "magically" flows your way.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#824

Post by wseqyrku »

upupupandaway wrote: post #182
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Eliezer Yudkowsky sounds a lot like Neil deGrasse Tyson of computer science.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#825

Post by M95D »

mangosteenjuice wrote: post #342
toth wrote: post #337

Earlier quoted context omitted.

You are right, but "human civilization will be extinct" is an awkward phrase. Human extinction via climate change is off the table. The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's t…

Right, the collapse of our current civilization is inevitable. That doesn't mean it won't be succeeded by more advanced ones in the future

Once it collapses, it won't rise again to even an industrial level civilization. Iron ore, copper, coal are no longer easy-accessible. Can't have a new industrial age without those. We probably need a Foundation (Asimov).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#826

Post by runeks »

dclowd9901 wrote: post #609
aldousd666 wrote: post #393

So if there is some fringe element who's willing to use AI as a terrorist implement, they'll have very little competition and they'll be very little knowledge about how to counter them if we try to outlaw open source models. I don't really believe that we can make this tool any worse than any other tool in history that's capable of killing people in some way if it's misused, but simply driving it underground won't ma…

Fissile material is tightly controlled and regulated with a good deal of success, I might add. You might not like what it would take to prevent AI from being developed, but it can be stopped.

Fissile material is not information. Controlling information is nothing like controlling the sale and distribution of toxic substances.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#827

Post by maeil »

topspin wrote: post #171
maeil wrote: post #156

Earlier quoted context omitted.

If tomorrow TSMC, NVIDIA and ASML get destroyed in the next 9/11, it's "stopped". It's completely possible.

Fantasies, bordering on threats.

I'm not suggesting to do that, I'm just showing that it's incredibly possible. You could also go with "Tomorrow a massive storm strikes the Netherlands and huge earthquakes striking Taiwan and the US Westcoast", if you have emotional objections to my first example.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#828

Post by maeil »

YetAnotherNick wrote: post #216
maeil wrote: post #156

Earlier quoted context omitted.

If tomorrow TSMC, NVIDIA and ASML get destroyed in the next 9/11, it's "stopped". It's completely possible.

I think current hardware already produced till now is either enough to get us to AGI, or it is very far away. I don't believe AGI is 30 years away. It is either very near or more than 1000s of years away.

The sum of all hardware produced, sure. But the amount of hardware a single player currently posseses? Doubtful.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#829

Post by maeil »

Maken wrote: post #167
maeil wrote: post #156

Earlier quoted context omitted.

If tomorrow TSMC, NVIDIA and ASML get destroyed in the next 9/11, it's "stopped". It's completely possible.

It's delayed at best.

Incredibly delayed, giving at least decades to figure things out.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#830

Post by bondarchuk »

dghlsakjg wrote: post #815
bondarchuk wrote: post #662

Earlier quoted context omitted.

I addressed science fiction in my comment. The crucial difference is taking it seriously and actually trying to do something tangible about slowing/stopping AI progress, which is not something Stanley Kubrick did.

You edited the comment after I posted. Your original made no mention of sci-fi, and made it sound as if no one had imagined a malicious AI before yudkowski

I did not edit my comment.

(edit: you can see that https://news.ycombinator.com/item?id=38116456 responds to the scifi part of my comment as well and was posted before yours https://news.ycombinator.com/item?id=38116494 (if my assumption that ids are chronological is correct))

Post reply on HN