Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

Post reply on HN
Print view View original

621–630 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#621

Post by johnthewise »

hollerith wrote: post #542
m_a_g wrote: post #475

Earlier quoted context omitted.

That I understand, but my thinking automatically goes to control and restrictions. A superintelligent AGI that's as restricted as chatgpt in terms of what it can actually control seems... not so existentially dangerous? In that scenario, that actual problem would rise by not-so-smart humans listening to that AGI blindly, no?

I'm confused by your "as restricted as chatgpt". Hasn't chatgpt been given unrestricted access to the web? Surely if chatgpt were sufficiently capable, it would be able to use that access to hack web sites and thereby gain control over money. Then it can use the web to buy things with that money just like you and I can. It is a lot harder to prevent the sysadmins who have temporary control over you from noticing that…

It doesn't even need to hack or do anything illegal. It can make money trading stocks, predicting or modelling far better than market participants, use money to influence the real world, bribe politicians etc. Like with few billions, i bet you can create a political party, influence campagins and end up with real human followers and lobbying power.

It doesn't even need to run on its own, if there was a trading bot that said, in order for it to make me money, I need to give it internet access and run it 24/7, rent gpu clusters and s3 buckets etc, I'd do it. this is the most probable scenario imo, AI creating beneficial scenarios for subset of people so that we comply with its requests. very little convincing is necessary

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#622

Post by randomdata »

peterth3 wrote: post #605
randomdata wrote: post #559

Earlier quoted context omitted.

According to the dictionary, godfather is defined as: "a man who is influential or pioneering in a movement or organization" Are these prominent AI researches not seen as being prominent exactly because they influenced movement in relation to AI?

That’s definition #2 with #1 being: > a man who presents a child at baptism and promises to take responsibility for their religious education. So, the original definition is a gendered and religious term. Sounds silly to me. Especially in this context.

Clearly #1 has no applicability here and the intent is to convey #2, which is quite apt. Words have multiple meanings. Such is life.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#623

Post by jwozn »

danaris wrote: post #459
lampington wrote: post #319

Earlier quoted context omitted.

> If they are Am I the only one slightly disturbed by the use of "they" in that sentence? I know that the HN commentariat is broad-based, but I didn't realise we already had non-human members ;-)

It's perfectly reasonable to refer to "human civilizations" as a plural, since that phrase can very commonly be used to refer to different cultures.

I believe they are referring to being disturbed by using "they" vs "us", implying they are not a part of human civilization.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#624

Post by RandomLensman »

brutusborn wrote: post #592
RandomLensman wrote: post #266

Earlier quoted context omitted.

Only works with fairly limited riches, otherwise what is happening leaks out (kind of playing out that way already).

How would it leak out?

Transactions and positions are not all necessary super secret. People and other machines notice things. You need to take a lot of care to be unnoticed unless tiny and once you move to much it is basically impossible (might not know who you are, but might notice your actions and the impact).

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#625

Post by aeternum »

thomasahle wrote: post #72
aeternum wrote: post #63

Earlier quoted context omitted.

Arguably, this has already happened at least twice in the past. One with the discovery of the Black-Scholes model, and arguably again with the advent of HFT (and the ability to basically front-run trades). The interesting thing about markets is you generally can only make money when things are mispriced. This limits the total potential gain even for actors with perfect information. Suppose we did have an AI model tha…

> The interesting thing about markets is you generally can only make money when things are mispriced. Why would you think things are not terribly mispriced right now? If only we were a lot smarter we'd know how.

Yes, things might be significantly mispriced, but the amount of mispricing is a finite quantity.

This is clear in the classic pump and dump scheme. During the pumping stage, the unscrupulous actor loses money by injecting some amount of mispricing by purchasing the security and bidding up its value. The hope is to generate momentum and hype that triggers others to amplify that mispricing. Then during the dump scheme, the unscrupulous actor can capitalize by removing the mispricing.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#626

Post by RandomLensman »

aeternum wrote: post #617
RandomLensman wrote: post #345

Earlier quoted context omitted.

You could calculate the value of things under the discounting changes. The idea of perfect foresight of the future is kind of insane anyway. Not sure why all of sudden we would go back to believing in a deterministic universe.

The universe can be both deterministic and mostly unpredictable. In fact it most likely is both of those things. Entropy bounds computation and prediction of chaotic systems requires extreme amounts of computation.

Our current understanding puts some things into the category random, could be wrong, of course.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#627

Post by randomdata »

danenania wrote: post #614
randomdata wrote: post #570

Earlier quoted context omitted.

> but at least some of the low level people working for OpenAI seemed to think the risk is worth taking seriously Everyone has an opinion, but the experts seem strangely silent on this matter. I mean experts in human risk, not experts in AI. The latter love to ramble on, but are unlikely to know anything about social and humanitarian consequences – at least no more than the stock boy at Walmart. Dedicating one's life…

Conversely, why should we expect an expert in "human risk" with no understanding of AI to have anything productive to add to the conversation? We should be looking for people who are well-grounded in both areas, if any exist.

What additional information would understanding AI actually bring to the table? Worst case you just assume the AI can become human-like.

Certainly all the vocal AI experts can come up with is things that humans already do to each other already, only noting that AI enables it at larger scale. Clearly there is no benefit in understanding the AI technology with respect to this matter when the only thing you can point to is scale. The concept of scalability doesn't require an understanding of AI.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#628

Post by bondarchuk »

upupupandaway wrote: post #182
JoshTriplett wrote: post #52

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

If your main critique of EY is the unfeasibility of his proposed solutions then you might still take him seriously for his identification of the problems.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#629

Post by hodgesrm »

cjbprime wrote: post #600
hodgesrm wrote: post #521

Earlier quoted context omitted.

> He's pretty sure that human civilization will be extinct this century. This is certainly possible, but if so extinction seems far more likely to come from nuclear warfare and/or disasters resulting from climate change. For example, it's become apparent that our food supply more brittle than many people. Losing access to fertilizers, running out of water, or disruptions in distribution netorks can lead to mass starv…

(Neither nuclear war nor climate change are considered as this-century extinction risks by the singularity-minded side of the existential risk community, just because there would necessarily be pockets of unaffected people.)

I'm always a bit surprised at how people seem to wish away the destructiveness of nuclear weapons or the likelihood of their use. Another commenter alluded to a "failure of imagination" which seems very applicable here. You would think just watching videos of above-ground tests would cure that but apparently not. [0]

[0] https://www.youtube.com/watch?v=gZuyPn0_KRE

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#630

Post by TeMPOraL »

bnralt wrote: post #561
upupupandaway wrote: post #182

Earlier quoted context omitted.

The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…

Every time I see Yudkowsky’s writing I wonder why people take him seriously. Just a couple of months back he was duped by a fake story about an airforce AI deciding to kill it’s operators in a simulation. It turned out the whole thing was thought experiment. Asking Yudkowsky what we should do about AI is like asking Green Peace what we should do about nuclear power.

If I had a cent for every time I see a HN semi-regular (i.e. not a throwaway account) falling for some rather transparent bullshit, I'd enjoy my retirement on a significant "FU money" cache... Everyone gets duped sometimes, often publicly.

I remember skimming a summary of the story itself. Sounded like textbook failure mode of reinforcement learning. I was actually saddened to later learn it apparently was just a thought experiment, and not a simulated exercise - for a moment I hoped for a more visceral anecdote about a ML model finding an unusual solution after being trained too hard, rather than my go-to "AI beating a game in record time by finding and exploiting timing bugs" kind of stories.

Post reply on HN