Live data from Hacker News

Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

businessinsider.com

141–150 of 873 posts

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#141
post #20

The moat he describes the one-percenters might aquire is government regulation. Creating an environment where only an exclusive club of companies is able to work on AI. Is that possible? Has that happened in any other industry? I can imagine it happening in a single country. But wouldn't research route around that country then? How is the situation in Europe, China, India, Canada, Japan ... ?

>Is that possible? Has that happened in any other industry? It's absolutely happened to the nuclear weapons industry. And Big AI is lobbying for the government to treat AI development the same way it treats nuclear proliferation.

There are more nuclear warheads in Russia than in the USA.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#142

The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

> "Stop" is not regulatory capture.

"Stop" is a fiction that exists exclusively inside your head. Unless your solution anticipates a planet wide GULAG system run by a global government your "stop" is infeasible.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#143

Earlier quoted context omitted.

> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…

You can’t _stop_ it. You can make it illegal in some countries. But outlawing it just means some obvious candidate countries or private actors develop it first instead.

I think these attempts will simply point out that math is a higher power than law. If something becomes possible, and becomes easy, there is nothing law can do to stop it.

Just look at the attempts to stop drugs in North West Europe. Huge sentences (so huge they are in fact never fully carried out), tens of dead police officers on a yearly basis, triple that in civilian victims and drugs are easier to get than ever before.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#144
Its a delightful turn of events. LeCun still works for Meta so his public stance is pressumably sanctioned as congruent with corporate objectives.

The precise internal calculus does not immediately matter [1], the outcome is having a moneyed entity that is de-facto more qualified to opine on this debate than practically anybody on the planet (except maybe Google) arguing for open source "AI".

It makes perfect sense. Meta knows about the real as opposed to made-up risks from algorithms used directly (without person-in-middle or user awareness) to decide and affect human behavior. They know them well because they have done it. At scale, with not much regulation etc.

The risks from massive algorithmic intermediation of information flow are real and will only grow bigger with time. The way to mitigate them is not granting institutional rights to a new "trusted" AI feudarchy. Diffusing the means of production but also the means of protection is the democratic and human-centric path.

[1] In the longer term, though, relying on fickle oligopolistic positioning which may change with arbitrary bilateral deals and "understandings" (like the Google-Apple deal) for such a vital aspect of digital society design will not work. Both non-profit initiatives such as mozilla.ai and direct public sector involvement (producing public domain models etc) will be needed to create a tangibly alternative reality on the ground.

Remember the actual theory of how markets and capitalism are supposed to work is that the collective sets the rules. Fullstop. All these players require license to operate. There is more than enough room for private profit in the new "AI" landscape, but you dont start by anointing permanent landlords.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#145
post #77

Earlier quoted context omitted.

And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

And there it is.

The full agenda.

Blood and empire.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#146

Earlier quoted context omitted.

Arguably, this has already happened at least twice in the past. One with the discovery of the Black-Scholes model, and arguably again with the advent of HFT (and the ability to basically front-run trades). The interesting thing about markets is you generally can only make money when things are mispriced. This limits the total potential gain even for actors with perfect information. Suppose we did have an AI model tha…

It sounds like you also need an AI to generate that mispricing. Like a form of SEO, the misinformation bots tweet to all the trading bots and try to trick them into mispricing shares, so your trading bots can quickly gain advantage. Corewars using the world economy.

The best way to predict the future is to invent it. --Alan Kay

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#147
post #77

Earlier quoted context omitted.

And... How do you do that? Seriously, people have been talking about stoping CO2 emissions for what, 50 years? We've barely made progress there.

With threat of military intervention. People have been talking about curtailing carbon emissions for 50 years, but they haven't been serious about it. Being serious doesn't look like saying "oh shoot, we said we will... try to keep emissions down, but we didn't... even try; sorry...". Being serous looks like "stop now, or we will eminent domain your data centre from under you via an emergency court order, or failing…

But that's just not going to work in the real world is it?

If a country uses military force in another country, that's a declaration of war. We'll never convince every single country from banning AI research. And even if we do, you don't need much resources to do AI research. A few people and a few computers is enough.

This is not something like uranium refining.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#148

People focus on LLM's and diffusion models because they are so omnipresent now. But an AI for stock picking and prediction that would be realy next level and outcompete the current ones cosistently would siphon of so much wealth it would basically own society if the operators were clever enough not to get so gready short term that the system would litterally collapse overnight.

Arguably, this has already happened at least twice in the past. One with the discovery of the Black-Scholes model, and arguably again with the advent of HFT (and the ability to basically front-run trades). The interesting thing about markets is you generally can only make money when things are mispriced. This limits the total potential gain even for actors with perfect information. Suppose we did have an AI model tha…

If every investment became risk free because of perfect information about the future, it would likely increase the total market value by quite a lot.

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#149
> Every new technology is developed and deployed the same way:

> You make a prototype, try it at a small scale, make limited deployment, fix the problems, make it safer, and then deploy it more widely.

This makes an assumption: that the problematic technology is an intentional development rather than an emergent feature of an intentionally developed technology.

We already have a name for such emergent features: "bugs". No one really deployed Heartbleed, especially not in a limited deployment. Spectre? Rowhammer? And we all had/have to deal with the fallout, and we're not even done.

Who says that the danger of the technology cannot stay hidden until it's universally deployed?

Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario

#150

Earlier quoted context omitted.

A weak dismissal for a weak argument. The strong dismissal is that many processes, even relatively simple ones, are formally chaotic in the https://en.wikipedia.org/wiki/Chaos_theory sense, meaning that predicting how they evolve over time becomes exponentially more difficult the further ahead in time we try to predict. Meaning we quickly get to the point where all the energy in the known universe wouldn't be suffici…

Intelligence needed to be existentially threatening to all human life <<<<<<< intelligence necessary to survey the totality of atoms and particles in the universe. The former is still very possible.

LessWrongers seem to fear a superintelligence that was 95%+ effective at persuading/manipulating humans to achieve whatever its ends were. But this would require the AI being able to predict the future to a degree that wouldn't be remotely possible given the computational power available to it, due to exponentially increasing difficulty.
Post reply on HN