They’re basically handing over AI dominance to china if these laws come into reality
Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
271–280 of 873 posts
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#272Earlier quoted context omitted.
Global coordination using coercion has worked for nukes. It's very hard but not impossible to ban NVidia or any competitor and create spy network in all large companies and if any engineer is found to create AI, mark them as terrorist and give them POW treatment. And monitor chip fab like nuclear enrichment facility and launch war with country found to create chip fab, which I belive is hard to do it in complete secr…
Sure, I think that kind of "coordination" is a little more credible than what I thought Eliezer was proposing. Maybe I just didn't read carefully enough. One big difference here, however, is that the barrier to entry is lower. You can start doing meaningful AI R&D on existing hardware with _one_ clever person. The same was never true for nuclear weapons, so the pool of even potential players was pretty shallow for a…
I think in a few years you'll only be able to run signed software. And maybe any code you write will be communicate to some government entity somewhere to check for terrorism-impact? Well, the future is going to be very fun.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#273Earlier quoted context omitted.
That might be true but there's no way of knowing for sure. Planet wide gulag system(a colorful and a bit wrong name for prisons) already exists and is pretty aligned on topics like killing fellow humans as a result of conscious, deliberate action. I find Max Tegmark's argument pretty convincing. We seem to agree to "STOP" on things like genetically modifying ourselves at conception or birth or whatever to gain X or Y…
Unlike for AI, in those fields we do not need to stack endless hypotheses on top of each other to arrive at something bad (and even then some prohibitions are tenuous in their reasoning at best).
I don't know about that. Cloning/modifying humans has potentially so much benefit and yet we seem to stack hypotheses on top of it and conclude it's not worth it. There is also a lot to be said about all the hypotheses of killing all the Hitlers preemptively. I think that whatever we do, we usually arrive at a bit modified trolley problem and try to go from there. We really should never expect that the one particular route that we choose carries all the benefits and the other all the downsides.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#274People focus on LLM's and diffusion models because they are so omnipresent now. But an AI for stock picking and prediction that would be realy next level and outcompete the current ones cosistently would siphon of so much wealth it would basically own society if the operators were clever enough not to get so gready short term that the system would litterally collapse overnight.
Diffusion models came from finance and have been applied there many many years before it was brought to this wave of AI
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#275Earlier quoted context omitted.
The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…
We do not have AGI yet. In order to get there, very specific chips are needed that can only be built by a handful of companies in a handful of factories in the world. On top of that, the available stock of existing cards is predominantly in the hands of large tech companies. If China and the US agreed - perhaps even if the US alone wanted it -, development of AI would be severely constrained if not stopped tomorrow.
Government will rally to make this internally as part of their secrets projects. This will be worse. The US sanction mechanism has shown that it's highly flawed with the Russian sanctions. This will not work but I can see some bureaucrat out there going all in on this.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#276Earlier quoted context omitted.
He actually doesn't think it's viable, but thinks its the only card left to play. He's pretty sure that human civilization will be extinct this century.
> He's pretty sure that human civilization will be extinct this century. If they are, it'll almost certainly[1] be climate change or nuclear war, it won't be AI. [1] leaving some wiggle room for pandemics, asteroids, etc.
There is no scenario where climate change leads to human extinction.
Worst case scenario is global population goes down by a single digit percentage. Horrible, horrible stuff, but not extinction. And most likely scenarios are not nearly that bad.
Nuclear war, if the worst nuclear winter predictions are correct, could be a lot worse but there would still be some survivors.
Unaligned ASI though, could actually make us extinct.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#277The most notable thing about AI right now is that it's the new widget. The economy wanted it, the zeitgeist wanted it, and for that purpose no more development is necessary. It's already reshaped McKinsey's advice, Microsoft's operating system and the public's trust in the exceptionalism of art creation. take Auto-Tune: before it blew up, one uncritically accepted that a great sounding vocal performance was simply th…
>I can't recall a time where the general populace was this vulnerable, ill-informed, traumatized and submissive. I don't know why you think this is the case; the general populace seems less submissive now than ever before. Consider how much opposition there is to Israel's bombing of Gaza among the western populace, even though almost all the elites support it. Or how the uptake rate for the latest covid vaccine boost…
> to Israel's bombing of Gaza among the western populace, even though almost all the elites support it. Or how the uptake rate for the latest covid vaccine boosters is under 10%, even though almost all the experts and elites support it.
I'm not sure how significant either of those examples are, although I agree with your point for other reasons (mainly "Extinction Rebellion").
For Israel vs. Palestine, there is a large scale propaganda war on both sides that is, unusually, actually getting through this time. Partly because X isn't filtering such things anymore, but also propaganda is a constantly moving target.
For the latest Covid booster, while I have had the latest booster, I only know it existed because my partner already got it. No government campaign telling me about it.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#278Earlier quoted context omitted.
What's your view on where the X risk idea breaks down? Is it that it's impossible to make something smarter than a human? Or that such a thing wouldn't have goals/plans that require resources us humans want/need? Or that such a thing wouldn't be particularly dangerously powerful?
It could break down on any of those, for example (with the first one not forever, but could take much longer than what is mainstream thinking now). A lot of existential risk involves bold hypotheses about machines being able to "solve" a lot of things humans cannot, but we don't know how much is actually just solvable by superior intelligence vs. what just doesn't work in that way. Human collective intelligence faile…
"It could" implies to me that there's a chance they don't.
What would you say the odds are on each of those three components? And are those odds independent?
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#279Earlier quoted context omitted.
Unlike for AI, in those fields we do not need to stack endless hypotheses on top of each other to arrive at something bad (and even then some prohibitions are tenuous in their reasoning at best).
I don't know about that. Cloning/modifying humans has potentially so much benefit and yet we seem to stack hypotheses on top of it and conclude it's not worth it. There is also a lot to be said about all the hypotheses of killing all the Hitlers preemptively. I think that whatever we do, we usually arrive at a bit modified trolley problem and try to go from there. We really should never expect that the one particular…
My point was more that it really isn't stretching anything to say: if something bad gets into human reproduction that has unfortunate effects. We can observe that already. So at least the risk case doesn't rely on a lot of hypotheses that might turn out wrong.
Whether or not all regulations there are good or optimal is then a separate point again.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#280Earlier quoted context omitted.
My estimation of Eliezer Yudkowsky just dropped a notch — not because I don't take the existential threat posed by AI seriously — but because he seems to think that global coordination and control is an even remotely possible strategy.
How are you anti-doomers arguments still this naive years later? No one is saying global coordination is in anyway remotely easy at all. They’re saying that it doesn’t matter what the difficulty is, the other option is fucking catastrophic
I don't think that's a fair representation of my position. I'm not suggesting that it's "hard" — I'm suggesting that it's absolutely futile to attempt it, and at least as bad as doing nothing.
I'm also suggesting that there are lots of other options remaining that are not necessarily catastrophic. I'm not particularly optimistic about any of them, but I think I've come to the opposite conclusion to you: I think that wasting effort on global coordination is getting is closer to that catastrophe. Whereas even the other options that involve unthinkably extreme violence might at least have some non-zero chance of working.
I guess the irony of this reply is that I'm implying that I think _your_ position is naive. (Nuh-uh, you're the baby!) I suspect our main difference is that we have somehow formed very different beliefs about the likelihood of good/bad outcomes from different strategies.
But I want to be very clear on one thing: I am not an "anti-doomer". I think there is a very strong probability that we are completely boned, no matter what mitigation strategies we might attempt.