[flagged]
Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
371–380 of 873 posts
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#372Earlier quoted context omitted.
Plants are one thing we do it well enough on. As far as I'm aware we have yet to do it (at scale) on livestock, or even pets.
There is: https://en.wiktionary.org/wiki/Enviropig or https://en.wikipedia.org/wiki/AquAdvantage_salmon or things to the likes of https://www.avma.org/news/genetically-modified-cattle-may-be...
Looks like the fish is the only one that's gone to market. Pretty clearly still early days. I have no doubt that with enough time we'll get there, but to say this is proof that we're technically good to go ahead and produce gmo humans seems a bit rushed -- to reiterate, the point being that the ethical issues here are much more clear than a trolley problem or stack of hypotheses.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#373Earlier quoted context omitted.
Only in exactly the same way that the police have the same agenda as the mafia. International bans need treaties, with agreed standards for intervention in order to not escalate to war. The argument is that if you're not willing to go all the way with enforcement, then you were never serious in the first place. Saying you won't go that far even if necessary to enforce the treaty is analogous to "sure murder is illega…
I don't agree. The police enforce civil order by consent expressed through democracy. There is no analogy in international affairs. Who is it that is going to get bombed? I am thinking it will not be the NSA data centre's in Utah, or any data centres owned by nuclear armed states.
> The police enforce civil order by consent expressed through democracy. There is no analogy in international affairs.
Which is the point exactly. All agreements in international affairs are ultimately backed by threats of violence. Most negotiations don't go as far as explicitly mentioning it, because no one really wants to go there, but the threat is always there in the background, implied.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#374Earlier quoted context omitted.
The resources are also asymmetrical. The number of people who want to do doomsday levels of harm is small and they are poorly resourced compared to people who want benevolent outcomes for at a minimum their own groups. There are no guarantees obviously, but we have survived our technological adolescence so far largely for this reason. If the world were full of smart comic book nihilists we would be dead by now. Even…
I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…
A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack
Chemical weapons are absolutely terrifying, especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings).
If AI makes this stuff easier to carry out then we are completely fucked.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#375The way incumbents are attempting to seize power via regulatory capture, using "X-Risk" and fantastical claims of sudden human extinction is maybe one of the most cynical ploys in the history of tech so far. Open source is one of our greatest gifts, and the push to close off AI with farcical licensing and reporting requirements (which will undoubtedly become more strict) is absurd. The laws we have already cover male…
This is more or less exactly where I land on this. The extinction risk argument is somewhat plausible in the long term[0], but it doesn't seem that we are especially close to developing that class of problem. Meanwhile, there are very real and immediate risks to this type of technology -- in various forms of giving some humans more power over other humans.
The attacks proposed by the extinction risk people actively make those immediate risks worse. The only way we've found to prevent some from leveraging almost any technology to control others is to democratize said technology -- so that everyone has access. Limiting access will just accrue more power to those with access, even if they need to break laws to get it.
[0]- Say, the lower bound of the 95% confidence interval is 50-100 years out. People who I know to be sensible disagree with me on this estimate, but it's where I honestly end up.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#376I don't know why people insist that there can only be one. Maybe both sides of this debate are describing the real doomsday scenarios.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#377Earlier quoted context omitted.
I've recounted this before on HN, but a decade and a bit ago I was visiting friends in the Rocky Mountains, they're interesting and clever people, a bit out of the ordinary and somewhat isolated. Somehow the discussion turned to terrorism and we started to fantasize 'what if we were terrorists' because we all figured that we were quite lucky that in general the terrorists seem to be not all that smart when it comes t…
A real world taste of what happens when smart terrorists decide to attack: https://en.m.wikipedia.org/wiki/Tokyo_subway_sarin_attack Chemical weapons are absolutely terrifying , especially the modern ones like VX. In recent years they have mostly been used for targeted assassinations by state actors (Kim Jong Nam, Salisbury Novichok poisonings). If AI makes this stuff easier to carry out then we are completely fucked…
Yes, exactly. That's the sort of thing you should be worried about, infrastructure attacks. They're a form of Judo, you use the system to attack itself.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#378Earlier quoted context omitted.
> seize power via regulatory capture Not even a little bit. "Stop" is not regulatory capture. Some large AI companies are attempting to twist "stop" into "be careful, as only we can". The actual way to stop the existential risk is to stop. https://twitter.com/ESYudkowsky/status/1719777049576128542 > the push to close off AI with farcical licensing and reporting requirements "Stop" is not "licensing and reporting requ…
The fact that anyone may be taking Eliezer Yudkowsky seriously on this topic is mind-blowing to me. Simply mind-blowing. To imagine that anybody, including the UN, would have any power to collective put a stop to the development of AI globally is a laughable idea. The UN cannot agree to release a letter on Israel-Hamas, imagine policing the ENTIRE WORLD and shutting down AI development when necessary. We can't stop c…
I like to think of this same argument in another (slightly tongue-in-cheek) way: doing this would require solving the human alignment problem. Since this is certainly harder than the AI alignment problem, if we could do it, we wouldn't need to.
edit: s/easier/harder/
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#379Earlier quoted context omitted.
> He's pretty sure that human civilization will be extinct this century. > There is no scenario where climate change leads to human extinction. He was talking about the human CIVILIZATION...
You are right, but "human civilization will be extinct" is an awkward phrase. Human extinction via climate change is off the table. The end of human civilization? Depends on what one means by it. If it means we'll go back to being hunter gatherers it's ridiculous. Some civilizational wide regression is not impossible, but even the direst IUPAC scenario projects only a one digit drop in global GDP (of course, that's t…
> If it means we'll go back to being hunter gatherers it's ridiculous.
Indeed, but even going back to the 19th century would have dire consequences, given our current global dependence on the industrial fixing of nitrogen.
If civilization were to collapse (and I don't think it will, other than from a near-extinction event), I doubt it would be like going back to any earlier time.
Re: Yann LeCun: AI one-percenters seizing power forever is real doomsday scenario
#380[flagged]
I can't tell if this comment is just naive or sarcastic antisemitism. Can you elaborate?