Anthropic researcher says more than 10% chance AI "could kill all humans"

> AI to me is like the Nuclear race again

I don't pretend to know the chances of AI wiping out humanity, but I'm not sure the nuclear race is a good comparison.

Enriched uranium being very difficult to aquire/process makes it practically viable to have some level of proliferation containment when it comes to nukes.

There appears to be no such natural gating factor on AI proliferation.

This is a risk yes, however there can be something done against it. From my point of view, both of what you are describing are a regulatory problems. For the first, hold the AI companies accountable. In this second they either prevent their agents from going rogue or go bankrupt, because the claims of damages are too high. The IP infringement thing is the same, do we want to protect IP, if yes pass a law that they have to pay as everyone else.

Similar for the jobs ... if no one, or nearly no one is left for having a job. Who do you think will consume? Yeah, luxus companies can always sell products like megayachts or super expensive handbags, but these companies are not the backbone of the economy. Economy will simply collapse if no one is buying all the products. So this is a problem that will sort it self out.

Right now, if you want to pay money to a stranger on the internet, and have them draw you a high effort picture using a pencil, this is hard. Recently this was easy.

Instead, what is extremely likely is that you will pay more than the cost of tokens, and get back AI generation. You won't make this mistake more than a few times before you stop trying.

This leads to impoverishment once we get to a point where employing a human to do anything is hard- try to get your sink fixed, exercise your moral principles to pay extra for a human plumber ($100 bucks! The robot plumbing service only charges 99c!), human shows up with a robot and doomscrolls on your porch while the robot does the work. Times are tough and you don't have that much money to waste on bullshit like this. Next time you just hire the robot.

This leads to extinction once paying UBI to a human is hard because robots are much better at applying for UBI than humans.

Talk is indeed cheap.

Thing is, while there are many historical examples of a highly organised bunch of people with a lot of resources who engage in campaigns of violence for various reasons, many such attempts fail disastrously.

Furthermore, given the goal is "no really everyone stop now", this isn't something where violent direct action in e.g. the USA alone would suffice. Has to be global to work, otherwise you just change which language and timezone the disaster starts in. You'd not only need to convince the, e.g. US government that this act of terrorism shouldn't be met with lethal force and even more of the surveillance state we saw since 9/11 (and remember, lots of the people pushing AI now are already big on surveillance capitalism, and the VC money comes from the people running this very forum, so when I say "this will not go well", I kinda mean "I assume at least one person currently working at each of Meta, Alphabet, OpenAI, Anthropic, SpaceX, and Palantir are reading this thread even without any automated processes of their own to tell them about it" :P)

This is a coordination problem on par with historical examples such as "Communism". Which, er, yeah. 70 years after the book was written, Russia had a short flirt with a lite form of it before it became an excuse for a series of "meet the new boss same as the old boss" who stayed around until Glasnost.

This would be a really really bad thing to replicate. Both for the "new boss same as the old boss" part, and the "not actually global" part.

I see it in much the same light - and the spending going into it leads me to infer that those doing the spending see the same thing.

Whoever gets to RSI first and has the compute to act on it, wins the future - assuming they don’t lose control of it.

Similarly to the nuclear arms race, Teller raised the reasonable concern that a detonation could propagate through the entirety of earth’s atmosphere. Thankfully that turned out to not be true, but the parallel is that the need/desire to win this race is similarly strong, and the brinkmanship and game theory in play is effectively identical.