Statement on AI Risk

I've already read that thread.

I don't have twitter and I agree his tweets have an aura of lunacy, which is a shame as he's quite a lot better as a long-form writer. (Though I will assume his long-form writings about quantum mechanics is as bad as everyone else unless a physicist vouches for them).

But, despite that, I don't understand why you chose that specific example — how is giving a list of what he thinks an AI probably can and can't do, in the context of trying to reduce risks because he thinks loosing is the default, similar to a fundamentalist Christian who wants to immanentize the eschaton because the idea the good guys might lose when God is on their side is genuinely beyond comprehension?

Humans also have an "off switch". It's fail safe as well, they die of dehydration and starvation even if you do nothing. If that's too slow you can just shoot them in the head.

"nico" has spoken

The reason these two ideas get commingled is because in practice they're indivisible. A high redistribution society requires high degrees of coercion and fascism.

To get around this the usual Star Trek analysis (by fans, the series itself doesn't talk about it much) is that after replicators were invented, there didn't need to be capitalism anymore and so there's no money in the future and everyone just works on perfecting themselves. It's a wafer thin social idea that was never fleshed out because the writers themselves didn't believe in it. Roddenberry insisted but the writers often couldn't make it work which is why there are so many weird hacks, like saying the replicators can't replicate things as big as star ships and they mostly just ignore the whole topic. Also the replicators kill a lot of drama because they mean there can't be any valuable objects.

There are obvious and basic objections to this idea that replicators = communism (in either direction). One is that you can't replicate services, and much economic activity today is the service economy. We see that the Enterprise has staff who do things like wait tables, cut hair and sign up for red uniform missions in which they will surely die, but why they do this in the absence of money is never explained. There's just this ambient assumption that everyone works because work is awesome.

Getting back to the thread, the lack of philosophical rigor here is all on McFarland unfortunately. He doesn't actually have a critique of capitalism. He doesn't even seem sure what capitalism is, appearing to use the term to just mean contemporary society and anyone he doesn't like. Even his arguments against his strawman enemies are garbled and useless! He shits on Musk, saying that if Elon invented a replicator he'd patent it and hoard the tech to himself, ignoring that Tesla gave away its entire patent pool so anyone else could build electric cars using their tech. Musk - arch capitalist - did the OPPOSITE of what McFarland claims capitalists do, and he didn't even notice! All the rest of his argument is also like that. He makes absurd claims about governments, Republicans killing animals in TV ads, some non-sequitur about meatless sausages ... it's just this total grab bag of incoherent thoughts that make no sense and don't seem connected to each other, wrapped as "capitalism sucks, communism rules".

If this were an essay I'd grade it an F. But in the end it's just a set of tweets. Those looking for philosophical rigor on the idea of an abundance machine need to look elsewhere.

I thought someone might say that.

This thread is really a perfect demonstration of my point. Our society is so in thrall to self-proclaimed intellectuals that you can literally make a movie presenting falsifiable claims with 100% confidence, people can say at the time "this is absurd and will not happen", you can spend years attacking those critics, it can then not happen and still you will have an army of defenders who dodge behind weasel-words like "generally right".

Of course the usual trick is to express only 95% confidence. Then when it doesn't happen you say, well, I never said for sure it would, just that it seemed likely at the time.

See? It's a winning playbook. Why would anyone not deploy it?

> His opponents at the time were mostly claiming that it either wasn't happening at all, or wasn't largely the result of human activities. What do you think is the current credibility of those claims?

Pretty high, having looked at the evidence. The usual rebuttal is to express disgust and displeasure that anyone might decide these claims via any method other than of counting "smart people". But those "smart people" are who Al Gore was listening to when he made that claim about Kilimanjaro, so they can't be that smart can they?