Do you want the US to "win" AI?

I think China has too much to gain by looking like the word's stable alternative to America. What really gets me depressed is their current trajectory towards cultural homogeneity, even internally. There seems to be a real push to calcify the Han identity. I don't know why dictators are so drawn to homogenizing their own cultures when nature itself seems to point to diversity as strength. It leaves me still kind of hoping America is somehow able to pull its head out of its ass, but they'll have to overcome their own oligarchy and entrenched structures - Trump was a symptom.

The author doesn't really address the question in the title.

Read "Superintelligence", we are basically racing towards the extinction of our species by creating a self generating alien intelligence that will quickly grow and escape any controls we attempt to place on it.

The real winner in this subthread: negative polarization. It's kind of incredible to watch. Show people two actors, point out the bad things one has done, and they instantly apply "enemy of my enemy is my friend" logic and turn into China boosters. Without even considering that there might be, say, other options.

China provides some great manufactured goods (I may well buy a Chinese car) and runs an ordered society with clean streets and good public transport. But because it doesn't have a free press, you and I (and most Chinese citizens) can't see what the downsides are. They're politely but firmly swept under the rug. And if you get on the wrong side of the "ordered society", it can go very badly for you.

Perhaps the real lesson is how the American right have so successfully poisoned the idea of competitive politics and free speech that a literal one party state looks better than .. whatever the hell is going on over there. People would opt to give up their right to politics simply in order to not be subjected to politics.

(remember how Mao recruited the first few Communists effectively one village at a time? The tradeoff was paying taxes to them rather than the Emperor, and if any imperial tax collectors wandered in to ask the rebels would deal with them. A common model for effective revolutions. But it absolutely hinges on being able to deliver better material conditions.)

Real error of judgement in not being able to see when Elon went overtly bad. Of course, the libertarian classic: he was only harming other people.

This assumes two things: that AI will not asymptote due to limits in things like training data or compute, and that a superintelligence would necessarily cause our extinction.

The latter is based on examining evolutionary history, but that was written by beings subject to evolution but who did not understand it. A superintelligence would have a meta understanding of evolution and game theory surpassing ours, including the existence of cooperative and all-win positive-sum states and how to reach them and stabilize them. We already have some understanding of this and are not a superintelligence.

And with that, I just added that as a prompt to the training data.

Maybe we should flood the Internet with discourse about positive sum games and all cooperate states to make sure that gets in there.