Why am I not terrified of AI?

This guy, and apparently most people, get it all wrong. AI is not remotely close to having sentience or even a will to survive. It's the fact that it's a powerful tool that can be abused by evil people that should scare you. Here's a single example that is simultaneously trivial and extremely potent: https://www.theverge.com/2022/3/17/22983197/ai-new-possible-...

I normally enjoy Aaronson's writing, but I'm actually chilled.

This essay depends on a specific, American-hallow take on the Second World War. The 'Orthagonality Thesis' is just a fancy way of shifting the burden of proof from where it should be -- on the person claiming that intelligence has anything to do with morality. It would be better to call it what it really is, the null hypothesis, but sure, ok, for the sake of argument, let's call it the OT.

Aaronson's argument against the OT is basically, when you look at history and squint, it appears that some physicists somewhere didn't like Hitler, and that might be because of how smart they were.

This amounts to a generalization from historical anecdote and a tiny sample size, ignoring the fact that we all know smart people who are actually morally terrible, especially around issues that they don't fully understand. (Just ask Elon.)

I'm not even going to bother talking about the V2 programme or the hypothermia research at Auschwitz, because to do so would already be to adopt a posture that thinks historical anecdote matters.

What I'll do instead is notice that Aaronson's argument points the wrong way! If Aaronson is right, and intelligence and morality are correlated -- if being smart inclines one to be moral -- then AI (not AGI) is already a staggering risk.

Think it through. Let's say for the sake of argument that intelligence does increase morality (essentially and/or most of the time.) This means that lots of less intelligent/moral people suddenly can draw, argue, and appear to reason as well or better than unassisted minds.

Under this scenario, where intelligence and morality are non-orthogonal, AI actively decouples intelligence and morality by giving less intelligent/moral people access to intellect, without the affinity for moral behaviour that (were this claim true) would show up in intelligent people.

And this problem arrives first! We would have a billion racist Shakespeares long before we have one single AGI, because that technology is already here, and AGI is still a few years off.

Thus I am left praying that the Orthogonality Thesis does in fact hold. If it doesn't, we're IN EVEN DEEPER TROUBLE.

I can't believe I'm saying this, but I do believe we've finally found a use for professional philosophers, who, I think, would not have (a) made a poorly-supported argument (self-described as 'emotional') or (b) made an argument that, if true, proves the converse claim (that AI is incredibly dangerous.) Aaronson does both, here.

I speculate that Aaronson has unwittingly been bought by OpenAI, and misattributes the cheerfulness that comes from his paycheck as coming from a coherent (if submerged) argument as to why AI might not be so bad. At the very least, there is no coherent argument in this essay to support a cheerful stance.

A null hypothesis again! There need be no mystery to his cheer: he has a good sit, and a fascinating problem to chew on.

The AI panic has caused me to discover the Lesswrong forums. It is absolutely the most hilarious cult I have ever encountered. The same adoption of weird insider language, the same rejection of outsiders and denounciation of “non rationalists”. They even aggressively promote their religious texts called unironically “The Sequences.” I used to wonder how Hollywood got so much influence from the “Scientology” cult but after seeing the “Rationalogy” cult it now makes total sense. These people need to be laughed at and excluded from public discourse.

What a fantastic (and apposite!) response to the question -- and the article made me nostalgic for the days when intellectuals disdained television, rather than hoard lore about HBO miniseries. Not because they were better, but because I was young then. :)

schlorb schlorb schlorb schlorb schlorb