If Anyone Builds It, Everyone Dies

> Eliezer Yudkowsky

Yep, same old fearmongering that somehow keeps him in business.

If you want to make a case against building super AI, you must first prove that building a super intelligent AI is even possible, which is not as obvious as it may seem.

I.e at what level of detail does an AI need to simulate reality (including human behavior) to be "ahead" of it, so it can make accurate decisions?

It’s obvious that, since the mass of humanity could easily take everything from the rich through sheer disregard of the rules but don’t, that there must be a stable social reason. It seems to be the idea that working for a living is honorable, and the best at it can even become the new rich. This counterbalances the absurd desire of the wealthy to own everything. AI is going to work harder than anyone at anything, disrupting this balance. You’ll have no more economic power than you would have at Disneyland with no money. So the question is what happens when absolutely no one is needed to do “work” anymore.

Believe it or not, the book makes a much stronger argument than you do.