Artificial Morality

The article's underlying thesis seems to be summarized by

> The practitioners of AI are not up-front about the genuine allure of their enterprise, which is all about the old-school Steve-Jobsian charisma of denting the universe while becoming insanely great.

But that... doesn't seem true? Most modern AI practitioners, especially the prominent ones, profess that their work is going to change the world forever and that's why they want to do it. So I'm not sure what the author is really proposing that AI practitioners should do. Work on less important problems? Play it by ear instead of trying to develop moral principles preemptively?

In my experience, the AI Safety people are less concerned with a nebulous "morality", and more concerned with practical things like making AI self-improve slowly enough that we can catch it before it's a runaway process; or making AI not think in ways where it imagining torturing people means there's an actual ephemeral consciousness experiencing the qualia of being tortured. Engineering stuff, that happens to interact with morality... in about the same way that genetic engineering could be said to interact with the morality of the resulting organism.

> Technological proliferation is not a list of principles. It is a deep, multivalent historical process with many radically different stakeholders over many different time-scales. People who invent technology never get to set the rules for what is done with it

Well said. Pessimism is a reasonable reaction to this sweeping line, but I offer a quick summary of what I hope is non-naive optimism:

1) Problem: When we think technology is neutral, we miss how it creates niches outside our intended design and social/political constraints, since the environment of economics thwarts all. We have agency as individuals, yes, but the environment rewards and select for any uses of the technology which yield advantage. “If you incentivize something, it will happen.”

2) Solution?: Nobody fucking knows. But “Game ~B” is what people are calling a nascent research effort to identify what attributes a more cohesive civilization must have to allow better collective “sense-making” and “choice-making” so we may become capable of directing technology instead of leaving it to the emergent dynamics of hapless competition. What's also interesting are the considerations for employing antifragile, transitionary social structures that can thrive in our current entrenched “Game A” dynamics to plant said attributes. Gotta change the underlying rules instead of designing atop those which are broken.

3) Future Ethics: I've also seen some excitement about a so-called non-relativistic ethical framework within Game~B, called The Immanent Metaphysics. It claims to be commensurable with — but not derivable from — science, rigorously formalizing intuitive ethical principles, to at least create a consistent grounding for making effective ethical choices.[1] The hope is to better inform (but not determine) our use of technology after we step into a more coherent phase of civilization. Tech can NOT be put back in the bag, so we must develop our wisdom to carefully wield the godlike powers they give, which we are currently making a mess of.

[1] only a few people currently understand the dense framework, but I'm currently working through it and may be able to offer a summary of what I know so far here. neat stuff

But because there's a race to the bottom, and information is duplicated with marginal values near zero, sites which were free and didn't require monetization would squash sites that did over time.

I see convergent evolution towards ad networks/data brokers, monthly payment walled gardens, and crowdfunding.

I wonder how "pay to view" Internet would've worked. How do you know if the page/site you're going to open will be worth the 0.05 cents? There are/were kids in Macedonia who made fake news sites to make money from Google ads: https://www.wired.com/2017/02/veles-macedonia-fake-news/ , imagine that sort of behavior minus the G-middleman.

Could they have built an Internet where there was a system to claim refunds, with a "judge" who would decide? Could there have been a rating system ("90 out of 154 visitors said the information on this page was worth the money"), and how could that have been abused?