Earlier quoted context omitted.
Yes, reasonable assumptions! But according to whom? The golden rule can be reasonable to you but not to me. But does that make me bad and you good? Not really. It just means we disagree on what it means to be a moral person. Anyone who doesn't agree with your "reasonable assumptions" has bad morals can be twisted to mean any moral values based on what you mean by reasonable. For example, the church can say it is reas…
The reasonable assumption I was referring to is what the words in the golden rule mean, not what actions are moral or not. The golden rule is almost as far as it gets from a “specific fixed rule”, despite the name, and it doesn’t prescribe what’s bad or good or moral, and nobody upthread said anything about what makes you bad or good. It’s nothing more than a rule of thumb, an aphorism, a (perhaps extremely obvious)…
The take-home being, as long as peoples preferences are different, the rule has some failure states based upon that.
And the wider example that this points to is that as long as everyone's values are different.. the rule loses a lot of it's luster. The more different another person's values are, the less you can rely on projecting your own values as a method to gainfully interact with them.
In place of that biblical aphorism (born from a monoculture no less), there is a lot of room available for us to build something more robust and sophisticated and the AI alignment problem offers a fine excuse to get cracking and try to formalize something already! ;)