Even this is a little unsophisticated for the same reason as the Golden Rule. The problem is that other people aren't just mirror versions of you.
In small children "theory of mind" develops, the child supposes that perhaps this experience they're having, whatever that is, is also an experience the other people who look like them are having. And this theory works well, you can make useful predictions with it. Because Daddy is like you, and you like apple drink better than pond water, you can surmise correctly that Daddy will prefer the apple drink over the pond water too. Being smacked in the face (e.g. by your own unco-ordinated limbs) hurts, and so it's not a surprise that Daddy doesn't like a smack in the face either.
But the explanatory power of theory of mind is limited if you stop there. I don't like Peanut Butter. But it turns out that other people _do_ like Peanut Butter, their minds are similar but not identical. If I see some bread with Peanut Butter spread on it, to me that bread is ruined, but to someone else that's a delicious snack.
I don't want to marry. So for me a Categorical Imperative forbidding marriage is no problem. But it's clear to me that other people value marriage and so, against my better judgement, it should continue to be permitted. Some people feel marriage should be restricted, that couples ought to be of opposite biological sex, or the same skin colour, or of child-bearing age, or whatever other constraint. I have no dog in that fight but it's simplest if we're to allow something I have no need for, to assume that everybody will avail themselves of this custom only if they value it, and so it may as well be open to everyone until we see some clear evidence of ill-effects.