Sure, the author should be responsible. Yes, GitHub should have a better UX around this action. But... There is another thing to consider: Is it really necessary that a repo that is accidentally made private and then made public should lose its stars anyway? Is that really what the repo owner or the people who starred the repo even want to happen?
When youtube videos are taken down or made private, they are not deleted necessarily from my playlists. There's simply an indicator that they video is 'no longer available'. I don't see why something similar can't be the behaviour for GitHub stars and watches?