Earlier quoted context omitted.
I found it oftentimes easier to say no without saying reason. The issue for me was that when I state reason, people wanted to negotiate, attempted to "advise" me on how organize my time better and so on and so forth. Overtime (at least against many people) I found it much less draining to simply insist on politely stated no without further explanation.
Counterpoint: when people do this to me, I instinctively assume that the true reason is laziness. On top of that, I feel alienated – what, you think you're so important that you can brush me off just like that, and I'm not even worthy being given a reason? Be careful with this one. It's very much possible to state a reason and still express that you won't change your mind on the matter. Although it is more effort, yo…
It's not rudeness, but honesty that makes me dislike giving reasons. Sometimes I give the reason "I don't want to" which seems about as correct as I can understand.