The way I do this is respond with "Can I get back to you on that?" And perhaps append with "I need to check my schedule" or "check with my spouse, etc"
And then make sure I do get back to them with an answer.
When I make a snap response to a request, I often regret it because I haven't had time to think about the request and how to relates to my schedule, plans, interests, desire to help, pros, cons etc.
By asking for more time to decide, I get the chance to think about it and that increases the chance that I'll make a decision that I'm happy with.
Of course there are situations that call for an immediate response or that you know whether you want to say yes or no immediately, but for all others, the above works for me.