I still feel guilty when telling people that I am only open for meetings from 11 am to 5 pm. The only exceptions are people in distant time zones.
It can also feel wrong if your SO is on a regular 9 to 5 and you are getting up without an alarm clock, going shopping and making appointments in the middle of the day, and generally setting your own work habits according to your own preferences. Last night I was working at 1 am because I couldn't sleep and there was something I wanted to get done.
OTOH, she never has to go food shopping and I have always been there for the kids in ways that are not possible for her - helped my adult child load and unload a moving truck last Wednesday afternoon, and over the years took both kids to school activities, driving lessons, and other events that my spouse couldn't attend.