At the time I was starting my first, my eldest daughter, then 2 years old, was diagnosed with leukemia, and between that time and cashing out, my wife and I had three more children, and she lost her father to cancer.
My point is: life happens, often while you're busy making other plans.
Other people here have brought up the "late hours" issue. Obviously, you and your spouse ought to both be comfortable about the kind of hours you keep.
But another issue-- more important, in my book-- is how you and your spouse relate to risk. Startups are, by their nature, perceived as being riskier than a stable, nine-to-five job. (Whether they actually are or not is immaterial-- it's the perception that's relevant here.)
When I got started, it became clear to me that I had a much higher tolerance for financial risk than my wife. This was not a problem, as I was able to make it clear to her that if there was ever a problem providing security for the family, I'd give up the startup and get a "real job".
By the way: based on your "possibly one with kids, ugh" comment, I'd suggest you make sure that you and your spouse are on the same page as more fundamental things as well. If you view the possibility of children as warranting an "ugh", you should be working to remove that possibility, for the sake of the eventual kids, if nothing else.