My day job is just hitting 5 years. Of the original team hired under the founders only about 30% of us are left (we've hired some replacements), and the founders were part of the group that was cut.
Watching the founders leave was the hardest because you could watch them slowly come to terms over the first couple of years that:
a) this actually is harder work than what we left
b) it's not just somebody tossing money at you to go work on your pet projects and screw around, treating the expense account like a personal luxury account
c) there is a reasonable expectation by the investors at the end of all of this
that they will make their money back off of your work
Surprisingly, making the transition to a professional management team was easier than working with the original founders. During the tenure with the founders they made all of the mistakes above, plus acted like petulant children when they ran into each of these problems. Whining, foot stomping (literally), crying, flying cross-country to be away from the team and showing up unannounced at a satellite office. Bizarre behavior.
I guess what I'm saying is that it's not just the work that's hard. It's the intestinal fortitude to deal with technology, business and people, day in and day out -- especially when all three of those things are often acting at odds with each other that makes startup life hard, and being small, with startups those things can be magnified out of all proportion. Minor office quibbles turn into nuclear explosions, a broken technology stack a looming iceberg, lack of sales ability into a waiting sink hole.
People often remark about the perks of working at a startup, but it's those perks that make the often miserable part of the work tolerable. Or at least act as transparent incentives to work more, cry less and quit all the whining -- because at least you get three free meals a day!