I don't even want kids. I don't know why the 'time spent with one's children' line hurts so much, but it does. It sharply rises, before plummeting at age 40. That's got to be a jarring transition. No wonder so many folks have trouble with the 'empty nest' and mid-life crisis.
You don't have to have kids. I'm 35 years old; my wife and I (10 years together now) revisit this question yearly - neither of us want kids or wanted kids in these years. We're most likely never going to have kids (by choice). The way I see it, bringing a sentient being into this mad world needs to have a good personal reason, it shouldn't be the default that you need to find a reasons against.
As I look back on the last ~40 years of my life, it's mostly about struggles sprinkled with brief periods of joy lasting no more than a couple of days at most. Most of the time, I'm studying to get good grades so that I can get scholarships, without which I'd not be able to go to college; helping my widowed mom with her chores (including taking care of younger siblings) because she is occupied with making money to survive; and even now that I'm an adult with a stable income, I spend most of my time working for someone (selling my 8+ hours a day to my employer) while worrying about if I'll have enough money saved by mid-50s when I hope to be able to enjoy life a little more (e.g., going on vacations) with some spare money to spend.
All I'm trying to say is that life is a struggle for most people (the poorer family you were born into, the truer that statement becomes). We never have moments of peace because we have things to worry about constantly. I'd never bring another human being into this mess unless I can have a discussion with my future progeny before s/he is conceived (which is, of course, impossible). In other words, unless some ghost/spirit is begging me to let him/her incarnate because s/he see value in this rat race, I'd not make this decision one-sidedly.
As a tangential note, I am a strong proponent of euthanasia and hope that one day, it'll be commonly accepted by the society.