One story I have that I think agrees with you is from when I was looking after my first child.
I was left alone to care for her, my wife needed a break and so went away for a night off. Btw, this is a good thing to arrange if you have a fairly new baby in the house - it gives the person getting the night off something to look forward to and provides real respite.
Back to the story, the baby (unusually, for she was a sweet little thing) would just not stop crying. She sensed mummy was away and was not happy (probably a commentary on my skills). I spent what felt like half the night with a howling baby, but in hindsight was probably a lot less than an hour. During that time I suffered a deep, deep trough of despair. I was under pressure at work, we were frankly bust financially, I wasn't all that physically well at the time and I was a shitty dad. I felt like I could see my life rolling out in front of me like no man's land in the European first world war (think, mud bath/moonscape/wilderness).
The thing that helped me was the fact that I could think back to evenings spent on tropical beaches, weeks in the mountains of different continents and meals in restaurants designed by Michaelangelo. I thought "Well, if this is what life is to be then that's ok. It was my decision and I have had a great time already, there is nothing I regret" and that really helped. Without that I don't know what I would have done, probably gone downstairs and drunk the contents of the booze cabinet, it's terrible to even write it. But, I understand why people in similar situation take drugs or go crazy.
Of course, 1 hr later my daughter was asleep and my mind set altered back to the reality of my privileged middle class existence. My problems didn't go away, but they were of course minor in the scale of real problems (as we all find out eventually). Some people end up in the literal no mans land of a european war, some people end up homeless, some people have their babies taken from them. None of this happened to me, thank whatever deity.
Go take that big adventure, it will give you a perspective that may prove priceless.