Yeah, I ran my own e-mail server from 1995 until about 10 years ago. It used to be fairly simple, but between incoming spam and attacks, the various standards that were developed, and my e-mail not getting through... this became a problem that was worth paying someone a few dollars a month to solve for me.
I still use my own domain, but I'll let actual delivery and security experts deal with the day-to-day running of things, while I run my business (which definitely isn't that).
It was a bit sad to give up, but the time and frustration it saved have been more than worthwhile.