I too have been running my own mail server for a couple of years now using Postfix (SMTP) and Courier (IMAP). I never got around to setting up webmail.
Despite not doing much beyond SPF, I have never had any problems dealing with my outgoing mail being blacklisted/ignored/considered spam.
Downtime has not been a serious issue for me as my host (Linode) has generally been very reliable. Most of the downtime my mail server has experienced has been due to me. If having 100% mail uptime was really important to me, I'd set up a secondary server at a different data center.
For backups, even though Linode offers backups, I use Amanda to back up everything to my local backup server.
For security, I allow TLS connections, but relatively few servers eagerly try to encrypt connections. I also use full disk encryption on my server. Ultimately, I think that if you have sensitive information to e-mail, you cannot rely on the mail system to protect it. You should rely on something like GPG to encrypt the contents of the e-mail.
As far as searches and tagging go, that's really a mail user agent problem. So that needs to be solved either in your webmail or mail client software. I do run maildrop to do some preliminary sorting on a lot of my e-mail. It's a program similar to procmail, but it works my virtual mailboxes.
In the end, running my own mail server has been a good experience. It took some time to get it properly configured, but running it day-to-day hasn't been much of a problem.
I use PostfixAdmin to manage things like adding new mail aliases. It's pretty easy to use. My wife, who is tech-savvy but doesn't know anything about running a mail server, uses it to manage her own mailboxes