TL;DR is:
If you're interested in inexpensive email hosting for custom domains, consider OpenSRS.
Recently Google stopped support for custom domains for free*, so I moved my domains recently for a similar reason.
I have a couple domains and they all point back to a few mailboxes. Many of the services I looked at were going to charge me per domain per mailbox not just per mailbox so the prices were artificially high.
I signed up as a domain reseller at OpenSRS and pointed my domain's MX entries over to them and... that was it. Super easy.
This gets me a mailbox for $0.50 per 5gb per month. I have two real mailboxes, dozens of aliases spanning multiple domains, and an email forwarder (sending to certain addresses emails to both my wife and myself) for $1.10/mo.
The negatives:
* I've been spoiled by gmail's search capabilities.
* I'm not a domain expert and not a mail admin, so I worried I was going to mess it up. I didn't, it wasn't terribly painful, you can do it.
* I was told they didn't support plus addresses (jane+homedepot@example.com) so I purged my plus-address logins, only to find they really do support plus logins later.
On desktop I use a normal imap client (their web email is decidedly spartan) and on my phone I still use the GMail App.
The price beat the socks off of anyone else I had looked at and finally encouraged me to get off of GMail.
* I'd been a "legacy" customer, and yes, they backtracked but by the time they *officially* agreed to keep supporting me I'd already needed to make a decision