It was cheap to set up, all the cost is in time to learn to implement in a secure and performant fashion
Hardware actually cost money. Here's a breakdown:
> 4 pcengines alix boxes for openbsd router/firewall appliances
These were around $120 each with 4GB flash storage at the time. They're half that now. Low power, no cooling required, x86, 3 ethernet. You could buy APU2 now for more power
> 3 supermicro Opteron servers with KVM/corosync/Sheepdog/csync2 for hosting all VMs
I used cheap cases, eBay MB/CPU/RAM, tiered storage (green/black/ssd) to keep costs down and infiniband for 10gbit interconnects ($15/card on ebay!). I made sure to get quality components (esp MB/power supplies). One of the servers is also my desktop. I'm guessing they were about $800 each. Having the 3 node cluster is nice. When we had a forest fire threaten our town and we were ordered to evacuate I just grabbed one box and all my data was already replicated to it. When we returned home I plugged it back in and it re-synched back up.
Maintenance has been a non-issue. I haven't had any components die except the occasional HD. Power costs are the main thing. Estimate about $500/yr at $0.10/KWh
> PF + CARP + pfsync + OpenBGPD for routing
I use a local indy ISP that gives me a bunch of static IPs and lets me route a /29 with BGP. I know them fairly well, and get a sweet deal. Doesn't cost more than a regular consumer connect, though it is slower.
I've used OpenBSD since the early 2.x days, and find it very easy to administer. The release and documentation quality are second to none, and I've found the community to be very helpful as long as you've tried to help yourself first.
> Unbound + NSD + Bind for DNS
I keep my Bind server vlanned off and serve everything out thru unbound/nsd. Both of those programs are very easy to set up, the real beast being bind. I know there are better alternatives out there, but I know Bind well and have lots of custom config I don't want to throw away.
> SSH/OpenBSD ipsec/apache Guacamole for roaming and permanent site-to-site VPN (pcengines ALIX hosted at my inlaws in Japan)
If I were to pick one outstanding program on this list it'd probably be Guacamole. Pure HTML5 rdp/vnc/ssh/telnet/etc client that is seriously amazing. I've set it up at a half dozen places now, and it's never so much as hiccuped.
OpenBSD IPSEC is VERY easy to set up, if you've had nightmare experiences with other packages!
> Apache + Lets Encrypt + awstats + relayd for serving web pages and analysis
Apache is the old standard, and awstats is cool for keeping tabs on what is going on in the logs (geoip as well).
Lets encrypt was amazingly easy. I'm using certbot and set it up in under an hour. I'm forcing SSL on all my web services now.
Relayd is another "so simple and it just works" package from OpenBSD. I use it as a front-end load balancer.
> ZoneMinder for video monitoring. Tied into legacy security system for automation
On Debian (my Linux distro of choice), this was simple to set up. Perl scripts to integrate it into my DSC security system. Auto arm/disarm camera recording and relay light control required the IT serial integration board and programmers manual for the system
> Postgres for database work. Some mysql/redis
I've been using postgres forever, so setup and use are second nature. An absolutely incredible piece of software engineering.
> NetDisco + Nagios + NagVis + NFSen + MRTG + Smokeping + PNP4Nagios + NUT + Splunk + Racktables for monitoring. All configs are dynamically generated from netdisco db
This is another stack I've set up at many locations (including businesses). They are a real timesink to integrate together. I have MANY custom scripts to make the config generation from netdisco work properly, but once setup you have total insight into every aspect of your network (and I forgot to list RANCID!). Netdisco/NFSen on their own are still a killer combination, and work as well or better than packages that cost tens of thousands of dollars. I'm happy to help any one trying to set these up if you PM me.
> OpenSMTPD + Citadel (webcit) for email delivery and webmail
Citadel is maybe the weakest thing I have in my stack. I'm looking at the other webmail solutions in this thread carefully
> Minetest server for kids. We use this tons as a family, and the kids spend lots of time modding. TW2002 server. TShock server.
When your kids are asking to learn LUA, you know something is working!
> OpenELEC for diskless netboot KODI machines around the house
Amazing and easy to set up if you already have your own DHCP server you can modify. Just need tftp and nfs after that. Using OLD desktop PCs for this works great. I'm using cast off dell gx290s
> Samba4 Domain controller + NFS for sharing files in different applications
I've been using Samba4 since pre-alpha (TP series) when you had to run your own LDAP server. Things are so easy now its hard to overstate. Using Bind makes it a bit trickier since I need to add some magic entries, but if you use the builtin its a single python script between you and a full SSO AD domain.
> SVN for source control and Config diffs for all servers/tools/network devices
I found SVN config to be a bit of a head-scratcher. I think this is another one where other tools are probably better nowadays. I'm looking at some of the other things people are suggesting.
> Asterisk via FreePBX / NCID for all phone/CallerID services, including remote handsets at VPN locations.
Another timesink. PBXs are hard to configure, and I'd move to another system if there was something less esoteric.
As to why I do it? I find it satisfying to learn how things work, like the idea that I'm master of my own destiny and know how my data is being used
Any specific questions, let me know!