61–70 of 236 posts
Earlier quoted context omitted.
Cheaper in terms of $$$. If you like managing a database, I'm sure it's a good deal!
It was very little effort to setup. Used puppet to define everything I wanted the infra to be, and have pretty much left it be. It's only reachable from the app servers, backed up to tarsnap. It's really not a lot to learn and hasn't been a chore to operate.
Earlier quoted context omitted.
They're talking about running the db on a vm, setting it up and tuning it themselves. As opposed to a managed solution which is setup and tuned and hosted on a vm controlled by another company.
I'm still unclear on if the VM here is on a machine they own/on their own premises? Or is it under someone else's control? "In the cloud" sounded like the latter but it sounds like the intention was the former?
Whereas you can get a machine on AWS and setup the database yourself.
IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…
In my country, after certain size, pg on aws ec2 with a few capable dba/devops salary still cheaper than rds or aurora
Self-hosting MariaDB with multi-master replication. Works pretty well for my cases. Have a friend that was surprised by the AWS pricing changes on Aurora (they charge per query now instead of bandwidth/usage?)
As traffic grew we'd encounter evermore outages - it seemed that some percentage of requests between nodes would be dropped (despite AWS's insistence to the contrary) and the nodes couldn't seem to gracefully recover. With the rapid growth, and tiny team we didn't have enough time to fully get to the bottom of it sadly.
The final straw for Galera for us was when we started exploring multi-region replication (us-east-1 and Singapore). The bandwidth between regions was just not high enough to support a cluster. Aurora offered all that and more (though MariaDB to MySQL was a bit fiddly). Since migrating life (and uptime) has improved immeasurably.
You can use a cheap virtual server + attached storage. In my mind, it's not as difficult as people let you believe (until you hit a scaling issue and want horizontal scaling for example).
I also teach how to do it in my upcoming book[0] where I even have a scripted demo to create your first cluster including SSL, SELinux, attached storage,...[1].
For work stuff, I would just use managed offering in the cloud the company already has. So far, that was AWS and Azure.
[0] https://gumroad.com/l/deploymentfromscratch [1] https://gist.github.com/strzibny/4f38345317a4d0866a35ede5aba...
Yes, I selfhost Postgres/Redis/Mongo for small projects (db + app server on the same machine or a small number of VMs). Usually in docker, sometimes systemd units. It's amazing what you can do on a single $5-50/month machine if HA isn't super important. On larger projects (typically once k8s gets involved) I'm running on a cloud provider anyway and I might as well use a hosted version like RDS for the main database.…
IMHO self hosting your database (even in the cloud) is the best way to do it. You have control over the version. You have control over features. You have control over performance. It’s tons cheaper for greater performance - especially when you go over a few hundred gigs. Yes, the hosted ones have built in replication - but my data is far too valuable to put in the hands of a third party. If they lost it - they could…
A startup, where you'd rather have your engineers working on adding features to your product instead of "managing and operating" a database? Have a lot of money to throw at this problem, being aware that you are paying for convenience? then using RDS and such would probably be a better choice.
An established company/product with a team of people working at a non-crazy pace, and catering to steady enterprise customers? Would like to build a sustainable long-time infrastructure setup that is optimised for efficiency and cost? running your own Database chosen well for your needs, backed up and tested rigorously, and managed by people who cares for the company's infrastructure would be a better idea.
It is weird when large companies with a dedicated platform team choose to run so many databases for their teams/services on a PaaS. That clearly is an optimisation opportunity.
We are running mariadb on a debian vm in azure and using both vm backups and mariabackup.
Works fine.
Earlier quoted context omitted.
I'm mostly thinking about all the stuff that goes wrong. Corrupted data files, server(s) die, monitoring, getting paged, backups, testing restores, yada yada. It works until it doesn't, and then the value you derive from self-hosting is questioned when there's a failure. If you can self-insure against shit happening & like doing so, then it's probably worth it. Mostly just point this out for the potential self-hoster…
There are definitely some things to be considered here, however I find that most people drastically overestimate the amount of work associated with hosting things. Also they tend to underestimate the amount of work required when using managed solutions. For example, you'll certainly want to do secondary backups and test restores even for managed options.
If your database fits one server, you should consider it.
[0] https://gist.github.com/strzibny/4f38345317a4d0866a35ede5aba...
I self hosted everything from Postgres to ElasticSearch. IMHO, managed solution ins't that much reliable than running your own with the right planning. Failover strategy of AWS RDS is just absurb. It's literally just promote other instance, switch DNS over. And I do see issue where the new master are behind a few transaction compare with the replica... Managed database is just to move the responsibility when the data…
Amazing. What are some advices that you would give for someone who is looking to self host?