Upgrading database versions is mentioned briefly in the article, but is pretty much a show stopper at the moment for some databases.
e.g. Postgres (https://github.com/docker-library/postgres/issues/37)
Why Databases Are Not for Docker Containers
71–80 of 184 posts
Re: Why Databases Are Not for Docker Containers
#72Alright, all you brave people who run databases in docker: where do you store the actual data? - Host mounts? So what happens when a container gets rescheduled to another node? - Docker volumes? What happens when a container gets rescheduled to another node? - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess? - Shared filesystem like NFS, or Ceph? How's the performance fo…
Mount a volume on the host and make sure only one database container runs on that host. Done. Then it's exactly like a traditional setup except better because if the process crashes or config gets corrupted it can be replaced automatically in a few seconds, good as new. There's nothing brave about that.
This is what we do. We use Docker Swarm and label our database host. Then add a constraint to the service/container i our docker-compose file so that the database is always scheduled to that host.
Great thing about this setup is that we can now run the entire production system locally on our laptops and test servers with or without swarm. Everything is deployed in the same way and everyone uses the same versions and configurations (even for our database).
Re: Why Databases Are Not for Docker Containers
#73Alright, all you brave people who run databases in docker: where do you store the actual data? - Host mounts? So what happens when a container gets rescheduled to another node? - Docker volumes? What happens when a container gets rescheduled to another node? - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess? - Shared filesystem like NFS, or Ceph? How's the performance fo…
Mount a volume on the host and make sure only one database container runs on that host. Done. Then it's exactly like a traditional setup except better because if the process crashes or config gets corrupted it can be replaced automatically in a few seconds, good as new. There's nothing brave about that.
Same here. Mounted on the host with the restriction to running containers.
Re: Why Databases Are Not for Docker Containers
#74Of course the counter example of this is Google, which, as I understand it, runs everything on containers. It seems like if its good enough for Google, then its good enough for the rest of us.
Google runs NOTHING on Docker. They have their internal secret proprietary container technology. The concept of containers is fine. The troubles lies in the implementation that is available.
Google does publish https://github.com/google/lmctfy at least, though.
Edit: did publish
Re: Why Databases Are Not for Docker Containers
#75I still can't really figure out what a container is. Every time I think of a use case for one, I read something like this which says that's a terrible idea. The use-case I need solved most often is the following: Create a standalone "server" that accepts and responds to network traffic, has some way to store data, and whose dependencies (i.e. system packages, frameworks, etc) I can manage independently of any of the…
You might want to create your own containers with standard bash commands instead of using docker. You can try cgroups, which are a standard Linux kernel feature.
For Ubuntu https://help.ubuntu.com/lts/serverguide/cgroups.html
For RedHat https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...
Everything should be easier to understand after that.
Re: Why Databases Are Not for Docker Containers
#76> But what about Configuration Management systems? They’re designed to solve this kind of routine by running one command. The problem with this for most of the developers you see praising containers, is that with a containerized setup, you've already got the rest of your deployment process down to `docker service update --image myorg/myservice:1.3.0 myservice` (And, in fact, maybe you're even running that code agains…
And what's gonna install CoreOS? And what's gonna install Docker? And what's gonna configure Docker? And what's gonna give a configuration to run the Docker image and how?
The ops team that uses some form of config management that the developers never touch.
Re: Why Databases Are Not for Docker Containers
#77Docker is a packaging tool. How you chose to deploy your software package has nothing to do with your operational and DB administration procedures. Mixing the two topics is very confusing.
Re: Why Databases Are Not for Docker Containers
#78This article is terrible. It's a lot of wishy-washy explanations devoid of technical detail - because their isn't a technical explanation or justification for this list. I've run extensive benchmarks of Hadoop/HBase in Docker containers, and there is no performance difference. There is no stability difference (oh a node might crash? Welcome to thing which happens every day across a 300 machine cluster). Any clustered…
Are you telling me that you'd trust MySQL auto replication failover enough to have it activates multiple times per day (even with Percona)? On a busy cluster with, say, 300GB?
Seems like a straw-man. Why is a process running within a container failing over more frequently than a process running directly on bare metal -- seems like more of a resource/process scheduling issue than anything to do with containers.
Re: Why Databases Are Not for Docker Containers
#79This article mentions offhand that the storage drivers are unreliable, even for data volumes. Is that actually the case? Is there a serious risk that a database will be corrupted by a container crash, as the article claims? A regular crash of the computer should not be able to corrupt a database, is a container more dangerous in this regard?
In order for a database to provide such guarantees it needs some guarantees from an underlying hardware.
Mainly that when disk say that data was written, it really was written, and that the data was written in the right order.
The danger with overlay systems is that they might not provide these guarantees, which makes database writes unsafe. Given that current overlay drivers are unstable as it is, I doubt they put any effort to enforce such guarantees.
Not exactly a database but for example with ZFS it is known that ZFS can't provide guarantees that usually provides if you run it in a VM, unless you have a hardware that supports VT-d (a.k.a. PCI pass-through) and have it enabled.
Re: Why Databases Are Not for Docker Containers
#80Earlier quoted context omitted.
> Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. can you go into a little more depth? my understanding of a VM is that it installs the OS in a dedicated memory partition, and allocates hardware resources separately from that of the host machine, such that resource contention between host and VM never happens. the VM allocated resources just go dark for the host machine while…
I'm doing a bit of a simplification here so please someone correct me if I'm not saying this correctly Every single instance of a VM has its own Kernel. When a VM boots up, it gets allocated a portion of hardware and boots up a kernel and allocates memory to itself. VMs each are isolated from each other in that they don't share resources and each VM is free to do whatever it wants to do with the hardware it is given.…
VMs share resources from the host: disk, network, memory. Just like a container shares resources from the host.
Containers re-use the running Operating System from the host, it saves memory but it can only run a single OS.
A VM can run any operating system, and each VM runs its OS independently. VMs are memory intensive, there is a base 100-500MB to pay to run any VM because of the independent OS. (Note that the advanced VM managers have evolved to have memory deduplication and COW across VMs.)
Containers exist to save memory. That was the critical pain at the time they came to existence. Memory was expensive, it was a major problem when you wanted to run 10 hello world applications as 10 separate VMs.