Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Databases Are Not for Docker Containers

myopsblog.wordpress.com

Post reply on HN
Print view View original

131–140 of 184 posts

Re: Why Databases Are Not for Docker Containers

#131

Post by tayo42 »

mstump wrote: post #7
fabian2k wrote: post #4

This article mentions offhand that the storage drivers are unreliable, even for data volumes. Is that actually the case? Is there a serious risk that a database will be corrupted by a container crash, as the article claims? A regular crash of the computer should not be able to corrupt a database, is a container more dangerous in this regard?

I've been running a couple of petabytes in production with Docker and Cassandra for a couple years (around 2k nodes). I've rarely seen FS corruption, however I must qualify that this is on bare metal. They could be running into issues with the interaction with EBS? This is more of a screed than an argument backed by specific details and facts.

How do you handle making updates to configuration with all your nodes in containers? Do you blue green deploy the cluster or something? Run config management in the container?

Re: Why Databases Are Not for Docker Containers

#132

Post by brianwawok »

state_less wrote: post #125
brianwawok wrote: post #109

Earlier quoted context omitted.

> I've run Oracle in a container Danger, danger https://www.theregister.co.uk/2016/02/24/oracle_vmware_licen...

Relax, nothing is fucked here dude... https://blogs.oracle.com/developer/entry/creating_and_oracle... https://github.com/oracle/docker-images/tree/master/OracleDa...

Just because they write a blog on something does not mean they won't sue you for using it (and then offer to drop the suite if you subscribe to buying a cloud license)

Re: Why Databases Are Not for Docker Containers

#133

Post by cookiecaper »

wstrange wrote: post #46

Kubernetes StatefulSets [1] are intended to address this kind of use case. They provide stable network identity and stable storage. Kubernetes enables a container to declare the resources that it requires, including things like dedicated CPU and memory requirements. There are still some rough edges (example: how do you set the amount of kernel shared memory you need), but those issues are being ironed out. [1] https:…

StatefulSets are a new feature marked as "beta". They were first available in the newest k8s release, 1.5.

Re: Why Databases Are Not for Docker Containers

#134

Post by gnufied »

user5994461 wrote: post #114
subway wrote: post #101

Earlier quoted context omitted.

At no time was the claim made that containers were more reliable than any other method of running a process, only that running a process within a container is not inherently less reliable than un-contained. Unless you've gone out of your way, Docker (as well as other Linux container systems) are just namespacing your process. There's no extra abstraction layer, it's just a more restricted execution environment.

FYI: The networks and the disks are entirely abstracted, with multiple extremely complex abstraction layers.

Volume mounts inside containers bypass unionfs. There isn't anything inherently different about using a partition from container or from a host.

Re: Why Databases Are Not for Docker Containers

#135

Post by brobinson »

user5994461 wrote: post #114
subway wrote: post #101

Earlier quoted context omitted.

At no time was the claim made that containers were more reliable than any other method of running a process, only that running a process within a container is not inherently less reliable than un-contained. Unless you've gone out of your way, Docker (as well as other Linux container systems) are just namespacing your process. There's no extra abstraction layer, it's just a more restricted execution environment.

FYI: The networks and the disks are entirely abstracted, with multiple extremely complex abstraction layers.

Not necessarily. You could use --net=host and -v /db_data:/db_data (equivalent of `sudo mount --bind /db_data /the/containers/root/fs/dir/db_data`)

Run like this, there is no disk or network performance difference between running the db process directly on the host or via a Docker container.

As others have mentioned, this is a really poor article.

Re: Why Databases Are Not for Docker Containers

#136

Post by tinco »

eeZah7Ux wrote: post #117
tinco wrote: post #21

Earlier quoted context omitted.

People make it seem as if Docker is some bleeding edge magical technology, but in reality its most useful features are just thin wrappers around stable linux kernel features and some nice automation. We have also been running databases in Docker (on the tb scale though) for around 3 years, we had the odd issue here and there, but nothing terrible and certainly nothing fundamental or resulting in data loss. If your da…

> People make it seem as if Docker is some bleeding edge magical technology, but in reality its most useful features are just thin wrappers around stable linux kernel features and some nice automation. That's one of the things to dislike. The company and the community are trying to sell it as the best thing since sliced bread and usually forget to assign merit to the kernel developers. On top of that, its 180k lines…

Well obviously it has a bunch more features than just being the thin layer, but Docker really is the best thing since sliced bread.

Re: Why Databases Are Not for Docker Containers

#137

Post by anonfunction »

cookiecaper wrote: post #133
wstrange wrote: post #46

Kubernetes StatefulSets [1] are intended to address this kind of use case. They provide stable network identity and stable storage. Kubernetes enables a container to declare the resources that it requires, including things like dedicated CPU and memory requirements. There are still some rough edges (example: how do you set the amount of kernel shared memory you need), but those issues are being ironed out. [1] https:…

StatefulSets are a new feature marked as "beta". They were first available in the newest k8s release, 1.5.

They were known as PetSets beforehand.

Re: Why Databases Are Not for Docker Containers

#138

Post by closeparen »

raarts wrote: post #43

Alright, all you brave people who run databases in docker: where do you store the actual data? - Host mounts? So what happens when a container gets rescheduled to another node? - Docker volumes? What happens when a container gets rescheduled to another node? - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess? - Shared filesystem like NFS, or Ceph? How's the performance fo…

Most are running in a cloud environment with a dynamically attachable block store abstraction, so basically an external SAN.

I'm very interested in approaching this problem on bare metal, though.

Re: Why Databases Are Not for Docker Containers

#139

Post by closeparen »

user5994461 wrote: post #63
mstump wrote: post #36

Earlier quoted context omitted.

Some systems cope with failure better than others. Everything you've said is also true of DB running on top of a uniform linux stack. From my experience (500+ large scale production deployments) this doesn't happen very often. Does it solve all problems? No. Does it make the world a little better and is it better than monolithic single points of failure? Yes.

Some systems fail more often than others.

Sure, but if it fails often enough that you need to prepare to deal with failure, then the number of times you invoke CleanupAfterFailure() doesn't matter so much.

Re: Why Databases Are Not for Docker Containers

#140

Post by ec109685 »

user5994461 wrote: post #115
subway wrote: post #78

Earlier quoted context omitted.

Seems like a straw-man. Why is a process running within a container failing over more frequently than a process running directly on bare metal -- seems like more of a resource/process scheduling issue than anything to do with containers.

> Why is a process running within a container failing over more frequently than a process running directly on bare metal Because the way to change anything in a container is to kill it and restart it. That's a fundamental difference compared to managing/maintaining a database not in a container.

You don't have operate your container like that. If you need to push configuration to it, you can make it writeable.
Post reply on HN