Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Databases Are Not for Docker Containers

myopsblog.wordpress.com

Post reply on HN
Print view View original

31–40 of 184 posts

Re: Why Databases Are Not for Docker Containers

#31

Post by user5994461 »

mstump wrote: post #11

Part of the problem is that you're using databases that can't cope with failure. In large scale production systems things fail all the time. If you've got tech that can cope with failure it's not an issue. Additionally, Docker is pretty handy when you're attempting to manage clusters consisting of thousands of nodes. In that instance enforcing best practices, automating workflows, scaling teams, auditing and preventi…

There is no tech in the universe that can cope with cascading failures, like ALL instances of a docker container crashing on ALL hosts one by one in quick succession. This usually happen because an app hits an unexpected bug in the docker disk or the docker network stack and this is the major source of concerns I have with Docker.

Re: Why Databases Are Not for Docker Containers

#34

Post by tracker1 »

I make an exception to this.. that would be something like Redis, or memcached as a localized (or perhaps sharded) cache cluster serving the systems running on those docker hosts.

One of the things that bug me in terms of cache as a service in AWS/Azure etc, it you're dedicating compute nodes to mostly use memory... the big win for memcached early on was utilizing unused memory on existing systems. You lose that when you don't have the caching services on the same nodes as compute/data/etc.

Re: Why Databases Are Not for Docker Containers

#35

Post by donpark »

ender7 wrote: post #17

I still can't really figure out what a container is. Every time I think of a use case for one, I read something like this which says that's a terrible idea. The use-case I need solved most often is the following: Create a standalone "server" that accepts and responds to network traffic, has some way to store data, and whose dependencies (i.e. system packages, frameworks, etc) I can manage independently of any of the…

A container is just changes you make to a file system to get something running. To run a container, Docker applies changes listed in its image then executes its entry point. When it stops, changes to file systems are preserved but memory is not. When removed, everything is gone so next time it runs you're starting fresh from original image.

Magic with container are: 1) those changes are only visible from code running in the container, and 2) changes can be layered on top of each other (FROM in Dockerfile).

Re: Why Databases Are Not for Docker Containers

#36

Post by mstump »

user5994461 wrote: post #31
mstump wrote: post #11

Part of the problem is that you're using databases that can't cope with failure. In large scale production systems things fail all the time. If you've got tech that can cope with failure it's not an issue. Additionally, Docker is pretty handy when you're attempting to manage clusters consisting of thousands of nodes. In that instance enforcing best practices, automating workflows, scaling teams, auditing and preventi…

There is no tech in the universe that can cope with cascading failures, like ALL instances of a docker container crashing on ALL hosts one by one in quick succession. This usually happen because an app hits an unexpected bug in the docker disk or the docker network stack and this is the major source of concerns I have with Docker.

Some systems cope with failure better than others. Everything you've said is also true of DB running on top of a uniform linux stack. From my experience (500+ large scale production deployments) this doesn't happen very often.

Does it solve all problems? No. Does it make the world a little better and is it better than monolithic single points of failure? Yes.

Re: Why Databases Are Not for Docker Containers

#37

Post by metaphorm »

NikolaeVarius wrote: post #19
ender7 wrote: post #17

I still can't really figure out what a container is. Every time I think of a use case for one, I read something like this which says that's a terrible idea. The use-case I need solved most often is the following: Create a standalone "server" that accepts and responds to network traffic, has some way to store data, and whose dependencies (i.e. system packages, frameworks, etc) I can manage independently of any of the…

There is nothing special about containers to really understand Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. You can run multiple containers on a single VM in the same way you can run multiple VMs on a single host. Ideally a container should be stateless. If a container crashes, you should be able to bring it up again without anything actually caring that it is technically a…

> Containers are a lightweight way of sandboxing a process. Think a level lower than a VM.

can you go into a little more depth? my understanding of a VM is that it installs the OS in a dedicated memory partition, and allocates hardware resources separately from that of the host machine, such that resource contention between host and VM never happens. the VM allocated resources just go dark for the host machine while the VM is running.

what is a lower level than that? I've understood containers to be thin wrappers around VMs, which would make them higher level, not lower level. do I have this wrong?

Re: Why Databases Are Not for Docker Containers

#38

Post by tracker1 »

derefr wrote: post #25

> But what about Configuration Management systems? They’re designed to solve this kind of routine by running one command. The problem with this for most of the developers you see praising containers, is that with a containerized setup, you've already got the rest of your deployment process down to `docker service update --image myorg/myservice:1.3.0 myservice` (And, in fact, maybe you're even running that code agains…

This is why I appreciate DBaaS offerings... It makes more sense to run DBs at least closer to the hardware, outside containers, but most developers don't want to be DBAs, so it's better to pay for someone that has all the maint/update scripts written, and not have to deal with many of those issues.

Re: Why Databases Are Not for Docker Containers

#39

Post by user5994461 »

fabian2k wrote: post #4

This article mentions offhand that the storage drivers are unreliable, even for data volumes. Is that actually the case? Is there a serious risk that a database will be corrupted by a container crash, as the article claims? A regular crash of the computer should not be able to corrupt a database, is a container more dangerous in this regard?

You'll want to read this, the links at the end and the comments: https://thehftguy.com/2016/11/01/docker-in-production-an-his...

There are major issues registered with AUFS, Overlay, Overlay2 and BTRFS. It's ridiculous.

Re: Why Databases Are Not for Docker Containers

#40

Post by tracker1 »

metaphorm wrote: post #37
NikolaeVarius wrote: post #19

Earlier quoted context omitted.

There is nothing special about containers to really understand Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. You can run multiple containers on a single VM in the same way you can run multiple VMs on a single host. Ideally a container should be stateless. If a container crashes, you should be able to bring it up again without anything actually caring that it is technically a…

> Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. can you go into a little more depth? my understanding of a VM is that it installs the OS in a dedicated memory partition, and allocates hardware resources separately from that of the host machine, such that resource contention between host and VM never happens. the VM allocated resources just go dark for the host machine while…

Containers aren't VMs... they're more of a sandboxed model of execution with a virtual network/disk abstraction. But lower/higher you are right, VM is probably lower-level abstraction of an entire system.

But a container can be a single executable, it doesn't have to be an entire OS structure. For example, a lot of the go app containers are just the single executable by itself as a default. Many will rely on a debian/ubuntu base as they want other systems to work. This is mainly because of shared environments/libraries that some non-dockerized systems need, but isn't a requirement.

Post reply on HN