Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Databases Are Not for Docker Containers

myopsblog.wordpress.com

Post reply on HN
Print view View original

41–50 of 184 posts

Re: Why Databases Are Not for Docker Containers

#41

Post by jcahill84 »

tinco wrote: post #21
Bombthecat wrote: post #14

Earlier quoted context omitted.

Wow, three years ago? Balls of steel!

People make it seem as if Docker is some bleeding edge magical technology, but in reality its most useful features are just thin wrappers around stable linux kernel features and some nice automation. We have also been running databases in Docker (on the tb scale though) for around 3 years, we had the odd issue here and there, but nothing terrible and certainly nothing fundamental or resulting in data loss. If your da…

This is one of the biggest problems with people hating on docker... They don't understand what it is and actually does. Great comment!

Re: Why Databases Are Not for Docker Containers

#42

Post by jinjin2 »

We are considering to use Realm to replicate our data out to all our docker instances. They would essentially work like databases local to each instance, and if the instance dies we simply spin up another one which will replicate out the same data again.

So far it is only at the experimentation stage for us, but it looks very promising. It is almost like having the ultimate cache (no network latency) right within each instance.

Re: Why Databases Are Not for Docker Containers

#43

Post by raarts »

Alright, all you brave people who run databases in docker: where do you store the actual data?

- Host mounts? So what happens when a container gets rescheduled to another node?

- Docker volumes? What happens when a container gets rescheduled to another node?

- External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess?

- Shared filesystem like NFS, or Ceph? How's the performance for you? And the filesystem itself runs outside of your docker cloud I guess?

- In the containers themselves? So you probably run some clustered database. So what if disaster strikes and all your containers go down? And get moved around?

Also, databases in many cases need to be tuned for performance. How do you do that in a cloud?

Maybe most of you are not running container scheduling. Which really is only taking containers half way.

Re: Why Databases Are Not for Docker Containers

#44

Post by user5994461 »

outside1234 wrote: post #33

Of course the counter example of this is Google, which, as I understand it, runs everything on containers. It seems like if its good enough for Google, then its good enough for the rest of us.

Google runs NOTHING on Docker. They have their internal secret proprietary container technology.

The concept of containers is fine. The troubles lies in the implementation that is available.

Re: Why Databases Are Not for Docker Containers

#45

Post by NikolaeVarius »

metaphorm wrote: post #37
NikolaeVarius wrote: post #19

Earlier quoted context omitted.

There is nothing special about containers to really understand Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. You can run multiple containers on a single VM in the same way you can run multiple VMs on a single host. Ideally a container should be stateless. If a container crashes, you should be able to bring it up again without anything actually caring that it is technically a…

> Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. can you go into a little more depth? my understanding of a VM is that it installs the OS in a dedicated memory partition, and allocates hardware resources separately from that of the host machine, such that resource contention between host and VM never happens. the VM allocated resources just go dark for the host machine while…

I'm doing a bit of a simplification here so please someone correct me if I'm not saying this correctly

Every single instance of a VM has its own Kernel. When a VM boots up, it gets allocated a portion of hardware and boots up a kernel and allocates memory to itself. VMs each are isolated from each other in that they don't share resources and each VM is free to do whatever it wants to do with the hardware it is given. Like you said, to the host machine, that hardware is no longer available for any other VM to use.

For Containers, they all live on a SINGLE kernel. They share resources across each other and the Kernel handles the multiple processes much like it would handle any other multithreaded process.

If you have 3 VMs that all require a specific set of resources to run an application, you need 3x that hardware. This is not true for containers. You can get away with less because the containers will share the resources that the kernel as access to.

I call them "lower level" in the sense that they do so much less than VMs. You CAN use containers as a VM in that a container can boot an entire Kernel, but generally you don't do this.

Re: Why Databases Are Not for Docker Containers

#46

Post by wstrange »

Kubernetes StatefulSets [1] are intended to address this kind of use case. They provide stable network identity and stable storage.

Kubernetes enables a container to declare the resources that it requires, including things like dedicated CPU and memory requirements. There are still some rough edges (example: how do you set the amount of kernel shared memory you need), but those issues are being ironed out.

[1] https://kubernetes.io/docs/tutorials/stateful-application/ba...

Re: Why Databases Are Not for Docker Containers

#48

Post by dgfgfdagasdfgfa »

tracker1 wrote: post #38
derefr wrote: post #25

> But what about Configuration Management systems? They’re designed to solve this kind of routine by running one command. The problem with this for most of the developers you see praising containers, is that with a containerized setup, you've already got the rest of your deployment process down to `docker service update --image myorg/myservice:1.3.0 myservice` (And, in fact, maybe you're even running that code agains…

This is why I appreciate DBaaS offerings... It makes more sense to run DBs at least closer to the hardware, outside containers, but most developers don't want to be DBAs, so it's better to pay for someone that has all the maint/update scripts written, and not have to deal with many of those issues.

Sure, but then you have other issues with black box databases:

1. Lack of control over configuration and tuning of database.

2. Little control over semantic upgrades, i.e. security updates should be automatic, but one breaking compatibility should not be, but MAY be necessary or desired, so it needs to be possible to upgrade.

3. Failover isn't easy, and the way of handling it is often app- and load-specific.

Granted, this won't matter to many people, but there are a lot of reasons not to entirely outsource database concerns from dev.

Re: Why Databases Are Not for Docker Containers

#50

Post by bdcravens »

outside1234 wrote: post #33

Of course the counter example of this is Google, which, as I understand it, runs everything on containers. It seems like if its good enough for Google, then its good enough for the rest of us.

> It seems like if its good enough for Google, then its good enough for the rest of us.

Sure, if you have Google's budget and scale.

Post reply on HN