Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Databases Are Not for Docker Containers

myopsblog.wordpress.com

Post reply on HN
Print view View original

51–60 of 184 posts

Re: Why Databases Are Not for Docker Containers

#51

Post by novembermike »

metaphorm wrote: post #37
NikolaeVarius wrote: post #19

Earlier quoted context omitted.

There is nothing special about containers to really understand Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. You can run multiple containers on a single VM in the same way you can run multiple VMs on a single host. Ideally a container should be stateless. If a container crashes, you should be able to bring it up again without anything actually caring that it is technically a…

> Containers are a lightweight way of sandboxing a process. Think a level lower than a VM. can you go into a little more depth? my understanding of a VM is that it installs the OS in a dedicated memory partition, and allocates hardware resources separately from that of the host machine, such that resource contention between host and VM never happens. the VM allocated resources just go dark for the host machine while…

Basically, a VM pretends it has a cpu and a disk and ram, a container pretends that it has a root directory and ports and all of that stuff. Like always it's a leaky abstraction and isn't usually sufficient for security purposes (ie. two containers on the same system might be able to talk to each other if you abuse the system) but it's good enough for most purposes and provides much better resource utilization than a bunch of VMs.

Re: Why Databases Are Not for Docker Containers

#52

Post by Jdam »

raarts wrote: post #43

Alright, all you brave people who run databases in docker: where do you store the actual data? - Host mounts? So what happens when a container gets rescheduled to another node? - Docker volumes? What happens when a container gets rescheduled to another node? - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess? - Shared filesystem like NFS, or Ceph? How's the performance fo…

Host mounts, because they are not rescheduled. If you say that no rescheduling only "takes containers half way", you're not fully aware of how containers are used nowadays. In my company for ex, reproducibility, immutability and a uniform deployment process is key.

Re: Why Databases Are Not for Docker Containers

#53

Post by user5994461 »

mentat wrote: post #18
Minikloon wrote: post #10

Earlier quoted context omitted.

Chicken and egg. Is that other instance containerized?

Decoupled failure modes, unless you think there's a global "fail all docker data stores".

There are major cascading failures with Docker.

For starter, failure modes are not decoupled at all. If one db hits a race condition in the storage driver on one instance, it's usually 100% guarantee that all other instances are sensitive to the same bug and it will happen sooner or later.

In practice, occurrences of a bug are highly correlated and usually happen in batch.

Re: Why Databases Are Not for Docker Containers

#54

Post by XorNot »

This article is terrible. It's a lot of wishy-washy explanations devoid of technical detail - because their isn't a technical explanation or justification for this list.

I've run extensive benchmarks of Hadoop/HBase in Docker containers, and there is no performance difference. There is no stability difference (oh a node might crash? Welcome to thing which happens every day across a 300 machine cluster).

Any clustered database setup should recover from failed nodes. Any regular relational database should be pretty close to automated failover with replicated backups and an alert email. Containerization doesn't make this better or worse, but it helps a lot with testing and deployment.

Re: Why Databases Are Not for Docker Containers

#55

Post by wstrange »

raarts wrote: post #43

Alright, all you brave people who run databases in docker: where do you store the actual data? - Host mounts? So what happens when a container gets rescheduled to another node? - Docker volumes? What happens when a container gets rescheduled to another node? - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess? - Shared filesystem like NFS, or Ceph? How's the performance fo…

Docker alone does not solve these problems. You need something like Kubernetes

>- Host mounts? So what happens when a container gets rescheduled to another node?

Persistent Volumes and Claims. Kube manages moving containers, and the associated storage.

> - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess?

Specify a storage class on the persistent volume claim. Most providers (Google, AWS, Azure,..) support SSD

> So what if disaster strikes and all your containers go down? And get moved around?

You let Kube handle rescheduling containers onto new nodes.

Re: Why Databases Are Not for Docker Containers

#56

Post by eicnix »

XorNot wrote: post #54

This article is terrible. It's a lot of wishy-washy explanations devoid of technical detail - because their isn't a technical explanation or justification for this list. I've run extensive benchmarks of Hadoop/HBase in Docker containers, and there is no performance difference. There is no stability difference (oh a node might crash? Welcome to thing which happens every day across a 300 machine cluster). Any clustered…

Hi XorNot,

did you publish your findings anywhere?

Re: Why Databases Are Not for Docker Containers

#57

Post by Bombthecat »

jcahill84 wrote: post #41
tinco wrote: post #21

Earlier quoted context omitted.

People make it seem as if Docker is some bleeding edge magical technology, but in reality its most useful features are just thin wrappers around stable linux kernel features and some nice automation. We have also been running databases in Docker (on the tb scale though) for around 3 years, we had the odd issue here and there, but nothing terrible and certainly nothing fundamental or resulting in data loss. If your da…

This is one of the biggest problems with people hating on docker... They don't understand what it is and actually does. Great comment!

Oh hell no, not hating docker! I love docker ( I'm the multiplier in my company and teaching docker,and doing presentations)

But putting pb of data in docker three years ago is just insane.

There are still things not clear right now,which are not stable ( not in unstable destroying data) but unstable in: if something goes wrong with your data you need to dig deep and find out what's going on. If stuff changes every few month. You will have a hard time.

Also, enterprise concerns like:

Will there be a docker standard from Google and Facebook? Will AUfs be Version 4 or scrapped again? Will the composer format stay? How to manage runtime upgrades? Ie syncronize 1000 dockers with one version update than another 1000 with another software. So that one update depends on another. Etc etc etc.

Three years ago it just wasn't there. Two years ago I would have said: "good enough" to play around. One year ago it started to get really interesting for enterprise.

That's why I say: you handle pb of data,with docker,three years ago?

Balls of steel.

Re: Why Databases Are Not for Docker Containers

#58

Post by banachtarski »

raarts wrote: post #43

Alright, all you brave people who run databases in docker: where do you store the actual data? - Host mounts? So what happens when a container gets rescheduled to another node? - Docker volumes? What happens when a container gets rescheduled to another node? - External SAN? Congratulations on your budget. That's not easily doable in public cloud I guess? - Shared filesystem like NFS, or Ceph? How's the performance fo…

I think you have some pretty big misconceptions which accounts for at least half your skepticism. Most databases are designed to restore from information that's stored on disk. If you use host volumes, there is no concept of a container "moving around." Just being dead or alive. If a container starts on a host that was previously running, that container now is the previous one.

Re: Why Databases Are Not for Docker Containers

#59

Post by xen2xen1 »

Docker is an app store. Once you reach that point of understanding everything gets easier. On your Windows box or Ubuntu box you can just install whatever you want. Docker is more like IOS or android. When was the last time you edited an .ini file on android? It works, it doesn't, take it or leave it. Or you can clear the local data or reinstall. Not much else. Docker is the same way, at least by intent. No wonder they don't want you to store data on it, if you have a SQL database running on android how much would you expect out of it? Would you really expect it to be persistent? It's easy to install, that's the point, but it takes away a lot of freedom just like the app store(s).

Re: Why Databases Are Not for Docker Containers

#60

Post by ofrasergreen »

As others have mentioned, Docker doesn't really do any magic which might harm the smooth running of a database, but just leverages process isolation built into the Linux kernel and provides a convenient way to package and distribute bundles of software. In the case of a database, the former can be handy any time you want to run a database on a host where you want to run other software too. As for the latter, being able to run the same configuration locally as on production servers, to replicate configuration over many nodes in a cluster, to distil both configuration and software into atomic units are as advantageous for databases as for any other software.

At my company we have been using PostgreSQL on Docker for over two years without and have been sufficiently satisfied with the results that we're in the process of turning out setup into a product in its own right: http://containable.co/

Post reply on HN