Docker at Spotify

Docker at Spotify
continuousdelivery.uglyduckling.nl

Overall, this has actually worked fairly well- getting certain containers up and running, and having them work together, hasn't been very difficult. But, there's one place that's consistently been a thorn in my side: PostgreSQL.

Currently, there is this article in Docker.io's docs[1] that claims to help you set up a Postgres container, but so far I've not seen it work for me or anyone else I've spoken to about it. It's primarily come from how Postgres is traditionally installed/ran:

1. Service is started. This can either be an 'active' Postgres server, or in the background. This creates a Postmaster.pid file, which ensures no other Postgres servers are running on the machine(I'm not entirely sure why, beyond ease-of-use scenarios).

2. Service is stopped. This deletes the Postmaster.pid file.

In the case of Docker containers, this presents a bit of a problem. If you try to start the Postgres container that way, it will run, but the Postmaster.pid seems to sit outside of it. In the process, once you shut that container down, you actually can't start another without error- the Postmaster.pid stops you.

The other alternative, is having a container running with Postgres as a background process to something else. This can work, but you start having to use services that can keep a process running while letting you access /bin/bash to still get at Postgres.

Neither of these potential solutions allow you to retain data, either.

I've done a lot of poking around, but I haven't had a lot of success in figuring out how to get past this. Those who are using Docker as a Puppet/Chef replacement, especially with Vagrant- what are you usually doing for a database solution?

1. http://docs.docker.io/en/latest/examples/postgresql_service/

As for running Postgres as a background process of something else: I actually recommend this approach. I think running a single process inside a Docker container is a bad idea. By doing that, it does not start important system services such as syslog, and does not allow the container to really work correctly as a proper Unix system should. That is why we have created https://github.com/phusion/baseimage-docker, and we base all our Docker containers on that. The Github page that I linked to explains the problem in detail, explains why a lot of Docker containers are actually not working properly, explains how it solves that problem, and explains how you can use the solution.

    FROM    boothead/saucy

    ENV     DEBIAN_FRONTEND noninteractive

    # Add repository and install PostgreSQL 9.3
    RUN     echo "deb http://apt.postgresql.org/pub/repos/apt/ precise-pgdg main 9.3" >> /etc/apt/sources.list
    RUN     apt-get update
    RUN     apt-get -y --force-yes -f install postgresql-9.3 postgresql-client-9.3 libicu48 wget

    RUN     wget http://www.mirrorservice.org/sites/archive.ubuntu.com/ubuntu//pool/universe/libv/libv8/libv8-3.7.12.22_3.7.12.22-3_amd64.deb
    RUN     dpkg -i libv8-3.7.12.22_3.7.12.22-3_amd64.deb
    RUN     apt-get -y --force-yes -f install postgresql-9.3-plv8


    # Postgres is started now, shut it down and replace config
    RUN     service postgresql stop
    ADD     pg_hba.conf     /etc/postgresql/9.3/main/
    ADD     pg_ident.conf   /etc/postgresql/9.3/main/
    ADD     postgresql.conf /etc/postgresql/9.3/main/

    # Create superuser, tempo db and plv8 language
    ADD     create_user.sh /
    RUN     /bin/bash /create_user.sh
    RUN     rm /create_user.sh

    # Configure for running in container
    EXPOSE  5432
    # VOLUME ["/var/lib/postgresql/9.3/main"]
    CMD     ["/bin/su", "postgres", "-c", "/usr/lib/postgresql/9.3/bin/postgres -D /var/lib/postgresql/9.3/main -c config_file=/etc/postgresql/9.3/main/postgresql.conf"]

This is fine for me to spin up a postgres instance at some known state for development, but it's a bit dissatisfying for managing the data separately from the container. Anyone got any tips for integrating volumes with this?

To split out the Postgres setup and the initial schema setup, one idea might be to have a the CMD for the container run a script to automatically check for a bootstrap file on startup and run it. The bootstrap file could be specified via a volume mount. To use the container with a different dev setup (i.e. a different bootstrap SQL script) you'd simply start up the container with the shared directory pointing somewhere else.

If you want to save the Postgres data files themselves outside of the container you can again do it with volume mounts but you'll need some way to keep track of which goes where. The volume mounts are specified each time you startup the container so you need something to save those. For Dokku specifically there's the PG plugin and it looks like it does exactly this[2]. I haven't use it but I guess it validates the idea.

This seems like a general trend with Docker; it's really cool tech but it's pretty low level so you need something atop it to make usage smoother.

[1]: https://github.com/sehrope/dokku-logging-supervisord

[2]: https://github.com/Kloadut/dokku-pg-plugin/blob/master/comma...