Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Getting Started with Docker

serversforhackers.com

Post reply on HN
Print view View original

31–40 of 78 posts

Re: Getting Started with Docker

#31

Post by gtaylor »

stevekemp wrote: post #7
Xdes wrote: post #3

This skips over the hard part: managing docker containers. Poking a hole directly to the container is a leaky abstraction. A reverse proxy like HAProxy or Varnish should be sitting in front of the container. Once you have the reverse proxy setup the next problem that arises is routing to containers based on the domain. Now your HAProxy or Varnish config is going to get bloated and every time you deploy a container th…

I've been thinking along these lines recently, specifically service discovery for front-end load-balancers. Most (all?) of the available reverse proxies will stop sending traffic to a server that is offline, but not discover them. There are solutions such as etcd which you can hook into, or you can write a toy application to use UDP-broadcasts to advertise "Hey I'm http://dev.local.com/ on port 4444", but there isn't…

> ObRelated: Varnish is a beast that few people can configure easily. I'd love to work on a caching reverse proxy that was simple, extensible, and fast.

It doesn't have as many caching-specific bells and whistles as Varnish, but nginx is an excellent reverse proxy with some caching abilities (and simple configuration).

Re: Getting Started with Docker

#32

Post by tomgruner »

Xdes wrote: post #3

This skips over the hard part: managing docker containers. Poking a hole directly to the container is a leaky abstraction. A reverse proxy like HAProxy or Varnish should be sitting in front of the container. Once you have the reverse proxy setup the next problem that arises is routing to containers based on the domain. Now your HAProxy or Varnish config is going to get bloated and every time you deploy a container th…

This is a great point. The initial Docker examples make everything seem easy, but we blew way past our estimated time in integrating docker into our workflow because of the points you mention. I am still happy with the choice to use docker though and our team will be better at server administration in the future.

One thing about this getting started guide is that it recommends the Phusion base image which boots init. That seems to go against the best practices outlined in a recent article by Michael Crosby - http://crosbymichael.com/dockerfile-best-practices-take-2.ht...

Re: Getting Started with Docker

#33

Post by fideloper »

tomgruner wrote: post #32
Xdes wrote: post #3

This skips over the hard part: managing docker containers. Poking a hole directly to the container is a leaky abstraction. A reverse proxy like HAProxy or Varnish should be sitting in front of the container. Once you have the reverse proxy setup the next problem that arises is routing to containers based on the domain. Now your HAProxy or Varnish config is going to get bloated and every time you deploy a container th…

This is a great point. The initial Docker examples make everything seem easy, but we blew way past our estimated time in integrating docker into our workflow because of the points you mention. I am still happy with the choice to use docker though and our team will be better at server administration in the future. One thing about this getting started guide is that it recommends the Phusion base image which boots init.…

Nice, hadn't read those. Thanks! I was wondering about that, but still need a solution for logging, cron jobs, and similar (perhaps running those on the host machine is the answer)

Re: Getting Started with Docker

#34

Post by SEJeff »

Xdes wrote: post #3

This skips over the hard part: managing docker containers. Poking a hole directly to the container is a leaky abstraction. A reverse proxy like HAProxy or Varnish should be sitting in front of the container. Once you have the reverse proxy setup the next problem that arises is routing to containers based on the domain. Now your HAProxy or Varnish config is going to get bloated and every time you deploy a container th…

Aurbnb wrote a set of apps they dub "smart stack" which do the haproxy config for you.

http://nerds.airbnb.com/smartstack-service-discovery-cloud/

Re: Getting Started with Docker

#35

Post by izietto »

vidarh wrote: post #26
izietto wrote: post #9

For non-Paas use cases (for example, a development server with a bunch of projects) I find schroot (1) simpler and more productive. For example, you can use the normal `service stop / service start` instead of writing manually init scripts, and you don't get stuck with sharing directories, which I found extremely tricky with Docker (for example, I couldn't start correctly mysql with supervisor sharing the mysql db di…

Here's one benefit you get with docker: Speed of rebuilds, and ensuring that your build instructions and list of dependencies accurately reflects your actual environment. I basically have a "basic dev setup" container for all my projects now, and each new project sits as sub-directories of a directory on the host that I bind mount into the docker containers. Each project then also has a Dockerfile which adds any proj…

Sure, but speed of rebuilds is important when you rebuild often, which is not my case.

In my case the Dockerfile is easily replaced with a bash set -e script, which has never gave problems to me: deboostrap, share volumes, aptitude install apache/nginx, mysql/postgresql, php/ruby, copy files, predeployment commands like f.e. bundle install, start services.

And you have the advantage of starting services with `service start ...`, instead of reinventing init scripts.

Re: Getting Started with Docker

#36

Post by kevinschumacher »

tonyhb wrote: post #17
fideloper wrote: post #10

Earlier quoted context omitted.

I've had the same issue with MySQL. It's an issue of timing - You install MySQL, and the MySQL data directory has its data/default databases. Then you share the directory with Docker, and the data lib directory is wiped out (the files don't exist on the host machine, after all). Getting it right in an automated way is a Hard Problem™. As of now, I'm keeping data persisted within the Container, which I don't necessari…

Here's a dockerfile setup I wrote for Postgres which uses a 'data container' for the entire Postgres database: https://github.com/codelittinc/dockerfiles/tree/master/postg... The gist of it is that you explicitly tell your DB container that there will be a shared directory on the container at runtime. This allows you to chown the directory before the data container is added. Then, when you're running, use --volumes-f…

An article on that would be great!

Re: Getting Started with Docker

#38

Post by markbnj »

I've been using docker for a couple of months, but we have only just begun experimenting with actual deployment in a test environment on ec2. Right now we use it primarily as configuration/dependency management. We're a small team and it seems to make setup easier, at least so far. Two examples: the first is a log sink container, in which we run redis + logstash. The container exposes the redis and es/kibana ports, and the run command maps these to the host instance. Setting up a new log server means launching an instance, and then pulling and starting the container. The second example is elasticsearch. We have a container set up to have cluster and host data injected into it by the run command, so we pull the container, start it, and it joins the designated cluster. The thing I like about this is the declarative specification of the dependencies, and the ease of spinning up a new instance. As I say, just experimenting so far, and I don't know how optimal all of this is yet, so would love any feedback.

One last quick thought on internal discovery. A method we're playing with on ec2 is to use tags. On startup a container can use a python script and boto to pull the list of running instances within a region that have certain tags and tag values. So we can tag an instance as an es cluster member, for example, and our indexer script can find all the running es nodes and choose one to connect to. We can use other tags to specify exposed ports and other information. Again, just messing around and still not sure of the optimal approach for our small group, but these are some interesting possibilities.

Re: Getting Started with Docker

#39

Post by tomgruner »

fideloper wrote: post #33
tomgruner wrote: post #32

Earlier quoted context omitted.

This is a great point. The initial Docker examples make everything seem easy, but we blew way past our estimated time in integrating docker into our workflow because of the points you mention. I am still happy with the choice to use docker though and our team will be better at server administration in the future. One thing about this getting started guide is that it recommends the Phusion base image which boots init.…

Nice, hadn't read those. Thanks! I was wondering about that, but still need a solution for logging, cron jobs, and similar (perhaps running those on the host machine is the answer)

I am still finding good solutions for those too, and trying to add some concepts to my toolbox like orchestration, service discovery, proxies, data containers, ambassador containers, and so on. It's hard for me to wrap my head around the different recommended ways to use docker compared to my initial expectations.

Re: Getting Started with Docker

#40

Post by zobzu »

VM CAN share binaries/libs/etc (otherwise called files)

also, VMs CAN "share" memory. ie VMs can dedup memory between themselves. On Linux at least.

Not saying docker/lxc and all things namespaces are bad at all - but setting things straight. VMs can do this:)

Checkout KSM for memory "sharing" and any overlay-style file system that is mounted by VMs (this one works exactly the same as when you use namespaces/docker/lxc in fact)

Post reply on HN