Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Getting Started with Docker

serversforhackers.com

Post reply on HN
Print view View original

61–70 of 78 posts

Re: Getting Started with Docker

#61

Post by yblu »

cjbprime wrote: post #56
yblu wrote: post #55

Can someone tell me what's the point of this? (I seriously love to know, not criticizing it.) Why would I need to have docker containers to install stuff on them instead of just installing stuff directly on host? Let's say I develop a new web app, I would install NodeJS, PostgreSQL and such on my machine. Before I deploy the app for the first time, I'll install them in the necessary servers. Now, it looks like I woul…

Imagine you wanted to run two apps on the same host, and they depended on different versions of those components, and you wanted to be able to install the app and its dependencies on a new machine really quickly for scaling reasons, and you wanted an exploit on one of the apps to be difficult to escalate over to the other app from, and you'll start understanding some of the pain points Docker solves. :)

Thank you. Do you normally setup your dev environment inside a container? Or you do development as usual and then when coming to deployment, you deploy your app into the container?

Re: Getting Started with Docker

#62

Post by calgaryeng »

I wish that people would stop writing tutorials on "getting started" with Docker, and actually start writing up examples of how to work with multiple containers, hosts, and linking.

That's the part that I (and I'm sure other beginners) get totally stuck on. Anyone can do docker commit/pull.

Re: Getting Started with Docker

#63

Post by e12e »

yblu wrote: post #60
robszumski wrote: post #57

Earlier quoted context omitted.

Docker containers let you isolate the entire environment for your app. Let's say your running an app on CoreOS in a container that needs python 1.2.3. On your laptop you can build and test the new version of the app that needs python 1.2.4. Once you decide that's ready to go, you can push the new container onto the same CoreOS machine, so it's running both containers. Without the containers, running two versions of p…

Thanks, this is good info. For NodeJS and Rails, I use nvm and rvm, so didn't have any problem with multiple environments. But yeah, I see your point that Docker can help in such scenario or when there's no equivalent of nvm/rvm.

Have a dependency on conflicting libc is a favourite problem that can be difficult to solve without some form of container (vm, chroot or something "in between" like lxc/jails). Another is dependency on different kernel (either major version on same os, or dependency on a different kernel, like freebsd), which docker (by design) doesn't solve.

I don't use ruby much, but it doesn't strike me as very easy to work with/very reliable for production deployment. But that might be me. How well does it handle dependencies on conflicting modules with parts written in c?

Perhaps the most important point is that (when it makes sense) a docker setup might allow easier horizontal scale-out, and or redundancy.

All that said, keeping things simple is generally a good thing. But sometimes adding complexity in one area makes the overall system less complex.

Re: Getting Started with Docker

#64

Post by e12e »

pg_fukd_mydog wrote: post #11

Would it be better to use FreeBSD and their Jails mechanism for all of this?

Joyent would probably claim that kvm+zfs would be best. But if you don't have kernel support for jails, then no, using jails isn't better. It's not an option. Oracle would probably claim solaris zones are better (and arguably, they'd be right).

Jails are (as far as I can tell) great -- but not so great that freebsd didn't include a new hvm assisted hypervisor in freebsd 10 (BSD Hypervisor (bhyve)).

LXC in many ways are *bsd jails for Linux.

Re: Getting Started with Docker

#65

Post by mateuszf »

derefr wrote: post #50
vidarh wrote: post #19

Earlier quoted context omitted.

Update etcd with connection details on container start/stop. Then use a script to watch the appropriate directory in etcd for changes and regenerate the config. Look at "fleet" from CoreOS, and especially their "sidekick" example that uses systemd dependencies to trigger etcd updates: https://coreos.com/docs/launching-containers/launching/launc... though you can certainly do this without fleet too. Then on the haprox…

This is what DNS is for. If you specify your backends by hostname instead of IP, then each time the load-balancer tries to connect to the backend, it'll get a list of A records from its DNS resolver and pick one in a round-robin fashion. Thus, if you have a dynamic DNS server that queries your presence service, it can return exactly the hosts that are up right now as the round-robin set. Right now, if you use SkyDNS[…

Doesn't it mean that every tcp connection would have to query dns at the beginning? What about client dns cache? Also, isn't that expensive in terms of connection time?

Re: Getting Started with Docker

#66

Post by e98cuenc »

MichaelGG wrote: post #53
stevekemp wrote: post #41

Earlier quoted context omitted.

The biggest problem is you cannot use `proxy_cache_purge` unless you pay for the commercial version/fork of nginx. That means you can't expire the cached content by URL.

Doesn't this open-source module do the same thing: http://labs.frickle.com/nginx_ngx_cache_purge/ It seems odd for nginx to try to commercialise such basic parts of the stack where 3rd parties can easily write such functionality.

A nice feature from the commercial nginx purge package is that it lets you purge by prefix. That's a feature that I've not seen in any of the open source purge modules.

If you are hosting data for several users on the same nginx cache and you want to purge only one of them, your only options are to scan the full cache on disk and delete the files that have a key with your prefix, or fork >$1K/year per nginx box for the commercial license.

Re: Getting Started with Docker

#67

Post by FooBarWidget »

bjt wrote: post #49
arianvanp wrote: post #37

Shouldn't "setting up a correct init process" be part of every "getting started with docker?" http://phusion.github.io/baseimage-docker/

No. That guide assumes you're running a bunch of processes in the container (or even a full system). That's not the case at all when you're doing an "application container" that doesn't need its own cron daemon, ssh daemon, etc. Containers can be much leaner than the kind discussed there.

I am one of the authors behind that guide. That guide does not assume that you're running "a bunch of processes" or "even a full system" in the container. Even if you're only running 1 process, it's still highly relevant.

The main point is the Unix process model and how zombie processes work. Things are just not setup properly if your system doesn't handle that properly. And except by using specialized apps (e.g. the my_init system used in baseimage-docker), it just isn't set up properly.

One of the Docker authors, shykes, stated: "In short, regular applications don't expect to be pid 1, and generally speaking they shouldn't."

The other point is to be able to login to the container to perform one-off sysadmin and debugging work. There was lxc-attach for that, but now that Docker supports multiple backends, SSH is the only portable solution that works no matter which Docker backend you use.

Re: Getting Started with Docker

#68

Post by FooBarWidget »

tomgruner wrote: post #32
Xdes wrote: post #3

This skips over the hard part: managing docker containers. Poking a hole directly to the container is a leaky abstraction. A reverse proxy like HAProxy or Varnish should be sitting in front of the container. Once you have the reverse proxy setup the next problem that arises is routing to containers based on the domain. Now your HAProxy or Varnish config is going to get bloated and every time you deploy a container th…

This is a great point. The initial Docker examples make everything seem easy, but we blew way past our estimated time in integrating docker into our workflow because of the points you mention. I am still happy with the choice to use docker though and our team will be better at server administration in the future. One thing about this getting started guide is that it recommends the Phusion base image which boots init.…

I'm one of the authors behind Phusion's baseimage.

Phusion's baseimage does not go against Michael Crosby's best practices. His best practices states not to boot init, and with that he means the normal init. He states that not because it's the init process itself that's a bad idea, but because the normal init performs all kinds of work that is either unnecessary or harmful inside a Docker container. In fact, this is exactly what the Baseimage-docker documentation also states: don't use the normal init.

The Phusion baseimage does not contain a normal init, but a special init that is specifically designed for use in Docker.

Re: Getting Started with Docker

#69

Post by nl »

otterley wrote: post #54
derefr wrote: post #50

Earlier quoted context omitted.

This is what DNS is for. If you specify your backends by hostname instead of IP, then each time the load-balancer tries to connect to the backend, it'll get a list of A records from its DNS resolver and pick one in a round-robin fashion. Thus, if you have a dynamic DNS server that queries your presence service, it can return exactly the hosts that are up right now as the round-robin set. Right now, if you use SkyDNS[…

SkyDNS looks interesting, but it doesn't appear to do any heath checks on the endpoint. I don't want clients to receive an answer to an A record query that contains the IP address of an endpoint that is down. Am I mistaken?

You probably don't want a healthcheck done for every DNS request.

Better to have a healthcheck service doing healthchecking, and modify SkyDNS along with whatever else happens when a service goes down.

Re: Getting Started with Docker

#70

Post by nl »

vidarh wrote: post #24
hootener wrote: post #23

Earlier quoted context omitted.

I'm probably just going to show my ignorance, here...but why doesn't container linking solve this problem? Could you not run multiple docker containers/services behind a single nginx or apache container on a production server? Then the nginx container basically gets one of your public IP addresses, and you use linking to that container to provide it with knowledge of the other running processes' IP addresses (each wi…

I don't like the container linking because it is basically tied to one server without extra complexity. CoreOS' "fleet" ( https://coreos.com/docs/launching-containers/launching/launc... ) gives a cluster-wide solution. But even without using fleet, the overall mechanism is fairly easy to adapt: Either use systemd dependencies like in their example, or have a script that queries docker on each host to spot changes in…

The "extra complexity" needed for multi-machine setups over linking containers is actually pretty minor.

Your services need to read something to get the IP, which ultimately comes from an ENV variable. In the linked container scenario Docker sets that variable. Otherwise you set it manually. That's the only extra complexity.

I was worried about this too, so I tried it out[1]. In this case I have a YAML config file, which can be overridden by ENV variables (which may come from Docker).

This isn't as automatic as CoreOS (eg, no failover etc), but it is a lot less complex.

[1] https://github.com/nlothian/Acuitra/blob/master/services/que...

Post reply on HN