Weave – The Docker Network • Hacker News Board

Re: Weave – The Docker Network

#21

Post by shykes » Tue, Sep 09, 2014, 2:32 PM UTC

jpgvm wrote: post #9

I hope all of these Docker overlay networks start using the in-kernel overlay network technologies soon. User-space promiscuous capture is obscenely slow. Take a look at GRE and/or VXLAN and the kernels multiple routing table support. (This is precisely why network namespaces are so badass btw). Feel free to ping me if you are working on one of these and want some pointers on how to go about integrating more deeply w…

I would love to take you up on that. I want to bake vxlan support into Docker upstream (as an optional plugin, like everything else).

Edit: Hi Joseph! Just realized it was you :)

Re: Weave – The Docker Network

#22

Post by St-Clock » Tue, Sep 09, 2014, 2:35 PM UTC

jfoutz wrote: post #18

lemieux wrote: post #17

Earlier quoted context omitted.

So, basicaly, you're saying that you don't want to mess with that setup and use existing stuff... which you could achieve with everything as long as someone else do the setup for you. I don't see where Docker is better for a developper in this case... providing a vagrant box is exactly the same.

Startup time for a docker container is way way faster than a VM. Also, you could run the exact binary state of production, which is helpful if you run into "works on my machine" types of problems.

I have yet to see real, complex, and distributed applications that share the exact same config in dev and production. I know that having the same versions of system libs in dev and prod can be a problem in some context and docker can help with that, but it's not the only solution and does not take care of the whole landscape (e.g., npm packages.json, pip requirements.txt, etc.).

I totally agree that startup time of a container is far less than a VM, but I don't see how docker "removes all the trouble of running applications that you need for your development: databases, application servers, queues"

You still need to install, configure these services, make sure that the containers can talk to each other in a reliable and secure way, etc.

Re: Weave – The Docker Network

#23

Post by saryant » Tue, Sep 09, 2014, 2:37 PM UTC

Question for weavenetwork: are containers addressable by hostname from other containers? Is there a good way to do that? I didn't see anything about it in the readme.

I suppose service discovery is out-of-scope for this project but having some sort of weave-wide hostsfile would certainly simplify it. Am I misunderstanding the project?

Re: Weave – The Docker Network

#24

Post by greenimpala » Tue, Sep 09, 2014, 2:37 PM UTC

Err can anyone spot the tests in the repo? I cannot.

Re: Weave – The Docker Network

#25

Post by cschneid » Tue, Sep 09, 2014, 2:44 PM UTC

zobzu wrote: post #20

What this really means security wise: http://i.imgur.com/Cko02do.png

Each `W` node in that graphic is a different physical host. So 3 kernels.

Re: Weave – The Docker Network

#26

Post by lemieux » Tue, Sep 09, 2014, 2:46 PM UTC

jfoutz wrote: post #18

lemieux wrote: post #17

Earlier quoted context omitted.

So, basicaly, you're saying that you don't want to mess with that setup and use existing stuff... which you could achieve with everything as long as someone else do the setup for you. I don't see where Docker is better for a developper in this case... providing a vagrant box is exactly the same.

Startup time for a docker container is way way faster than a VM. Also, you could run the exact binary state of production, which is helpful if you run into "works on my machine" types of problems.

Agreed, but my point was that Docker doesn't reduce dev setup time. Give a good vagrant config file to a dev and tell him to do vagrant up and you have the same result as what you're saying. You can replicate production state with vagrant too (and bash scripts if we stretch this) and avoid "works on my machine" problems.

I'm not saying that vagrant > docker. The way I see it, docker is great if your infrastructure is using it all the way. If your prod setup if not dockerized, using docker in dev seems to me counterproductive than spinning up a VM and provisionning it with ansible or puppet to achieve production replication. As @netcraft said, I don't see why I should "change my server architecture" to use docker in dev.

Re: Weave – The Docker Network

#27

Post by TheDong » Tue, Sep 09, 2014, 2:52 PM UTC

wastedhours wrote: post #16

vidarh wrote: post #10

Earlier quoted context omitted.

Nothing is stopping you from running MySQL in Docker on native xfs with aligned partitions: bind-mount whatever partition you want into the container by defining a volume in Docker. This will be persistent, and will survive when you destroy the container. I use this to e.g. share a /home directory between a dozen experimental dev container I use to run my various projects - each container ensures I keep track of exac…

Am intrigued (and, again, showing my current early-stage understanding of LXCs), can you link the same data store container to multiple application containers? As in, have both a beta application and production application pulling data from the same core DB? And do you simply define the container as a volume to ensure it stays persistent? That was the feeling I got from the docs, but again, might just be flagging how…

docker run --name=my-data -v /host/data:/container/data data-container

docker run --volumes-from=my-data app-beta-container

docker run --volumes-from=my-data app-prod-container

That would share the data store, however the real way you'd do this would be....

docker run --name=my-data -v /host/data:/container/data data-image

docker run --volumes-from my-data --name my-database database-image

docker run --link=my-database beta-app

docker run --link=my-database prod-app

Doing --link will allow those two containers to network-communicate and you should only be communicating with your database over the network anyways.

Re: Weave – The Docker Network

#28

Post by saryant » Tue, Sep 09, 2014, 2:56 PM UTC

jfoutz wrote: post #18

lemieux wrote: post #17

Earlier quoted context omitted.

So, basicaly, you're saying that you don't want to mess with that setup and use existing stuff... which you could achieve with everything as long as someone else do the setup for you. I don't see where Docker is better for a developper in this case... providing a vagrant box is exactly the same.

Startup time for a docker container is way way faster than a VM. Also, you could run the exact binary state of production, which is helpful if you run into "works on my machine" types of problems.

At the company I work for, we went through all the trouble of getting our distributed backend application running Vagrant using Chef so that we could have identical local, dev and production environments.

In the end, it's just so slow that nobody uses it locally. Even on a beefy Macbook Pro, spinning up the six VMs it needs takes nearly 20 minutes.

We're looking at moving towards docker, both for local use and production, and so far I'm excited by what I've seen but multi-host use still needs work. I'm evaluating CoreOS at the moment and I'm hopeful about it.

Re: Weave – The Docker Network

#29

Post by shykes » Tue, Sep 09, 2014, 2:57 PM UTC

wastedhours wrote: post #16

vidarh wrote: post #10

Earlier quoted context omitted.

Nothing is stopping you from running MySQL in Docker on native xfs with aligned partitions: bind-mount whatever partition you want into the container by defining a volume in Docker. This will be persistent, and will survive when you destroy the container. I use this to e.g. share a /home directory between a dozen experimental dev container I use to run my various projects - each container ensures I keep track of exac…

Am intrigued (and, again, showing my current early-stage understanding of LXCs), can you link the same data store container to multiple application containers? As in, have both a beta application and production application pulling data from the same core DB? And do you simply define the container as a volume to ensure it stays persistent? That was the feeling I got from the docs, but again, might just be flagging how…

Yes, you can definitely share volumes between any number of containers, it's a common usage pattern.

You don't need to make a container persistent: Docker, by design, will never remove anything unless you explicitly ask it to. If you want to separate the lifecycle of a directory within your container, so that it stays behind after you explicitly remove the container, or to share it between containers - that's when volumes are useful.

Re: Weave – The Docker Network

#30

Post by t0mas88 » Tue, Sep 09, 2014, 3:35 PM UTC

This looks like a great idea. For me this was a missing piece two months ago when playing with Docker.

However I have strong doubts about the network performance, not only the overhead of the UDP encapsulation (that should be quite small), but mostly the capturing of packets with pcap and then handling them in user-mode. Looks like a lot of context-switches, copying and parsing with non-optimal code paths. Are there any benchmarks available?

My feeling is that this will consume large amounts of CPU for moderate network loads and thus be unusable with most NoSQL kind of systems that benefit from clustering across hosts?