There are ways to make networking easy and performant at the same time without resorting to these user space hacks. Wait for much better products to show up (including one from us).
Weave is kinda slow
61–70 of 164 posts
Re: Weave is kinda slow
#62Earlier quoted context omitted.
Hey some of us believe that ipv6 is a good solution for containers. There are even products that support it (like Calico http://calicoproject.org/ ) and even a few cloud providers (not digital ocean who think that 16 ipv6 addresses is ipv6). If you want a real ip/ipv6 stack for your container, try unikernels.
Thank's for the IPv6 love on Project Calico, Justin. Have you been testing Calico's v6? If so, we'd love to talk to you (disclosure, I'm on the project calico team).
Not yet, planning to give it a go soon...
Re: Weave is kinda slow
#63Earlier quoted context omitted.
Flannel relies on etcd. I had stability issues with v1 of etcd which meant flannel couldn't route packets. Since then, the idea of using an immature SDN atop an immature distributed key value store fills me with dread.
What do you use instead?
I gave up on SDNs and fell back to doing what anyone does without an SDN: published ports to the host interface and advertised the endpoint : to etcd. Note this wasn't with kubernetes but with a similar system. Still reliant on etcd, which I wasn't happy with, but one less cog to go wrong.
Re: Weave is kinda slow
#64Earlier quoted context omitted.
Thank's for the IPv6 love on Project Calico, Justin. Have you been testing Calico's v6? If so, we'd love to talk to you (disclosure, I'm on the project calico team).
Not yet, planning to give it a go soon...
drop me a line when you do. Containers or OpenStack? cdl (at) projectcalico
Re: Weave is kinda slow
#65User-space overlays are slow, Weave more so than flannel. The really interesting data-point for me is that flannel with VXLAN has negligible overhead.
Actually, userspace packet switching can be fast (10GbE linespeed-fast) thanks to approaches like DPDK [1], where a userspace process has zero-copy, direct access to NIC ring buffers. 1 - http://dpdk.org/
The problem is Docker containers expect a system network endpoint, while dpdk gives you a userspace network endpoint.
Re: Weave is kinda slow
#66"Thankfully, the fact that these problems were solved decades ago has not stopped people from coming up with their own solutions, and we now all get to witness the resulting disasters." Spot on
So what are the decades old solutions to these problems?
Thankfully, they are all discussed in the linked article.
Re: Weave is kinda slow
#67Earlier quoted context omitted.
Actually, userspace packet switching can be fast (10GbE linespeed-fast) thanks to approaches like DPDK [1], where a userspace process has zero-copy, direct access to NIC ring buffers. 1 - http://dpdk.org/
The problem with DPDK is (as I understand it) that it doesn't handle multiplexing the connection between multiple cores/processes/containers (i.e. lack of polling makes scheduling hard/impossible). The nice thing about using the Linux kernel as your data plane is that you can still have all your bog standard routing, in addition to this fun VXLAN/GRE/etc stuff. That said, I haven't ever implemented anything using DPD…
It depends on how you use DPDK. If I use it from the container directly to the NIC, you certainly do loose all of the kernel capabilities. However, we believe (but have not tested) that you can use a DPDK virtual interface in the container/vm (memnic or virtio) that connects to the DPDK driver in the kernel, so the path from the container/VM is 0 copy. The kernel then does it's processing, and then, another DPDK path could (potentially) be used to 0 copy the traffic to the NIC (really uncertain about that last stage). Basically, you are just using DPDK to save on the copy cost.
This is all academic until tested, btw. As of yet, we (on Calico) haven't had anyone stand up and say that they need more performance than what the native data-path we use today is capable of delivering.
Re: Weave is kinda slow
#68Earlier quoted context omitted.
Yeah, I feel a little guilty after writing this article, as the speed of the implementation is simply a detail. However, I feel no such guilt in condemning Weave's security. This is a conversation I had with @weave a while ago about their encryption https://twitter.com/lclarkmichalek/status/544882194456776705
This "our project is open source, feel free to submit a patch" dismissal is so passive aggressive. If you mean "fuck you," then just say "fuck you." That said, you shouldn't be saying "fuck you" in the first place: it's rude, it contributes to bad vibes in the OSS community, and it hurts you more than anybody. Try instead something like: "I'm having trouble understanding your argument, do you mind explaining in more…
I don't view this kind of comment as a "fuck you". I think this says more about your outlook than it does theirs.
They're letting you know that they have their own priorities, but they're still receptive to your ideas, which is absolutely fine. What is wrong with that?
Re: Weave is kinda slow
#69I'm not sure I even understand the problem that weave and Docker bridging/NAT solves for real world cases. IP allocation for containers isn't a problem for most networks, is it? Certainly AWS can give you up to 8 IPs per instance, and every datacenter I've ever worked in can give you even more, if you ask. All you have to do is spin up additional virtual NICs with virtual MACs and use DHCP to assign IP addresses to them.
Or is there something fundamental that I don't understand? Please edify me.
Re: Weave is kinda slow
#70Networking is a disaster. Kernel level networking is nice, but that requires access to the kernel which you can't provide in a container. Doing so means containers no longer contain. IPV6 was supposed to solve a lot of this by having an address space so huge you could easily give every vm host a few billion IPs. But nobody uses it.
It is not a disaster. Its possible to build a performant and easy networking solution for containers. Except the vendors are in a hurry to get some hacky software out that takes back the clock 20 years.