I could not get into the site, so from archive:
https://web.archive.org/web/20230214134248/https://rob-turne...
A little bit over my head, but an interesting read.
Vxlan over WireGuard (On OpenBSD)
41–50 of 73 posts
Re: Vxlan over WireGuard (On OpenBSD)
#42I took this to extremes last year: I used it to run MAAS from Australia to sweden (which requires layer 2). Granted I used tailscale to make the WireGuard part even easier, but it was a lot of fun.
https://medium.com/@antongslismith/bare-metal-cloud-provisio...
Re: Vxlan over WireGuard (On OpenBSD)
#43This is fun, but applications requiring L2 adjacency do it to limit latency/distance. Creating a L2 domain between here and the moon, what are you gonna use it for? Certainly not anything other than fun.
Indeed check out my other post here - and it definitely was fun!
Re: Vxlan over WireGuard (On OpenBSD)
#44This can cause massive packet fragmentation. I'd be most interested in the performance degradation due to the L2 encapsulation. Are there any benchmarks available for this kind of project?
That's what I was thinking, unless you have jumbo frames you're going to have a hard time stuffing ethernet frames into IP payloads. Does Vxlan mitigate this somehow?
Re: Vxlan over WireGuard (On OpenBSD)
#45IP over Ethernet over VXLAN over UDP over IP over WireGuard over UDP over IP over Ethernet… sigh OpenBSD does support both routing domains and multiple routing tables and includes multiple routing daemons in the base system. I would recommend to the author to stop hacking at the keyboard, grab whatever not to structured visualisation tool works for them (e.g. a whiteboard, a block of paper, a random drawing app, Visi…
Using vxlan you can also connect L3 networks, not just L2 networks. i.e., virtualize an L3 network
Wireguard virtualizes L3 out of the box.
Re: Vxlan over WireGuard (On OpenBSD)
#46IP over Ethernet over VXLAN over UDP over IP over WireGuard over UDP over IP over Ethernet… sigh OpenBSD does support both routing domains and multiple routing tables and includes multiple routing daemons in the base system. I would recommend to the author to stop hacking at the keyboard, grab whatever not to structured visualisation tool works for them (e.g. a whiteboard, a block of paper, a random drawing app, Visi…
vMotion needs L2 adjacency to make live migrating VMs easy. Some software rely heavily on broadcast discovery messages and are thus designed for LAN usage not Internet connectivity but businesses try to stuff a square peg into a round hold.
Re: Vxlan over WireGuard (On OpenBSD)
#47This can cause massive packet fragmentation. I'd be most interested in the performance degradation due to the L2 encapsulation. Are there any benchmarks available for this kind of project?
I've tunneled VXLAN over Wireguard on Linux. In my setup, my WAN's MTU was 1500 bytes, and my Wireguard tunnel's MTU was 1550, with the VXLAN's MTU being 1500. Surprisingly, traffic and iperf3 tests going over the VXLAN had much better throughput than traffic going directly over the Wireguard connection. IIRC, over the VXLAN, I was pulling ~800Mbps over the VXLAN/WG setup with iperf3.
Where this would fall apart is if there are firewalls in between that silently drop UDP fragments. In a case like that, it may be necessary to do VXLAN/Wireguard/Wireguard to conceal the fragmented packets with MTUs of 1500/1550/1440 respectively, assuming IPv4 and WAN MTU of 1500. I bet this would come with a significant performance hit though.
Re: Vxlan over WireGuard (On OpenBSD)
#48IP over Ethernet over VXLAN over UDP over IP over WireGuard over UDP over IP over Ethernet… sigh OpenBSD does support both routing domains and multiple routing tables and includes multiple routing daemons in the base system. I would recommend to the author to stop hacking at the keyboard, grab whatever not to structured visualisation tool works for them (e.g. a whiteboard, a block of paper, a random drawing app, Visi…
I end up having to run basically this very setup (on OpenBSD, too) because I have a customer who has a Novell NetWare 5 setup and runs IPX only. Bad times.
NetWare 5 can do IP fine.
I used to run a cluster of DNS/DHCP servers that were the first on site to run 5. The rest were 4.11 until we binned them for 6. Three cream coloured Compaq 3U lumps.
Re: Vxlan over WireGuard (On OpenBSD)
#49Just do static routes or BGP over Wireguard. Simpler, scalable, less error prone.
"Here's how I did this thing." "You don't need to do that thing." How do you know?
I've been in or very close to networking for a long time. Here's a certainty: there's lots of ways to do anything. Here's a corollary to that: someone will be sure to tell you how bad any choice you make is.
Re: Vxlan over WireGuard (On OpenBSD)
#50IP over Ethernet over VXLAN over UDP over IP over WireGuard over UDP over IP over Ethernet… sigh OpenBSD does support both routing domains and multiple routing tables and includes multiple routing daemons in the base system. I would recommend to the author to stop hacking at the keyboard, grab whatever not to structured visualisation tool works for them (e.g. a whiteboard, a block of paper, a random drawing app, Visi…
Just the grateful that nobody has tried to wrap the entire thing in JSON over HTTP yet! I wouldn't be surprised if we get Wireguard over websockets for "enterprise" applications soon.
Sometimes you just need an L2 tunnel. Most of the time you don't, but when you do, you do. For example, if you use IPv6 over SLAAC in a private network, you'll need to route NDP.
In the rare cases that you do need an L2 tunnel between two different locations, you probably want some kind of authorisation and authentication of the traffic to prevent injection/spoofing attacks and to make life just a but harder for the NSA (Google's use of HTTP was one way the NSA managed to tap connections that were otherwise encrypted by HTTPS). After all, this isn't just any traffic, these are internal subnets.
In terms of authorised traffic, Wireguard is quite lightweight and foolproof. Perhaps IPSec is even more lightweight but it's a pain to set up. The alternative would be to wrap all internal network traffic in an encrypted protocol and set up the necessary whitelists in the upstream ISPs.
The impact of such layering depends on the network connection between the data centers. If you can get jumbo packets across, fragmentation won't be a problem at all. If you run your own fiber between data centers, there's basically no downside until you're reaching very high saturation network saturation.