[dead]
OpenBSD: PF queues break the 4 Gbps barrier
31–40 of 74 posts
Re: OpenBSD: PF queues break the 4 Gbps barrier
#32Earlier quoted context omitted.
The article is about allowing bandwidth restrictions in bytes/second that are larger than 2³²-1, not about how fast pf can filter packets. I guess few people with faster ports felt the need to limit bandwidth for a service to something that’s that large. FTA: “OpenBSD's PF packet filter has long supported HFSC traffic shaping with the queue rules in pf.conf(5). However, an internal 32-bit limitation in the HFSC servi…
> silently wrap around, producing incorrect and unpredictable Now I'm more scared to use OpenBSD than I was a minute before. I strongly prefer software that fails loudly and explicitly.
Yeah, that's pretty appalling.
Regardless of how good the philosophy of something is, if it's as niche and manpower constrained as OpenBSD is then it's going to accumulate problems like this.
Re: OpenBSD: PF queues break the 4 Gbps barrier
#33[dead]
Re: OpenBSD: PF queues break the 4 Gbps barrier
#34In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…
A lot of the time once you get into multi-gig+ territory the answer isn't "make the kernel faster," it's "stop doing it in the kernel." You end up pushing the hot path out to userland where you can actually scale across cores (DPDK/netmap/XDP style approaches), batch packets, and then DMA straight to and from the NIC. The kernel becomes more of a control plane than the data plane. PF/ALTQ is very much in the traditio…
pushing the hot path out to userland where you can actually scale across cores
What sort of kernel do you have which can't scale across cores?
Re: OpenBSD: PF queues break the 4 Gbps barrier
#35In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…
A lot of the time once you get into multi-gig+ territory the answer isn't "make the kernel faster," it's "stop doing it in the kernel." You end up pushing the hot path out to userland where you can actually scale across cores (DPDK/netmap/XDP style approaches), batch packets, and then DMA straight to and from the NIC. The kernel becomes more of a control plane than the data plane. PF/ALTQ is very much in the traditio…
The big things to avoid are crossing the user/kernel divide and communication across cores.
Staying in the kernel is approximately the same as bypassing the kernel (caveats apply); for a packet filtering / smoothing use case, I don't think kernel bypass is needed. You probably want to tune NIC hashing so that inbound traffic for a given shaping queue arrives in the same NIC rx queue; but you probably want that in a kernel bypass case as well. Userspace is certainly nicer during development, as it's easier to push changes, but in 2026, it feels like traffic shaping has pretty static requirements and letting the kernel do all the work feels reasonable to me.
Otoh, OpenBSD is pretty far behind the curve on SMP and all that (I think their PF now has support for SMP, but maybe it's still in development?; I'd bet there's lots of room to reduce cross core communication as well, but I haven't examined it). You can't pin userspace cores to cpus, I doubt their kernel datastructures are built to reduce communications, etc. Kernel bypass won't help as much as you would hope, if it's available, which it might not be, because you can't control the userspace to limit cross core communications.
Re: OpenBSD: PF queues break the 4 Gbps barrier
#36Earlier quoted context omitted.
> Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere Half the problem is lack of proper drivers. I love OpenBSD but all the fibre stuff is just a bit half-baked. For a long time OpenBSD didn't even have DOM (light-level monitoring etc.) exposed in its 1g fibre drivers. Stuff like that automatically kills off OpenBSD as a choice for datacentres where DOM stats are a non-negot…
How much work is it to port drivers between Free and Open BSD?
> How much work is it to port drivers between Free and Open BSD?
IIRC there are two problems at play:
First, I'm not a C coder so this is a bit above my pay-grade, but from what little I do remember about the subject, the problem relates to the OpenBSD requirement to adopt their security mechanisms such as pledge, unveil and strlcpy. IIRC the OpenBSD compiler is also (unsurprisingly !) more anal about stack protector W^X etc. So the porting process is perhaps more time-consuming and low-level than it might otherwise be on other porting projects.
Second, the licensing thing might come into it. OpenBSD has a high preference to most-permissive, and so things like GPL-licensed origins might not be acceptable. IIRC FreeBSD is a little more relaxed within reason ? And when you're working with network cards I would think that is perhaps hard to avoid to some extent if you're relying on certain bits being ultimately derived from Intel chipsets or whatever.
I'm open to correction by those more knowledgable than me on porting intricacies. ;)
Re: OpenBSD: PF queues break the 4 Gbps barrier
#37Earlier quoted context omitted.
How much work is it to port drivers between Free and Open BSD?
> How much work is it to port drivers between Free and Open BSD? IIRC there are two problems at play: First, I'm not a C coder so this is a bit above my pay-grade, but from what little I do remember about the subject, the problem relates to the OpenBSD requirement to adopt their security mechanisms such as pledge, unveil and strlcpy. IIRC the OpenBSD compiler is also (unsurprisingly !) more anal about stack protector…
The difficulty of porting NIC drivers is probably not in differences in the userland API; kernel drivers don't likely pledge anything. But OpenBSD and FreeBSD diverged a long time ago, and I'd be surprised if their kernel APIs are very close anymore. How to detect and interface with devices is probably a bit different, and rx/tx packets will be different too.
I think most of the vendor supplied NIC drivers in FreeBSD are BSD licensed, so that shouldn't be an issue. I checked Intel, Melanox (now NVidia), Cavium/QLogic/Broadcom, Solarflare. The realtek driver in the tree is BSD licensed but not vendor provided; the vendor driver in ports is also BSD licensed. I'm not sure if there's a datacenter ethernet provider with in kernel drivers I missed; but I don't think license is a problem here either --- anyway you could ship a driver module out of tree if it was.
Re: OpenBSD: PF queues break the 4 Gbps barrier
#38Earlier quoted context omitted.
> silently wrap around, producing incorrect and unpredictable Now I'm more scared to use OpenBSD than I was a minute before. I strongly prefer software that fails loudly and explicitly.
Yeah, that's pretty appalling. Regardless of how good the philosophy of something is, if it's as niche and manpower constrained as OpenBSD is then it's going to accumulate problems like this.
I actually think this isn't even surprising from OpenBSD philosophically. They still subscribe to the Unix philosophy of old, moreso than FreeBSD and much much more than Linux.
That is, "worse is better" and it's okay to accept a somewhat leaky abstraction or less helpful diagnostics if it simplifies the implementation.
This is why `ed` doesn't bother to say anything but "?" to erroneous commands. If the user messes up, why should it be the job of the OS to handhold them? Garbage in, garbage out. That attitude may seem out of place today but consider that it came from a time when a program might have one author and 1-20 users, so their time was valued almost equally.
Re: OpenBSD: PF queues break the 4 Gbps barrier
#39In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…
One thing could also be that by the time you have 10GE uplinks, shaping is not as important. When we had 512kbit links, prioritizing VOIP would be a thing, and for asymmetric links like 128/512kbit it was prudent to prioritize small packets (ssh) and tcp ACKs on the outgoing link or the downloads would suffer, but when you have 5-10-25GE, not being able to stick an ACK packet in the queue is perhaps not the main issu…
[flagged]
Re: OpenBSD: PF queues break the 4 Gbps barrier
#40In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…
Isnt OpenBSD mainly used for security testing or do I have it wrong? Would be surprised if it was used in production datacenter networking hardware at all. Seems like most people would use one of the proprietary implementations (which likely would include specific written drivers for that hardware) or something like FreeBSD