Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenBSD: PF queues break the 4 Gbps barrier

undeadly.org

Post reply on HN
Print view View original

11–20 of 74 posts

Re: OpenBSD: PF queues break the 4 Gbps barrier

#11

Post by ralferoo »

In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue?

I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just surprised that nobody has felt a pressing enough need to fix this earlier in the previous few years.

Re: OpenBSD: PF queues break the 4 Gbps barrier

#12

Post by arsome »

throw0101d wrote: post #5
bell-cot wrote: post #2

"Values up to 999G are supported, more than enough for interfaces today and the future." - Article "When we set the upper limit of PC-DOS at 640K, we thought nobody would ever need that much memory." - Bill Gates

> "Values up to 999G are supported, more than enough for interfaces today and the future." - Article Especially given that IEEE 802.3dj is working on 1.6T / 1600G, and is expected to publish the final spec in Summer/Autumn 2026: * https://en.wikipedia.org/wiki/Terabit_Ethernet Currently these interfaces are only on switches, but there are already NICs at 800G (P1800GO, Thor Ultra, ConnectX-8/9), so if you LACP/LAGG t…

If you're moving those kind of speeds you're probably not doing packet filtering in software.

Re: OpenBSD: PF queues break the 4 Gbps barrier

#13

Post by citrin_ru »

ralferoo wrote: post #11

In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…

AFAIK performance is not a priority for OpenBSD project - security is (and other related qualities like code which is easy to understand and maintain). FreeBSD (at least when I followed it several years ago) had better performance both for ipfw and its own PF fork (not fully compatible with OpenBSD one).

Re: OpenBSD: PF queues break the 4 Gbps barrier

#14

Post by bell-cot »

elevation wrote: post #6
WhyNotHugo wrote: post #3

Earlier quoted context omitted.

Honestly, I'm really curious about this number. 10bits is 1024, so why 999G specifically?

Looks like an arbitrary validation cap. By the time we're maxing out the 64-bit underlying representation we probably won't be using Ethernet any more.

https://en.wikipedia.org/wiki/Ethernet#History (& following sections)

Calling something "Ethernet" amounts to a promise that:

- From far enough up the OSI sandwich*, you can pretend that it's a magically-faster version of old-fashioned Ethernet

- It sticks to broadly accepted standards, so you won't get bitten by cutting-edge or proprietary surprises

*https://en.wikipedia.org/wiki/OSI_model

Re: OpenBSD: PF queues break the 4 Gbps barrier

#15

Post by bitfilped »

bell-cot wrote: post #2

"Values up to 999G are supported, more than enough for interfaces today and the future." - Article "When we set the upper limit of PC-DOS at 640K, we thought nobody would ever need that much memory." - Bill Gates

Yes, we're already running 800G networks, so this phrasing seems really silly to me.

Re: OpenBSD: PF queues break the 4 Gbps barrier

#16

Post by toast0 »

ralferoo wrote: post #11

In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…

> Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue?

This looks like it only affects bandwidth limiting. I suspect it's pretty niche to use OpenBSD as a traffic shaper at 10G+, and if you did, I'd imagine most of the queue limits would tend toward significantly less than 4G.

Re: OpenBSD: PF queues break the 4 Gbps barrier

#17

Post by IcePic »

ralferoo wrote: post #11

In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…

One thing could also be that by the time you have 10GE uplinks, shaping is not as important.

When we had 512kbit links, prioritizing VOIP would be a thing, and for asymmetric links like 128/512kbit it was prudent to prioritize small packets (ssh) and tcp ACKs on the outgoing link or the downloads would suffer, but when you have 5-10-25GE, not being able to stick an ACK packet in the queue is perhaps not the main issue.

Re: OpenBSD: PF queues break the 4 Gbps barrier

#18

Post by throw0101d »

arsome wrote: post #12
throw0101d wrote: post #5

Earlier quoted context omitted.

> "Values up to 999G are supported, more than enough for interfaces today and the future." - Article Especially given that IEEE 802.3dj is working on 1.6T / 1600G, and is expected to publish the final spec in Summer/Autumn 2026: * https://en.wikipedia.org/wiki/Terabit_Ethernet Currently these interfaces are only on switches, but there are already NICs at 800G (P1800GO, Thor Ultra, ConnectX-8/9), so if you LACP/LAGG t…

If you're moving those kind of speeds you're probably not doing packet filtering in software.

But you may be using Unix-y software to manage the interfaces and do offload programming:

* https://en.wikipedia.org/wiki/Vector_Packet_Processing

* https://www.youtube.com/watch?v=ptm9h-Lf0gg ("VPP: A 1Tbps+ router with a single IPv4 address")

* https://en.wikipedia.org/wiki/Cumulus_Networks

Re: OpenBSD: PF queues break the 4 Gbps barrier

#19

Post by traceroute66 »

ralferoo wrote: post #11

In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…

> Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere

Half the problem is lack of proper drivers. I love OpenBSD but all the fibre stuff is just a bit half-baked.

For a long time OpenBSD didn't even have DOM (light-level monitoring etc.) exposed in its 1g fibre drivers. Stuff like that automatically kills off OpenBSD as a choice for datacentres where DOM stats are a non-negotiable hard requirement as they are so critical to troubleshooting.

OpenBSD finally introduced DOM stats for SFP somewhere around 2020–2021, but it doesn't always work, it depends if you have the right magic combination of SFP and card manufacturer. Whilst on FreeBSD it Just Works (TM).

And then overall, for higher speed optics, FreeBSD simply remains lightyears ahead (forgive the pun !). For example, Decisio make nice little router boxes with 10g SFP+ on them, FreeBSD has the drivers out-of-the-box, OpenBSD doesn't. And that's only an SFP+ example, its basically rolling-tumbleweed in a desert territory if you start venturing up to QSFP etc. ...

Re: OpenBSD: PF queues break the 4 Gbps barrier

#20

Post by Someone »

ralferoo wrote: post #11

In the days when even cheap consumer hardware ships with 2.5G ports, this number seems weirdly low. Does this mean that basically nobody is currently using OpenBSD in the datacentre or anywhere that might be expecting to handle 10G or higher per port, or is it just filtering that's an issue? I'm not surprised that the issue exists as even 10 years ago these speeds were uncommon outside of the datacentre, I'm just sur…

The article is about allowing bandwidth restrictions in bytes/second that are larger than 2³²-1, not about how fast pf can filter packets.

I guess few people with faster ports felt the need to limit bandwidth for a service to something that’s that large.

FTA:

“OpenBSD's PF packet filter has long supported HFSC traffic shaping with the queue rules in pf.conf(5). However, an internal 32-bit limitation in the HFSC service curve structure (struct hfsc_sc) meant that bandwidth values were silently capped at approximately 4.29 Gbps, ” the maximum value of a u_int ".

With 10G, 25G, and 100G network interfaces now commonplace, OpenBSD devs making huge progress unlocking the kernel for SMP, and adding drivers for cards supporting some of these speeds, this limitation started to get in the way. Configuring bandwidth 10G on a queue would silently wrap around, producing incorrect and unpredictable scheduling behaviour.

A new patch widens the bandwidth fields in the kernel's HFSC scheduler from 32-bit to 64-bit integers, removing this bottleneck entirely.”

Post reply on HN