Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

F-Stack – A network development kit with high performance based on DPDK

f-stack.org

Post reply on HN
Print view View original

21–30 of 32 posts

Re: F-Stack – A network development kit with high performance based on DPDK

#21

Post by nsteel »

gonzopancho wrote: post #18
eqvinox wrote: post #9

Earlier quoted context omitted.

> In this space Kernel Bypass is still very much the norm if you want to get an efficient system. Unless you can get an ASIC to do it, then the ASIC is massively preferrable; just the power savings generally¹ end the discussion. (= remove most routers from the list; also some security appliances and load balancers.) ¹ exceptions confirm the rule, i.e. small/boutique setups

ASICs require years to develop and aren’t flexible once deployed

Even the ones supporting things like P4?

Re: F-Stack – A network development kit with high performance based on DPDK

#22

Post by lossolo »

ozgrakkurt wrote: post #16
touisteur wrote: post #11

Earlier quoted context omitted.

Recently did 400gb/s on a single core / 4x100gb nics (or just the one 400g nic, too) with dpdk. Mind you it's with jumbo frames and constant packet size for hundreds of mostly synchronized streams... You won't process each packet individually, mostly put them in queues for later batch-process by other cores. Amazing for data acquisition applications using UDP streams. I keep watching and trying io_uring and still can…

Sounds super cool but dpdk sounds like it won't be worth the difficulty from what I read so far. I also want to get into socket io using io_uring in zig. I'll try to apply everything I found in liburing wiki [0] and see how much I can get (max hardware I have is 10gbit/s). Seems like there is: - multi-shot requests - register_napi on uring instance - zero copy receive/send. (Probably won't be able to get into it) Did…

You don't even need io_uring for 10 gbit/s, epoll will do that easily, unless you have very niche workload.

Re: F-Stack – A network development kit with high performance based on DPDK

#23

Post by pclmulqdq »

rivetfasten wrote: post #10
js4ever wrote: post #4

Earlier quoted context omitted.

I don't think so, but my guess is raw performance rarely matters in the real world. I once explored this, hitting around 125K RPS per core on Node.js. Then I realized it was pointless, the moment you add any real work (database calls, file I/O, etc.), throughput drops below 10K RPS.

It's always a matter of chasing the bottleneck. It's fair to say that network isn't the bottleneck for most applications. Heuristically, if you're willing to take on the performance impacts of a GC'd language you're probably already not the target audience. Zero copy is the important part for applications that need to saturate the NIC. For example Netflix integrated encryption into the FreeBSD kernel so they could us…

TCP is generally zero-copy now. Zero-copy with io_uring is also possible.

AF_XDP is also another way to do high-performance networking in the kernel, and it's not bad.

DPDK still has a ~30% advantage over an optimized kernel-space application with a huge maintenance burden. A lot of people reach for it, though, without optimizing kernel interfaces first.

Re: F-Stack – A network development kit with high performance based on DPDK

#24

Post by pclmulqdq »

klaussilveira wrote: post #7
nurumaik wrote: post #3

Earlier quoted context omitted.

Anything more innovative happened since (honestly curious)?

https://asynciobench.github.io/

Sorry, but when one thing in your benchmark has 50x the performance of the baseline, you probably have a bad baseline. If you used AI to write these, it probably did not use io_uring or aio correctly, or you have some sort of system misconfiguration. You may have also failed to bypass the filesystem with those methods, which would explain a lot of the discrepancy.

Re: F-Stack – A network development kit with high performance based on DPDK

#25

Post by touisteur »

ozgrakkurt wrote: post #16
touisteur wrote: post #11

Earlier quoted context omitted.

Recently did 400gb/s on a single core / 4x100gb nics (or just the one 400g nic, too) with dpdk. Mind you it's with jumbo frames and constant packet size for hundreds of mostly synchronized streams... You won't process each packet individually, mostly put them in queues for later batch-process by other cores. Amazing for data acquisition applications using UDP streams. I keep watching and trying io_uring and still can…

Sounds super cool but dpdk sounds like it won't be worth the difficulty from what I read so far. I also want to get into socket io using io_uring in zig. I'll try to apply everything I found in liburing wiki [0] and see how much I can get (max hardware I have is 10gbit/s). Seems like there is: - multi-shot requests - register_napi on uring instance - zero copy receive/send. (Probably won't be able to get into it) Did…

I ... kind of agree with the difficulty. I don't get it - DPDK is at its core really not a complex API ! Allocate a pool of buffers, and in an infinite loop, ask your NIC to fill these buffers. There. After that, yes you have to decap every packet (ethernet then IP - don't forget reassembly - then whatever you have over - UDP is absolutely no effort, TCP... not so). It's wholly manageable to anyone knowing a bit of light C++ (more C-like) and lower layers (and can parse the sometimes very dry and cryptic doc, for all the utility fonctions. Interaction with the actual consumer of the data can be done with DPDK-provided primitives or simple shared memory... it's really not hard for a mid-level systems programmer. But I still find myself unable to hire people who can work at that level of the stack, a bit baffling. I can't see how they'd be better with io_uring or AF_XDP and all their inherent complexity. Anything harder than a socket and epoll and you're a wizard now...

One other big plus of DPDK for me is the low-level access to hardware offload. GPUDirect (when you can get it to work), StorageDirect or most of the available DMA engines in some (not so) high-end hardware. The flow API on mellanox hardware is the basis of many of my multi-accelerator applications (I wish they supported P4 for packet format instead, or just open-source whatever low-level ISA the controller is running, but I don't buy enough gear to have a voice). Perusing the DPDK documentation can give ideas.

So, yes, very low-level with some batteries included. Good and stable for niche uses. But far smaller hiring pool (is the io_uring-100Gb pool bigger ? I don't know).

Re: F-Stack – A network development kit with high performance based on DPDK

#26

Post by eqvinox »

gonzopancho wrote: post #18
eqvinox wrote: post #9

Earlier quoted context omitted.

> In this space Kernel Bypass is still very much the norm if you want to get an efficient system. Unless you can get an ASIC to do it, then the ASIC is massively preferrable; just the power savings generally¹ end the discussion. (= remove most routers from the list; also some security appliances and load balancers.) ¹ exceptions confirm the rule, i.e. small/boutique setups

ASICs require years to develop and aren’t flexible once deployed

You don't develop an ASIC to run a router with, you buy one off the shelf. And the function of a router doesn't exactly change day by day (or even year by year).

Re: F-Stack – A network development kit with high performance based on DPDK

#27

Post by eqvinox »

Eduard wrote: post #20

Hacker News is the kind of place where you can have _this_ submission (PRC-sponsored Tencent-owned network devkit) on the front page next to a submission about how PRC-sponsored cybercrime group Salt Typhoon pwned 'nearly every American': https://news.ycombinator.com/item?id=45074157

People in the PRC develop interesting things. People in the PRC hack their way around the planet.

People in the USA develop interesting things. People in the USA hack their way around the planet.

The Russians seem to be doing mostly the hacking part.

The Europeans run around like headless chickens.

You can probably guess I'm European.

Re: F-Stack – A network development kit with high performance based on DPDK

#28

Post by immibis »

lossolo wrote: post #22
ozgrakkurt wrote: post #16

Earlier quoted context omitted.

Sounds super cool but dpdk sounds like it won't be worth the difficulty from what I read so far. I also want to get into socket io using io_uring in zig. I'll try to apply everything I found in liburing wiki [0] and see how much I can get (max hardware I have is 10gbit/s). Seems like there is: - multi-shot requests - register_napi on uring instance - zero copy receive/send. (Probably won't be able to get into it) Did…

You don't even need io_uring for 10 gbit/s, epoll will do that easily, unless you have very niche workload.

For UDP Pixelflut, I was able to send 8Gbps on a 10Gbps link with a single thread running a tight loop doing byte shuffling and then sendmmsg. I didn't bother to multithread it because that's a convenient amount of headroom left over for actual communications.

Re: F-Stack – A network development kit with high performance based on DPDK

#29

Post by ZephyrP »

eqvinox wrote: post #26
gonzopancho wrote: post #18

Earlier quoted context omitted.

ASICs require years to develop and aren’t flexible once deployed

You don't develop an ASIC to run a router with, you buy one off the shelf. And the function of a router doesn't exactly change day by day (or even year by year).

Change keeps coming, even when the wire format of a protocol has ossified. I've spent years in security and router performance at Cisco, wrote a respectable fraction of the flagship's L3 and L2-L3 (tun) firewall. I merged a patch on this tried-and-true firewall just this year; it's now deployed.

As vendors are eager to remind us, custom silicon to accelerate everything between L1 to L7 exists. That said, it is still the case in 2025 that the "fast path" data-plane will end up passing either nothing or everything in a flow to the "slow path" control-plane, where the most significant silicon is less 'ASIC' and more 'aarch64'.

This is all to say that the GP's comments are broadly correct.

Re: F-Stack – A network development kit with high performance based on DPDK

#30

Post by js4ever »

jandrewrogers wrote: post #6
js4ever wrote: post #4

Earlier quoted context omitted.

I don't think so, but my guess is raw performance rarely matters in the real world. I once explored this, hitting around 125K RPS per core on Node.js. Then I realized it was pointless, the moment you add any real work (database calls, file I/O, etc.), throughput drops below 10K RPS.

Storage and database doesn’t have to be that slow, that’s just architecture. I have database servers doing 10M RPS each, which absolutely will stress the network. We just do the networking bits a bit differently now. DPDK was a product of its time.

What DB engine is it? What hardware?
Post reply on HN