Linux Kernel vs. DPDK: HTTP Performance Showdown
1–10 of 72 posts
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#2From: https://talawah.io/blog/extreme-http-performance-tuning-one-...
> I am genuinely interested in hearing the opinions of more security experts on this (turning off speculative execution mitigatins). If this is your area of expertise, feel free to leave a comment
Are these generally safe if you have a machine that does not have multi-user access and is in a security boundary?
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#3This was a fascinating read and the kernel does quite nicely in comparison - 66% of DPDK performance is amazing. That said, the article completely nails the performance advantage: DPDK doesn't do a lot of stuff that the kernel does. That stuff takes time. If I recall correctly, DPDK abstractions themselves cost a bit of NIC performance, so it might be interesting to see a comparison including a raw NIC-specific kernel bypass framework (like the SolarFlare one).
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#4From: https://talawah.io/blog/extreme-http-performance-tuning-one-... > I am genuinely interested in hearing the opinions of more security experts on this (turning off speculative execution mitigatins). If this is your area of expertise, feel free to leave a comment Are these generally safe if you have a machine that does not have multi-user access and is in a security boundary?
For regulatory compliance, that’s still not acceptable because it opens the door to cross user data access or privilege escalation.
If someone exploited a process with a dedicated unprivileged user, had legit limited access, or got in a container on a physical, they might be able to leverage it for the forces of evil.
There’s really no such practical thing as single user Linux. If you’re running a network exposed app without dropping privileges, that’s a much bigger security risk than speculative execution.
Now, if you were skipping an OS and going full bare metal, then that could be different. But an audit for that would be a nightmare :).
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#5At the point you've gotten syscall overhead is definitely going to be a big thing (even without spectre mitigations enabled) -- I'd be very curious to see how far a similar io_uring benchmark would get you.
It supports IOPOLL (polling of the socket) and SQPOLL (kernel side polling of the request queue) so hopefully the fact that application driving it is in another thread wouldn't slow it too much... With multi-shot accept/recv you'd only need to tell it to keep accepting connections on the listener fd, but I'm not sure if you can chain recvs to the child fd automatically from kernel or not yet... We live in interesting times!
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#6From: https://talawah.io/blog/extreme-http-performance-tuning-one-... > I am genuinely interested in hearing the opinions of more security experts on this (turning off speculative execution mitigatins). If this is your area of expertise, feel free to leave a comment Are these generally safe if you have a machine that does not have multi-user access and is in a security boundary?
For regulatory compliance, that’s still not acceptable because it opens the door to cross user data access or privilege escalation. If someone exploited a process with a dedicated unprivileged user, had legit limited access, or got in a container on a physical, they might be able to leverage it for the forces of evil. There’s really no such practical thing as single user Linux. If you’re running a network exposed app…
There’s definitely sneakernet Linux boxes. I’ve worked at a bunch of places with random Linux boxes running weird crap totally off the actual network because nobody was particularly sure how to get those programs updated. Technical debt is a pita!
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#7Hi Marc (talawahtech)! Thanks for the exhaustive article.
I took a short look at the benchmark setup (https://github.com/talawahtech/seastar/blob/http-performance...), and wonder if some simplifications there lead to overinflated performance numbers. The server here executes a single read() on the connection - and as soon as it receives any data it sends back headers. A real world HTTP server needs to read data until all header and body data is consumed before responding.
Now given the benchmark probably sends tiny requests, the server might get everything in a single buffer. However every time it does not, the server will send back two responses to the server - and at that time the client will already have a response for the follow-up request before actually sending it - which overinflates numbers. Might be interesting to re-test with a proper HTTP implementation (at least read until the last 4 bytes received are \r\n\r\n, and assume the benchmark client will never send a body).
Such a bug might also lead to a lot more write() calls than what would be actually necessary to serve the workload, or to stalling due to full send or receive buffers - all of those might also have an impact on performance.
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#8At the point you've gotten syscall overhead is definitely going to be a big thing (even without spectre mitigations enabled) -- I'd be very curious to see how far a similar io_uring benchmark would get you. It supports IOPOLL (polling of the socket) and SQPOLL (kernel side polling of the request queue) so hopefully the fact that application driving it is in another thread wouldn't slow it too much... With multi-shot…
I would love to see an io_uring comparison as well; while it's a substantial amount of work to port an existing framework to io_uring, at the point where you're considering DPDK, io_uring seems relatively small by comparison.
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#9For those like me going "......what is dpdk"
The Data Plane Development Kit (DPDK) is an open source software project managed
by the Linux Foundation. It provides a set of data plane libraries and network
interface controller polling-mode drivers for offloading TCP packet processing
from the operating system kernel to processes running in user space. This offloading
achieves higher computing efficiency and higher packet throughput than is possible
using the interrupt-driven processing provided in the kernel.
Re: Linux Kernel vs. DPDK: HTTP Performance Showdown
#10From: https://talawah.io/blog/extreme-http-performance-tuning-one-... > I am genuinely interested in hearing the opinions of more security experts on this (turning off speculative execution mitigatins). If this is your area of expertise, feel free to leave a comment Are these generally safe if you have a machine that does not have multi-user access and is in a security boundary?
For regulatory compliance, that’s still not acceptable because it opens the door to cross user data access or privilege escalation. If someone exploited a process with a dedicated unprivileged user, had legit limited access, or got in a container on a physical, they might be able to leverage it for the forces of evil. There’s really no such practical thing as single user Linux. If you’re running a network exposed app…
Which regulations?