Live data from Hacker News

Linux Kernel vs. DPDK: HTTP Performance Showdown

talawah.io

71–72 of 72 posts

Re: Linux Kernel vs. DPDK: HTTP Performance Showdown

#71

Earlier quoted context omitted.

Yea, it is definitely a fake HTTP server which I acknowledge in the article [1]. However based on the size of the requests, and my observation of the number of packets per second in/out being symmetrical at the network interface level, I didn't have a concern about doubled responses. Skipping the parsing of the HTTP requests definitely gives a performance boost, but for this comparison both sides got the same boost,…

> Yea, it is definitely a fake HTTP server which I acknowledge in the article It's not actually an HTTP server though... For these purposes, it's essentially no more useful than netcat dumping out a preconfigured text file. Titling it "HTTP Performance showdown" is doubly bad here since there's no real-world (or even moderately synthetic) HTTP requests happening; you just always get the same static set of data for ev…

> It's not actually an HTTP server though...

Correct, it is a fake HTTP server, serving a real HTTP workload. This post is about comparing two different networking stacks (kernel vs DPDK) to see how they handle a specific (and extreme) HTTP workload. From the perspective of the networking stack, the networking hardware, and the AWS networking fabric between instances, these are real HTTP requests and responses.

> I'd really love to see this adapted to do actual webserver work and see what the difference is.

Take a look at my previous article[1]. It is still an extreme/synthetic benchmark, but libreactor was able to hit 1.2M req/s while fully parsing the HTTP requests using picohttpparser[3].

From what I recall, when I played with disabling HTTP parsing in libreactor, the performance improvement was only about 5%.

1. https://talawah.io/blog/extreme-http-performance-tuning-one-...

Re: Linux Kernel vs. DPDK: HTTP Performance Showdown

#72

Earlier quoted context omitted.

Few questions if you will, it's an interesting work and I figure you're on ScyllaDB team? 1. Is 5s experiment with 1s warmup really a representative workload? How about running for several minutes or tens of minutes? Do you observe the same result? 2. How about 256 connections on 16 vCPUs creating contention against each other and therefore skewing the experiment results? Aren't they competing for the same resources…

Am ScyllaDB but Marc did completely independent work. The client vcpus don't matter that much, the experiment compares the server side, the client shouldn't suck. When we test ScyllaDB or other DBs, we run benchmarks for hours and days. This is just a stateless, static http daemon, so short timing is reasonable. The whole intent is to make it a learning experience, if you wish to reproduce, try it yourself. It's alig…

I'm myself doing a lot of algorithmic design but I also enjoy designing e2e performance testing frameworks in order to confirm theories I or others had on a paper. The thing is that I fell too many times into a trap without realizing that the results I was observing weren't what I thought I was measuring. So what I was hoping for is to spark a discussion around the thoughts and methodologies other people from the field use and hopefully learn something new.
Post reply on HN