From my benchmark, i will keep using oha (https://github.com/hatoo/oha). Oha is more complete than gcannon and have similar req/s rate while handling ipv6, https, etc...
Linux extreme performance H1 load generator
11–18 of 18 posts
Re: Linux extreme performance H1 load generator
#12From my benchmark, i will keep using oha ( https://github.com/hatoo/oha ). Oha is more complete than gcannon and have similar req/s rate while handling ipv6, https, etc...
oha is one of the slowest load gen, you should look into h2load if you need h2/h3 support. I just tried oha and it pulls more CPU than the server I am testing, not to mention h2 and h3 results are just nonsense
Re: Linux extreme performance H1 load generator
#13Interesting, I made something similar years ago when io_uring wasn't around yet and it is just a couple threads blocking on sendfile: https://github.com/evelance/sockbiter Of course it needs to pre-generate the file and you need enough RAM for both the server running and caching the file but it needs almost zero CPU during the test run and can probably produce even more load than this io_uring tool.
Very cool! So I just tried your tool and it just hangs, I see you're sending close requests, is this configurable to keep-alive, or even better, nothing? In Http/1.1 keep-alive/close is better not used at all, never try to enforce this as it is not mandatory. A lot of servers just ignore the close and don't close the connection (like the one I am using) so this can be the issue I am having.
Cool, thanks for trying it.
Try the -shutwr option if the server doesn't close the connection itself. I used it to test lots of exotic implementations and there are weird things going on in overload situations and around connection management. NodeJS for example started dropping connections on localhost(!!) on high load.
The tool was built for high values of keepalive requests, if the server is too fast just use more requests, e.g. -n 1000000 or something similar. Unfortunately some servers close keepalive connections after quite few requests, nginx has a default of 1000 for example.
This is just a simple tool I hacked together as a student to collect some data, didn't spend any time making it more accessible/user friendly, sorry.
Re: Linux extreme performance H1 load generator
#14Earlier quoted context omitted.
Very cool! So I just tried your tool and it just hangs, I see you're sending close requests, is this configurable to keep-alive, or even better, nothing? In Http/1.1 keep-alive/close is better not used at all, never try to enforce this as it is not mandatory. A lot of servers just ignore the close and don't close the connection (like the one I am using) so this can be the issue I am having.
Cool, thanks for trying it. Try the -shutwr option if the server doesn't close the connection itself. I used it to test lots of exotic implementations and there are weird things going on in overload situations and around connection management. NodeJS for example started dropping connections on localhost(!!) on high load. The tool was built for high values of keepalive requests, if the server is too fast just use more…
I ran into some lua erros and fixed them, eventually I got it running with -shutwr but the results are basically impossible
----------- Summary ---------- Successful connections: 8 out of 8 (0 failed). Total bytes sent . . . . . 2599999960.00 B Total bytes received . . . 82520.00 B Benchmark duration . . . . 85.94 ms Send throughput . . . . . 30252779546.89 B/sec Receive throughput . . . . 960176.69 B/sec Aggregate req/second . . . 93085476.96
The received data is too low. Also 93 million requests per second, the only way this is possible is due the fact that the load generator is not waiting for the server response and processing it. But I guess this is expectable since there might be some issues as I am using a much more recent kernel than you did when building this
I used -n 10000000 (10M)
Re: Linux extreme performance H1 load generator
#15Really stupid question from someone who doesnt know much about io_uring. Wouldn't doing all this i/o async make the latency measurements less accurate? How do you know when the i/o starts if you are submitting it async in batches of 2048?
[deleted]
Re: Linux extreme performance H1 load generator
#16Earlier quoted context omitted.
Cool, thanks for trying it. Try the -shutwr option if the server doesn't close the connection itself. I used it to test lots of exotic implementations and there are weird things going on in overload situations and around connection management. NodeJS for example started dropping connections on localhost(!!) on high load. The tool was built for high values of keepalive requests, if the server is too fast just use more…
I ran into some lua erros and fixed them, eventually I got it running with -shutwr but the results are basically impossible ----------- Summary ---------- Successful connections: 8 out of 8 (0 failed). Total bytes sent . . . . . 2599999960.00 B Total bytes received . . . 82520.00 B Benchmark duration . . . . 85.94 ms Send throughput . . . . . 30252779546.89 B/sec Receive throughput . . . . 960176.69 B/sec Aggregate r…
As per the README responses are not checked by the tool.
If you only received ~80KB for 10M requests the server probably terminated the connection early before processing all requests (like nginx does after 1k requests on one TCP keepalive socket if you use the default configuration). Check the responses-XXX.txt files to see what happened. You then need to either adjust the server configuration or use multiple sockets with the max keepalive requests the server can handle.
If you run this tool on the same machine as the server process, the requests file is likely held in the file system cache (RAM and shared by all threads) and every recv() call by the server under test is essentially a memory copy at the speed of the machine's memory bandwidth, which can easily be >>10GB/s or millions of requests per second per connection. This is also way faster than typical servers can even parse HTTP/1.
But highly optimized servers running straight HTTP/1 without TLS or backend logic on multiple threads should absolutely hit multiple millions of requests per second with this tool. Researching how fast an HTTP/1 server can get was the reason I made this in the first place.
Re: Linux extreme performance H1 load generator
#17Earlier quoted context omitted.
I ran into some lua erros and fixed them, eventually I got it running with -shutwr but the results are basically impossible ----------- Summary ---------- Successful connections: 8 out of 8 (0 failed). Total bytes sent . . . . . 2599999960.00 B Total bytes received . . . 82520.00 B Benchmark duration . . . . 85.94 ms Send throughput . . . . . 30252779546.89 B/sec Receive throughput . . . . 960176.69 B/sec Aggregate r…
As per the README responses are not checked by the tool. If you only received ~80KB for 10M requests the server probably terminated the connection early before processing all requests (like nginx does after 1k requests on one TCP keepalive socket if you use the default configuration). Check the responses-XXX.txt files to see what happened. You then need to either adjust the server configuration or use multiple socket…
Ah, I think I understand now, we are bombarding the server in a H/1.1 pipelined approach so basically not waiting for server response to send the next request and theoretically using infinite pipelined depth as we never really check any response and simply jam the server with as many requests as possible. That would explain the results - the issue with that is that we don't really check if the server is able to process the extremely high number of requests and most of them are likely just lost and never processed by the server - so we are basically measuring the tool output capability not the server performance.
I can see in the results .txt that only a small portion of the sent requests actually result in a response, also not every server supports H/1.1 pipeline so they will flush once per request (typical workload), servers that support pipeline will have way higher throughput
Re: Linux extreme performance H1 load generator
#18Earlier quoted context omitted.
As per the README responses are not checked by the tool. If you only received ~80KB for 10M requests the server probably terminated the connection early before processing all requests (like nginx does after 1k requests on one TCP keepalive socket if you use the default configuration). Check the responses-XXX.txt files to see what happened. You then need to either adjust the server configuration or use multiple socket…
Ah, I think I understand now, we are bombarding the server in a H/1.1 pipelined approach so basically not waiting for server response to send the next request and theoretically using infinite pipelined depth as we never really check any response and simply jam the server with as many requests as possible. That would explain the results - the issue with that is that we don't really check if the server is able to proce…
Exactly. For GET reqeuests HTTP/1 conformant servers must support pipelining or close the connection.
So this is the best way to generate extreme load and stress-test the internal architecture of an HTTP/1 server. But yeah the sendfile approach only works for this kind of testing and not in the generic case.