Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Epoll vs. io_uring in Linux

sibexi.co

Post reply on HN
Print view View original

61–70 of 79 posts

Re: Epoll vs. io_uring in Linux

#61

Post by toast0 »

ahepp wrote: post #36
toast0 wrote: post #11

> But my students weren’t as happy as I was - they wanted to build something genuinely useful, and they were really disappointed that our “product” had strong architectural limits and couldn’t outperform titans like nginx and haproxy. I took a (very brief) look at the github repo [1], it doesn't look like you're doing anything with cpu pinning. You can probably eke (thanks) out a bit more performance if you cpu pin y…

I would be interested to see benchmarks for that patch

I don't have the right setup to make good benchmarks for this right now, but when I had the chance to put it into practice, the improvement between no cpu alignment and full alignment was quite large. That was on a 28 core machines (with 16 nic queues); many years ago, but IIRC, I got at least 10x the connections/sec out of the boxes after tuning and after tuning 12 cores were idle ... the machines were repurposed, if they were ordered for this, they should have had one core per nic queue in a single socket. The difference is likely smaller on a 4 core machine as described in the article.

The hardest part is going to be generating enough load. I had production load, which has the benefit that you don't need to generate it. Otoh, it was a transitional need, and I couldn't reasonably test above 50% of peak traffic on a single machine ... I hit that mark around the time traffic started dropping, and then it wasn't fun anymore.

Re: Epoll vs. io_uring in Linux

#62

Post by topspin »

MathMonkeyMan wrote: post #4
mrlonglong wrote: post #2

Boost asio if you love C++ and asynchronous networking.

I switched out asio's epoll backend for its io_uring in a database server and CPU utilization shot up. Probably depends on usage and the specifics of how it's integrated into the event code.

Classic.

Know that the increase in CPU utilization may mean you've improved the performance of your "database server," because now your CPU cores are waiting less on IO. It also may not mean this, but just looking at htop won't tell you either way.

Re: Epoll vs. io_uring in Linux

#63

Post by nottorp »

lukeh wrote: post #26
Cloudef wrote: post #23

Earlier quoted context omitted.

Quite depends, I had times when my posix emulation of io_uring (with poll, not epoll) was faster than io_uring. For large zero-copy buffers, io_uring is king however. Also io_uring is useful even for non asynchronous IO as it can implement chain of operations as single atomic operation (mkdir + open it for example). For something like networking, if you are maximizing packets per second, you'll hit kernel limits[1] v…

Also it’s nice for things like SPI which have no user space non-blocking API.

SPI the bus?

Re: Epoll vs. io_uring in Linux

#65

Post by Asmod4n »

mort96 wrote: post #44
Asmod4n wrote: post #31

Earlier quoted context omitted.

The main reason why it gets disabled is fixed now, the latest RC got cBPF support and as such you can restrict what OPs can be run now instead of just fully disabling it.

Well the reason it's disabled now is the recent history of pretty bad vulnerabilities. It probably needs to go a while without new vulnerabilities before it makes sense to enable by default. It's pretty complex completely unsafe C code, after all.

Then just disable that singular function which causes the issue instead of a whole kernel subsystem. The current release can do that.

Re: Epoll vs. io_uring in Linux

#66

Post by mort96 »

Asmod4n wrote: post #65
mort96 wrote: post #44

Earlier quoted context omitted.

Well the reason it's disabled now is the recent history of pretty bad vulnerabilities. It probably needs to go a while without new vulnerabilities before it makes sense to enable by default. It's pretty complex completely unsafe C code, after all.

Then just disable that singular function which causes the issue instead of a whole kernel subsystem. The current release can do that.

What's the singular function that has a chance of turning out to be exploitable in the future?

Re: Epoll vs. io_uring in Linux

#67

Post by cyphar »

Asmod4n wrote: post #54
insanitybit wrote: post #46

Earlier quoted context omitted.

How solved? AFAIK it's not meaningfully shipped but happy to hear otherwise.

you can now disable opertions with cBPF, like you would be able to with seccomp for normal syscalls.

Their point is that that functionality is not available on older kernels (such as those in RHEL 9 and 10) and so most sandboxes will continue to block it outright for a while, though eventually one would expect Red Hat to backport it.

(We haven't even added support for the new cBPF io_uring stuff to low-level container runtimes like runc yet, though I did review the patchset on LKML earlier this year and planned to get working on it when I have time. But as it requires spec changes, expect it to take 6-12 months at best...)

Re: Epoll vs. io_uring in Linux

#68

Post by [deleted] »

toast0 wrote: post #61
ahepp wrote: post #36

Earlier quoted context omitted.

I would be interested to see benchmarks for that patch

I don't have the right setup to make good benchmarks for this right now, but when I had the chance to put it into practice, the improvement between no cpu alignment and full alignment was quite large. That was on a 28 core machines (with 16 nic queues); many years ago, but IIRC, I got at least 10x the connections/sec out of the boxes after tuning and after tuning 12 cores were idle ... the machines were repurposed, i…

[deleted]

Re: Epoll vs. io_uring in Linux

#69

Post by Asmod4n »

cyphar wrote: post #67
Asmod4n wrote: post #54

Earlier quoted context omitted.

you can now disable opertions with cBPF, like you would be able to with seccomp for normal syscalls.

Their point is that that functionality is not available on older kernels (such as those in RHEL 9 and 10) and so most sandboxes will continue to block it outright for a while, though eventually one would expect Red Hat to backport it. (We haven't even added support for the new cBPF io_uring stuff to low-level container runtimes like runc yet, though I did review the patchset on LKML earlier this year and planned to g…

Thanks for that! Hope 2027 will then be the year we get the speed back all those spectre etc mitgitations ate for those syscalls io_uring can replace.

Re: Epoll vs. io_uring in Linux

#70

Post by Asmod4n »

mort96 wrote: post #66
Asmod4n wrote: post #65

Earlier quoted context omitted.

Then just disable that singular function which causes the issue instead of a whole kernel subsystem. The current release can do that.

What's the singular function that has a chance of turning out to be exploitable in the future?

With seccomp you can disable a single syscall which might be exploitable since you didn’t get around to patch your kernel, with cBPF you can now do the same for io_uring
Post reply on HN