Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Do Some Science: Zed's Test Stack for poll vs. epoll

sheddingbikes.com

Post reply on HN
Print view View original

31–40 of 45 posts

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#31

Post by zedshaw »

jacquesm wrote: post #16
zedshaw wrote: post #11

Earlier quoted context omitted.

If the test harness is wrong than every other test that everyone doing epoll comparisons with poll has done is wrong. This is the one they've all been using. So, either you write your own test to evaluate the same thing, without confounding it with other external factors (like this test does), or you use this test and prove me wrong by finding different data. However, I'm glad you finally went and got some metrics to…

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

You know jack squat about statistics. That's basically how I summarize your statement. You don't have a mean, median, mode, or standard deviation in your "10% 40%". You don't understand confounding and want to test the performance of poll vs. epoll on file descriptors by testing an entire server. You think science is to use available data, when it's nothing of the sort, and in fact the way you remove confounding is to remove data.

Basically, you're a FUD slinger who's got a beef and doesn't know enough to hold his own in a real discussion about measurement and analysis.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#32

Post by zedshaw »

gthank wrote: post #27
jacquesm wrote: post #16

Earlier quoted context omitted.

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

Having followed most of the comments on both these threads, I don't think it's fair to characterize Zed's position as saying "epoll is always bad". I think a better characterization would be: "epoll is not automagically better under every imaginable scenario". You may argue that nobody is making any such claim, but I can tell you that the sections of the blogosphere I skim via HN, reddit, etc. do imply it. Maybe the…

Thank you. I very much appreciate your comments.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#33

Post by zedshaw »

isaacforce wrote: post #24
jacquesm wrote: post #16

Earlier quoted context omitted.

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

>I'm challenging you to provide the data that apparently is still missing that proves that this was not 'a localhost test'. I think what he was saying in relation to it being not 'a localhost test' is that it isn't a test that touches the network stack, even to the loopback interface -- you seem to be equating localhost with anything that happens on a single local machine regardless of whether it has anything to do w…

Yes, nobody seems to get confounding at all. I'm testing poll vs. epoll on selecting active file descriptors to compare their performance. I'm not testing anything else, not claiming anything else. To then test that with a full on server that has an Amazon EC2 cluster blasting requests with HTTP parsing and serving files would completely confuse the analysis.

You don't measure a specific thing by inventing some full on "real world" evaluation. You test that specific thing. Why is it this simple concept seems to baffle so many coders?

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#34

Post by gthank »

zedshaw wrote: post #32
gthank wrote: post #27

Earlier quoted context omitted.

Having followed most of the comments on both these threads, I don't think it's fair to characterize Zed's position as saying "epoll is always bad". I think a better characterization would be: "epoll is not automagically better under every imaginable scenario". You may argue that nobody is making any such claim, but I can tell you that the sections of the blogosphere I skim via HN, reddit, etc. do imply it. Maybe the…

Thank you. I very much appreciate your comments.

I appreciate the work you're doing. I'm not sure I've worked on a system that would really benefit from Mongrel2, but I think it's a pretty interesting concept, and there's always a chance I work on something more interesting in the future. Even if I don't, I almost always learn something or gain a different perspective from reading your stuff, and that's reason enough to be thankful.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#35

Post by rbanffy »

zedshaw wrote: post #30
rbanffy wrote: post #23

Earlier quoted context omitted.

> there's potential gains to be had by using both Only if your ATR crosses the .6 boundary frequently. If it consistently remains on either side, the performance would be worse at the cost of added complexity. I agree with your assumption we need more instrumentation. I strongly suspect my ATRs are consistently closer to zero, but I never looked under the hood to check. Also, the ATR should vary according to what you…

Exactly, nobody knows because they've been operating under the assumption that epoll is always faster so they never looked. Now I've got some evicence that epoll isn't faster, and in fact it's a wash at 40% "utilization", which is sort of ridiculous. I'd want people to go and test and see what they get, or at least know the implications so they pick wisely. Ultimately though, I'd rather have the Linux kernel just fix…

> Ultimately though, I'd rather have the Linux kernel just fix epoll so it's always faster than poll.

Unless they break poll, that's very unlikely. The implementation of poll is much simpler and should be faster per fd. What could happen is that epoll gets some improvement that moves up the .6 boundary but I seriously doubt epoll could ever be faster than poll with 100% active fds - and I never encountered that kind of usage.

As with any optimization, we should be concerned with end result: how much slower does using epoll really makes the system. I am on the skeptical side here.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#36

Post by jfager »

zedshaw wrote: post #29
jfager wrote: post #25

Earlier quoted context omitted.

What I've been saying all along is something very simple. If the ATR > 0.6, poll wins. If ATR You left out the part where you accused people who advocate using epoll of being bullshitting touts who perpetuate fallacies: "why the touted benefits of epoll and most of the information out there is mostly bullshit because of some falacies people seem to have about epoll (mostly perpetuated by epoll's proponents)." Which y…

The touted benefits of epoll are mostly bullshit, since it was advocated as being O(1) and frequently mentioned as being O(1) and always faster than poll by others. In fact, right after mentioning that I thought epoll wasn't faster the first thing people said was it was O(1). Every person I talked to said it. I have evidence that contradicts both of the main assertions of epoll, so until someone comes up with counter…

I googled 'epoll "O(1)"'. You're the first hit (congratulations), followed by some random person on stackoverflow, the documentation for a CPAN module, the repetition of that documentation on github and in various package repos, and some random bloggers and mailing list responders who are either just referring to the work that you have to do to find an active fd when the wait call returns, or are confused.

The man page says nothing about O(1):

http://www.kernel.org/doc/man-pages/online/pages/man4/epoll....

The original epoll writeup says nothing about O(1), and shows graphs explicitly for dead connection scenarios (iow, it never claims better performance per active fd):

http://www.xmailserver.org/linux-patches/nio-improve.html

The Dan Kegel writeup that's pretty much universally cited on this issue says nothing about O(1):

http://www.kegel.com/c10k.html#nb.epoll

The link you threw up yesterday that "flat out gets it wrong" says nothing about O(1) and simply shows epoll doing exactly what it was designed to do (again, it never claims better performance per active fd), including graphs that actually do show poll outperforming epoll with small numbers of dead connections:

http://lse.sourceforge.net/epoll/index.html

Can you show any authoritative sources who claim epoll is O(1)? If not, then yes, you built a fluffy little strawman and took him down hard.

I have no problem with your work on superpoll. I'm skeptical it will help much, guessing there will be situations it will hurt, and overall not convinced it's worth the effort to find out, but it's your time. Have fun with it, and I'm eager to see the results.

I just don't get why you feel the need to imply that everyone who ever used epoll without finding the ATR performance cut-off is a blind moron snowed into doing so by some shadowy malevolent epoll cabal spreading misinformation about how it works.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#37

Post by haberman »

tzs wrote: post #15

The network interface in general on most operating systems seems kind of absurd from a server point of view. You've got data coming in from N clients. The data from all those clients gets physically multiplexed into a single data stream that comes in over the wire. Then in the kernel you've got something that demultiplexes it back into N separate streams, which it accumulates in N different buffers. The server softwa…

Probably the biggest argument against doing what you describe is that the status quo gives you control over scheduling which fds to service (of the ones that are active).

If you processed incoming network data in strict FIFO order, a single host on your LAN could totally DoS your system by flanking you with incoming data. That host would never get TCP flow-controlled because if you're processing anything, you're processing his obnoxiously voluminous input.

If he was just on FD among many, you would service him once in a while, but then you would service other clients. While you're servicing other clients, he gets TCP flow-controlled, so he doesn't singlehandedly demand all your attention.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#38

Post by haberman »

jacquesm wrote: post #17
tzs wrote: post #15

The network interface in general on most operating systems seems kind of absurd from a server point of view. You've got data coming in from N clients. The data from all those clients gets physically multiplexed into a single data stream that comes in over the wire. Then in the kernel you've got something that demultiplexes it back into N separate streams, which it accumulates in N different buffers. The server softwa…

Technically nothing stops you from opening the raw device and to start reading packets. Sooner or later you'll need to figure out when your packets can be forwarded to the next layer for processing so work can be done for a particular connection. And that's what the kernel did for you all along. The abstraction of file descriptors for clients is a useful one, it allows you to decide what to do. The decision here is n…

> But I think you'd soon find that you are simply re-implementing all that code, and that instead of a clean (poll or epoll, either is fairly clean) interface you'd be bogged down in maintaining your own tcp stack.

You're not understanding OP's proposal. He's not talking about implementing TCP in user-space. He said the kernel would still be handling TCP flow control, and would only be sending data destined for a specific port to this fd. So TCP is still in the kernel.

He's just proposing that the payloads for all connections come in on a single fd instead of one fd per connection. But each read from the fd tells you what remote host it came from.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#39

Post by scott_s »

zedshaw wrote: post #31
jacquesm wrote: post #16

Earlier quoted context omitted.

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

You know jack squat about statistics. That's basically how I summarize your statement. You don't have a mean, median, mode, or standard deviation in your "10% 40%". You don't understand confounding and want to test the performance of poll vs. epoll on file descriptors by testing an entire server. You think science is to use available data, when it's nothing of the sort, and in fact the way you remove confounding is t…

So where does his reasoning go wrong?

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#40

Post by [deleted] »

zedshaw wrote: post #31
jacquesm wrote: post #16

Earlier quoted context omitted.

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

You know jack squat about statistics. That's basically how I summarize your statement. You don't have a mean, median, mode, or standard deviation in your "10% 40%". You don't understand confounding and want to test the performance of poll vs. epoll on file descriptors by testing an entire server. You think science is to use available data, when it's nothing of the sort, and in fact the way you remove confounding is t…

[deleted]
Post reply on HN