Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Let's Do Some Science: Zed's Test Stack for poll vs. epoll

sheddingbikes.com

Post reply on HN
Print view View original

21–30 of 45 posts

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#21

Post by [deleted] »

echaozh wrote: post #18
tzs wrote: post #15

The network interface in general on most operating systems seems kind of absurd from a server point of view. You've got data coming in from N clients. The data from all those clients gets physically multiplexed into a single data stream that comes in over the wire. Then in the kernel you've got something that demultiplexes it back into N separate streams, which it accumulates in N different buffers. The server softwa…

It totally makes sense. You should also be able to choose to use more than one socket (but not so many as there are clients), in case you still want to take advantage of all the cores available.

[deleted]

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#23

Post by rbanffy »

zedshaw wrote: post #13
codexon wrote: post #5

Zed is probably right that poll is faster than epoll when all descriptors are active. There's no reason to doubt this since epoll has a callback for every file descriptor, while in poll, the OS only has to fill up an array of flags once per call. http://lwn.net/Articles/14168/ However, I will have to side with jacquesm in saying that regular internet servers will usually benefit from epoll. It is unfathomable that we…

There's nothing in anything I've said that disagrees with your assertion that regular web server (not internet servers) have lots of idle connections. What I've been saying all along is something very simple. If the ATR > 0.6, poll wins. If ATR That means, there's potential gains to be had by using both, and at a minimum you can use both and it won't hurt you very much. Additionally, Mongrel2 isn't a regular server.…

> there's potential gains to be had by using both

Only if your ATR crosses the .6 boundary frequently. If it consistently remains on either side, the performance would be worse at the cost of added complexity.

I agree with your assumption we need more instrumentation. I strongly suspect my ATRs are consistently closer to zero, but I never looked under the hood to check.

Also, the ATR should vary according to what you are serving. Longer streams would drive the ATR up while serving shorter bursts would drive it down.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#24

Post by isaacforce »

jacquesm wrote: post #16
zedshaw wrote: post #11

Earlier quoted context omitted.

If the test harness is wrong than every other test that everyone doing epoll comparisons with poll has done is wrong. This is the one they've all been using. So, either you write your own test to evaluate the same thing, without confounding it with other external factors (like this test does), or you use this test and prove me wrong by finding different data. However, I'm glad you finally went and got some metrics to…

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

>I'm challenging you to provide the data that apparently is still missing that proves that this was not 'a localhost test'.I think what he was saying in relation to it being not 'a localhost test' is that it isn't a test that touches the network stack, even to the loopback interface -- you seem to be equating localhost with anything that happens on a single local machine regardless of whether it has anything to do with the network or not. A difference in use of terminology, perhaps.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#25

Post by jfager »

zedshaw wrote: post #13
codexon wrote: post #5

Zed is probably right that poll is faster than epoll when all descriptors are active. There's no reason to doubt this since epoll has a callback for every file descriptor, while in poll, the OS only has to fill up an array of flags once per call. http://lwn.net/Articles/14168/ However, I will have to side with jacquesm in saying that regular internet servers will usually benefit from epoll. It is unfathomable that we…

There's nothing in anything I've said that disagrees with your assertion that regular web server (not internet servers) have lots of idle connections. What I've been saying all along is something very simple. If the ATR > 0.6, poll wins. If ATR That means, there's potential gains to be had by using both, and at a minimum you can use both and it won't hurt you very much. Additionally, Mongrel2 isn't a regular server.…

What I've been saying all along is something very simple.

If the ATR > 0.6, poll wins. If ATR

You left out the part where you accused people who advocate using epoll of being bullshitting touts who perpetuate fallacies:

"why the touted benefits of epoll and most of the information out there is mostly bullshit because of some falacies people seem to have about epoll (mostly perpetuated by epoll's proponents)."

Which you proceeded to back up with a strawman argument about claims of O(1) performance and a rant about how graphs that show that epoll accomplishes exactly what epoll is designed to accomplish "flat out get it wrong".

You're using "ATR" to find the inflection point in the performance curve from epoll's constant overhead per active fd, for the purpose of micro-optimizing Mongrel2. That's fine. But when you claim that people who just use epoll because it solves the problem they actually need solved are making decisions based on "bullshit", you shouldn't act so surprised that they don't respond positively.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#26

Post by yummyfajitas »

derefr wrote: post #10
codexon wrote: post #5

Zed is probably right that poll is faster than epoll when all descriptors are active. There's no reason to doubt this since epoll has a callback for every file descriptor, while in poll, the OS only has to fill up an array of flags once per call. http://lwn.net/Articles/14168/ However, I will have to side with jacquesm in saying that regular internet servers will usually benefit from epoll. It is unfathomable that we…

I believe that, other than in the new-for-the-sake-of-new sysadminning camp, people almost never start by deploying Mongrel. They deploy Apache, because it's easy (every distro comes with an excellent default set of LAMP packages, for obvious reasons.) Thus, Mongrel (or any other non-mindshare-majority server) is only sought out by those who don't like the one they've got—that is, by people who are not getting the pe…

Thus, Mongrel (or any other non-mindshare-majority server) is only sought out by those who don't like the one they've got - that is, by people who are not getting the performance they desire.

This is probably true for most non-apache servers now, but I don't think it will necessarily be true for Mongrel2.

Mongrel2 gives us a new development model - http calls -> messages, language doesn't matter. I can easily mix python and c++/Haskell, for instance [1]. It's rather different from what we've had before.

Mongrel isn't just a faster apache; people might use Mongrel2 for features rather than performance.

[1] Last time I tried to do this, I basically put a halfassed version of mongrel2 behind apache/django. I.e., apache/django handled http and sent messages to my c++ daemons. Fugly.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#27

Post by gthank »

jacquesm wrote: post #16
zedshaw wrote: post #11

Earlier quoted context omitted.

If the test harness is wrong than every other test that everyone doing epoll comparisons with poll has done is wrong. This is the one they've all been using. So, either you write your own test to evaluate the same thing, without confounding it with other external factors (like this test does), or you use this test and prove me wrong by finding different data. However, I'm glad you finally went and got some metrics to…

You should read more carefully. 40% was the highest observed, 10% the lowest. Also, I said 'the majority', which is not like making it seem like it is only 10%. So it's not '10%' or '40%', for this particular workload, on average you're hovering somewhere between 25 and 30%, since you seem to prefer the higher number make that 30%. And if you read a bit more you would see that there are some factors that could pull t…

Having followed most of the comments on both these threads, I don't think it's fair to characterize Zed's position as saying "epoll is always bad". I think a better characterization would be: "epoll is not automagically better under every imaginable scenario".

You may argue that nobody is making any such claim, but I can tell you that the sections of the blogosphere I skim via HN, reddit, etc. do imply it. Maybe the authors just assume everyone can infer that every decision has tradeoffs and there's no need to discuss them in depth for this particular case, or maybe—and the nature of the blogosphere leaves me inclined to think that this is the case—most of those authors never consciously thought about those tradeoffs because epoll happens to be a good fit for their particular problem. Either way, Zed is the first person I've seen trying to explicitly identify and quantify those trade-offs for poll vs. epoll. As such, I think he's doing the community a service.

If this research leads to a better implementation of epoll, or a server that requires less tuning to do well under a particular load, that is great for everyone. Even if it doesn't lead to any great improvements, a better-educated developer population that makes more informed decisions is still a net win for the world.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#28

Post by rbanffy »

zedshaw wrote: post #12
pphaneuf wrote: post #8

Earlier quoted context omitted.

Adding/removing from the poll() array is more or less free, but with epoll, you have to use epoll_ctl(), so you'd definitely want an hysteresis to avoid bouncing things around too much.

Interesting, you're like the 3rd or 4th person who's mentioned hysteresis.

It's a fairly common idea.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#29

Post by zedshaw »

jfager wrote: post #25
zedshaw wrote: post #13

Earlier quoted context omitted.

There's nothing in anything I've said that disagrees with your assertion that regular web server (not internet servers) have lots of idle connections. What I've been saying all along is something very simple. If the ATR > 0.6, poll wins. If ATR That means, there's potential gains to be had by using both, and at a minimum you can use both and it won't hurt you very much. Additionally, Mongrel2 isn't a regular server.…

What I've been saying all along is something very simple. If the ATR > 0.6, poll wins. If ATR You left out the part where you accused people who advocate using epoll of being bullshitting touts who perpetuate fallacies: "why the touted benefits of epoll and most of the information out there is mostly bullshit because of some falacies people seem to have about epoll (mostly perpetuated by epoll's proponents)." Which y…

The touted benefits of epoll are mostly bullshit, since it was advocated as being O(1) and frequently mentioned as being O(1) and always faster than poll by others. In fact, right after mentioning that I thought epoll wasn't faster the first thing people said was it was O(1). Every person I talked to said it.

I have evidence that contradicts both of the main assertions of epoll, so until someone comes up with counter evidence I can safely say that epoll information is bullshit, and my argument is nothing like a straw man.

So yes, it is bullshit, and if that offends you then awesome. Because nothing's worse than blindly believing some load of crap when you could have a better understanding of what you're doing.

Re: Let's Do Some Science: Zed's Test Stack for poll vs. epoll

#30

Post by zedshaw »

rbanffy wrote: post #23
zedshaw wrote: post #13

Earlier quoted context omitted.

There's nothing in anything I've said that disagrees with your assertion that regular web server (not internet servers) have lots of idle connections. What I've been saying all along is something very simple. If the ATR > 0.6, poll wins. If ATR That means, there's potential gains to be had by using both, and at a minimum you can use both and it won't hurt you very much. Additionally, Mongrel2 isn't a regular server.…

> there's potential gains to be had by using both Only if your ATR crosses the .6 boundary frequently. If it consistently remains on either side, the performance would be worse at the cost of added complexity. I agree with your assumption we need more instrumentation. I strongly suspect my ATRs are consistently closer to zero, but I never looked under the hood to check. Also, the ATR should vary according to what you…

Exactly, nobody knows because they've been operating under the assumption that epoll is always faster so they never looked.

Now I've got some evicence that epoll isn't faster, and in fact it's a wash at 40% "utilization", which is sort of ridiculous. I'd want people to go and test and see what they get, or at least know the implications so they pick wisely.

Ultimately though, I'd rather have the Linux kernel just fix epoll so it's always faster than poll.

Post reply on HN