Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zed Shaw: "poll, epoll, science, and superpoll" with R

sheddingbikes.com

Post reply on HN
Print view View original

91–100 of 149 posts

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#91

Post by gchpaco »

zedshaw wrote: post #62
neilc wrote: post #59

Earlier quoted context omitted.

It's entirely possible that you have servers that are at 80-90% ATR all the time I'd be curious if you have any evidence that this occurs in practice. Even a busy server with clients of uniform + low latency, intuitively I'd expect fairly low ATRs. I think what's more important in what I'm finding is that you really need both. I'm not sure you do: the performance advantage of poll seems marginal at best. When ATR is…

This is the point where talking about it does nothing. Go measure it like I have. In fact, I'll give you your hypothesis to test: "There are no servers that have an ATR of > 80%." That's easy to test, and I'm damn positive you could find some that disprove your assertion. More importantly though, you have this assertion: "Using both poll and epoll has no advantage in performance." Again, who knows, that's why I'm tes…

You have measured nothing about real world workloads. You have applied completely artificial benchmarks and formed some possible conclusions from them that mean absolutely nothing until you demonstrate that this is a real problem in the real world. There's ample evidence to believe that FDs spend the majority of their existence idle--between HTTP keepalive, processing time for the queries themselves, network bandwidth and the generally sparse nature of HTTP traffic, it is extremely plausible that the ATR, as you call it, is well below 60% almost all the time. Your numbers demonstrate that there is a tradeoff between epoll and poll, which is of some interest, but unless you actually measure the ATR of a real site you're shooting off your mouth.

I cannot collect this data because I don't possess a sufficiently high load web server. Go forth and measure, but measure useful information.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#92

Post by rbanffy »

pmjordan wrote: post #81
rbanffy wrote: post #76

Earlier quoted context omitted.

1) Will you maintain Mongrel forever? It's not your track record that's the question, but the one of all future maintainers of Mongrel that will have to deal with the added complexity this change creates. 2) The experience from 1992 still seems current. Adding complexity to any software project adds cost to maintain it in the future. My experience in 1992 showed how added complexity for a marginal performance gain di…

Will you maintain Mongrel forever? With all due respect, this seems a ridiculous question. Do you obtain written statements to the above effect from all maintainers of software (open source or otherwise) before using it? Yes, it'll have to be maintained, but Zed's article alone is more documentation than you could ever hope for from most programmers. It's clear he isn't "most programmers" but that's no reason to hold…

I am not holding him to ridiculous standards. It's obvious he won't maintain Mongrel forever. What I am confronting him is with the fact that he may be able to navigate any arbitrarily clever construct he invents, but that future maintainers who inherit Mongrel may not be as capable.

Again, it's his project and he's free to do whatever he feels like with it. Heck... I am not even a user. I offered him advice he is free to disregard. The fact my cleverness has been biting me ever since my 6502 days (quite likely before Zed was born) is a problem I sort of learned to deal with long ago - but it still comes to bite me from time to time.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#93

Post by jacquesm »

zedshaw wrote: post #58
jacquesm wrote: post #2

In real-life web serving situations, and not in benchmarks, the majority of the fds is not active. It's the slow guys that kill you. A client on a fast connection will come in and will pull the data as fast as the server can spit it out, keeping the process and the buffers occupied for the minimum amount of wall clock time and the number of times the 'poll' cycle is done is very small. But the slowpokes, the ones on…

So let's take your assertions and take them apart: > the ones on dial up and on congested lines will get you every time. Do you have numbers on the dial-up users for your server? My understanding is that there's far fewer, so this is bogus. Show evidence of high dial-up penetration first. > They keep the processes busy far longer than you'd want and you have to hit the 'poll' cycle far more frequently Again, you have…

So, here are the numbers from one of the webservers that I instrumented to log the active-to-total ratio over a couple of hours.

The webserver is custom job called yawwws (yet-another-www-server) that is used to serve up a variety of bits and pieces for a high traffic website, typically the requests are very short in nature (a 500 byte request followed by a After about two hours of running the active-to-total ratio varied between 10% to 40% for 5 minute intervals, with the majority of the 5 minute buckets around the 30% mark. I'm actually quite surprised at the spread.

The bigger portion of the time seems to be spent waiting for the clients to send the request, most if not all of the output data should fit in the TCP output buffers, so that actually skews the results upwards, for longer running requests sending more data to the clients the active-to-total ratios would probably be a bit lower.

So 10% to 40% of all the sockets were active at any given time, the rest was idle, waiting for data to be received or for buffer space to be freed up so data could be written.

In this situation epoll would be faster than poll because epoll only sends the user process those fds that it actually has to deal with rather than all of them, so the loop that takes the output of the system call will have less iterations.

So, as I wrote before, I think the typical web server is, when it is dealing with the client facing side more often than not waiting for the client to do something, and it seems that on my server that hasn't changed since I last looked at it.

This server runs with keepalive off. Switching it on will most likely make the active-to-total ratio dramatically lower but I don't feel like pissing off a large number of users just to see how bad it could get. There is a good chance that my socket pool will turn out to be too small to do this without damage.

Chances are that for different workloads the percentages will vary but this setup is fairly typical (single threaded server, all requests served from memory) so I wouldn't expect to see too much variation on different sites, and if there is variation I'd expect it to go down rather than up.

If I get a chance I'll re-run the test on some other websites to see if the numbers come out comparable or are wildly different.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#94

Post by zedshaw »

blasdel wrote: post #89
zedshaw wrote: post #74

Earlier quoted context omitted.

And, if there's a distribution of speeds then you can measure the distribution and see what works best. Again, my challenge still stands: Measure it or STFU.

How about you get measurements of ATR from real-world deployments instead of the wild conjectures you've laid in this thread? Your challenge applies even moreso to yourself: Measure it or GTFO.

Oh, you mean do what I'm already doing? Measuring and developing ideas then testing them?

It helps if you're going to comment that you actually read the words I use, not the ones you have in your head that make you sound like you're super smart.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#95

Post by zedshaw »

rortian wrote: post #85
zedshaw wrote: post #74

Earlier quoted context omitted.

And, if there's a distribution of speeds then you can measure the distribution and see what works best. Again, my challenge still stands: Measure it or STFU.

If only you practiced what you preached. Imagine the amount of self-righteous bile that your servers wouldn't serve.

I am practicing what I preach, you stupid 20%-er troll. That's why the whole blog post is full of measurements, testing hypotheses, and the assumption that I might be wrong.

Because unlike you, I actually go do shit rather than spout off in a comment thread.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#96

Post by zedshaw »

jemfinch wrote: post #87
zedshaw wrote: post #58

Earlier quoted context omitted.

So let's take your assertions and take them apart: > the ones on dial up and on congested lines will get you every time. Do you have numbers on the dial-up users for your server? My understanding is that there's far fewer, so this is bogus. Show evidence of high dial-up penetration first. > They keep the processes busy far longer than you'd want and you have to hit the 'poll' cycle far more frequently Again, you have…

> Do you have numbers on the dial-up users for your server? My understanding is that there's far fewer, so this is bogus. Show evidence of high dial-up penetration first. He doesn't need to show that it's high , only that it's high enough to cause a significant contingent of ordinary webservers' requests to be lingering slow connections.

I agree, but "high enough" is apparently just 60%. The standing question is, what's the actual level in different kinds of servers?

In other words, I've given a metsric, ATR at 60% is the break even point for poll vs. epoll. So far the only responses I've got haven't even tried to give out a metric, let alone say what their actual ATR is but they claim that it's low.

I'm a scientist, so in the same way I don't believe my own research, I don't believe their rhetoric.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#97

Post by frio »

zedshaw wrote: post #74
bdr wrote: post #69

Earlier quoted context omitted.

Read "on dial-up" as "slow". The argument depends only on there being a certain distribution of client speeds. It's not about dial-up in particular.

And, if there's a distribution of speeds then you can measure the distribution and see what works best. Again, my challenge still stands: Measure it or STFU.

You've already got a benchmarking system configured; why not benchmark using a congested (or at least, emulated) pipe?

It doesn't necessarily depend on dial-up, either. Imagine the number of people who leave bittorrent open in the background, stream porn, or whatever else that leaves their individual HTTP connections slow. Hell, latency alone (it takes at least a second for my connection to reach the east coast of the USA) would have an effect, and you can't underestimate the increasing number of mobile devices on slow(-ish, depending on congestion) 3G networks.

I'd provide statistics from my server (I serve an NZ gaming community), but I suspect my numbers would be disproportionate compared to the average workload. Here in NZ, we have far more people on crappy pipes (our DSL network is, famously, a gigantic pile of shit - although that has improved over the past couple of years and continues to), and far less people on smartphones (iPhones cost ~$800USD here).

Still, I believe the commenter has a point which you shouldn't ignore, or at least shouldn't pass off so easily :). I'd love to do some testing myself, but unfortunately between working a day-job, and spending my evenings trying to get a startup off the ground, I've got no time spare.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#98

Post by zedshaw »

blasdel wrote: post #86
zedshaw wrote: post #57

Earlier quoted context omitted.

> And as jacquesm points out, in a web-facing server, that's the case you should care about. Yes, but where's the evidence what people see for active/total ratios in the real world? I'm showing that unless it's below about 60% (probably more like 50%) then poll is the way to go. 60% active isn't entirely unrealistic at all. I can see quite a few servers hitting those thresholds, so in that cases, poll vs. epoll doesn…

> but where's the evidence what people see for active/total ratios in the real world? Yes Zed, where the fuck is it? You're claiming SCIENCE! based on your worst-case synthetic localhost benchmarks, and then turning around and wildly guessing as to real-world performance characteristics with internet latencies. Worse, your whole thesis hinges off of ATR but you made no effort to measure it anywhere, instead you're pa…

Wow here we are again, you not reading my article. I ran the same test that everyone else runs for poll vs. epoll, then used R to craft graphs and tested hypothesis. It was not a localhost test.

So far all you've got is trolling HN comments. YOU WIN!

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#99

Post by [deleted] »

blasdel wrote: post #86
zedshaw wrote: post #57

Earlier quoted context omitted.

> And as jacquesm points out, in a web-facing server, that's the case you should care about. Yes, but where's the evidence what people see for active/total ratios in the real world? I'm showing that unless it's below about 60% (probably more like 50%) then poll is the way to go. 60% active isn't entirely unrealistic at all. I can see quite a few servers hitting those thresholds, so in that cases, poll vs. epoll doesn…

> but where's the evidence what people see for active/total ratios in the real world? Yes Zed, where the fuck is it? You're claiming SCIENCE! based on your worst-case synthetic localhost benchmarks, and then turning around and wildly guessing as to real-world performance characteristics with internet latencies. Worse, your whole thesis hinges off of ATR but you made no effort to measure it anywhere, instead you're pa…

[deleted]

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#100

Post by zedshaw »

gchpaco wrote: post #91
zedshaw wrote: post #62

Earlier quoted context omitted.

This is the point where talking about it does nothing. Go measure it like I have. In fact, I'll give you your hypothesis to test: "There are no servers that have an ATR of > 80%." That's easy to test, and I'm damn positive you could find some that disprove your assertion. More importantly though, you have this assertion: "Using both poll and epoll has no advantage in performance." Again, who knows, that's why I'm tes…

You have measured nothing about real world workloads. You have applied completely artificial benchmarks and formed some possible conclusions from them that mean absolutely nothing until you demonstrate that this is a real problem in the real world. There's ample evidence to believe that FDs spend the majority of their existence idle--between HTTP keepalive, processing time for the queries themselves, network bandwidt…

I used the same test everyone has used for the last 8 years to compare poll vs. epoll performance. I also ran it in order to test a hypothesis then assumed I was wrong then ran it more and presented the information openly so others could try it.

Of course I'm going to keep doing this, but if you say that my test is invalid, then all of the tests people did to justify epoll are invalid.

Post reply on HN