Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zed Shaw: "poll, epoll, science, and superpoll" with R

sheddingbikes.com

Post reply on HN
Print view View original

131–140 of 149 posts

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#131

Post by zedshaw »

rortian wrote: post #103
zedshaw wrote: post #95

Earlier quoted context omitted.

I am practicing what I preach, you stupid 20%-er troll. That's why the whole blog post is full of measurements, testing hypotheses, and the assumption that I might be wrong. Because unlike you, I actually go do shit rather than spout off in a comment thread.

You may think you are. Far out. Why you think you can violate basic social norms, while others apparently support it, is beyond me.

Basic social norms also involve not saying ruthless comments you wouldn't say to my face.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#132

Post by zedshaw »

jacquesm wrote: post #113
zedshaw wrote: post #96

Earlier quoted context omitted.

I agree, but "high enough" is apparently just 60%. The standing question is, what's the actual level in different kinds of servers? In other words, I've given a metsric, ATR at 60% is the break even point for poll vs. epoll. So far the only responses I've got haven't even tried to give out a metric, let alone say what their actual ATR is but they claim that it's low. I'm a scientist, so in the same way I don't believ…

> I'm a scientist I've never come across a scientist that took criticism of their work the way you do and that responded in the way you do. Shouting down, deriding, insulting and in general being a jerk to those that don't agree with you because 'you're a scientist' is not the way of science.

I'm shouting you down because you're a FUD slinging troll. Very first thing you did was immediately reply to every branch of the comments with your agenda. I actually have no idea what your problem is, since I'm just presenting some information and working on my own software with it, but you've got some weird "epoll religion" you like to spread.

So, consider me the Richard Dawkins of epoll.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#133

Post by zedshaw »

blasdel wrote: post #105
zedshaw wrote: post #98

Earlier quoted context omitted.

Wow here we are again, you not reading my article. I ran the same test that everyone else runs for poll vs. epoll, then used R to craft graphs and tested hypothesis. It was not a localhost test. So far all you've got is trolling HN comments. YOU WIN!

That you didn't say anything about the machines you ran it on was a huge red flag. It looks like jacquesm has you nailed on the localhost testing, I hadn't noticed the link to the code before. I'll trust that you accurately measured the ATR boundary between poll and epoll in your specific synthetic benchmark. That's then completely undermined by your handwaving in this thread about what ATR looks like in the wild, an…

The original article had the link to the code so don't even try to imply that I was hiding jack squat. He hasn't nailed shit, it's the same test epoll proponents have used for years. It's not a "localhost" test, it's a test that makes a bunch of file descriptors and then compares the poll vs. epoll performance as active vs. total changes.

But hey, you can live in your own little fantasy world where you think you've won some kind of battle of the HN because you listened to some epoll fanatic weirdo and cheered him on.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#134

Post by zedshaw »

jacquesm wrote: post #117
zedshaw wrote: post #100

Earlier quoted context omitted.

I used the same test everyone has used for the last 8 years to compare poll vs. epoll performance. I also ran it in order to test a hypothesis then assumed I was wrong then ran it more and presented the information openly so others could try it. Of course I'm going to keep doing this, but if you say that my test is invalid, then all of the tests people did to justify epoll are invalid.

The tests are perfectly valid, it's the conclusion that is wrong. Of course there is a trade-off, of course you'll find a cross-over point because there is a trade off. The fact that the crossover point is at roughly 60% for your kernel config and machine setup is essentially meaningless, other than that it it at least falls within the expected range (50 to 80% or so). Your hypothesis is that web servers have the maj…

"Your hypothesis is that web servers have the majority of their fds active most of the time,"

Aha! Totally wrong. My hypothesis has not been that at all. You totally didn't even understand the hypothesis, and I stated it very clearly. My hypothesis has always been:

epoll is faster than poll when the active/total FD ratio is 0.6.

Nowhere in there do I say that all web servers have the majority of their FDs active. NOWHERE. I say some might, I say who knows, I say we need to go measure, but nowhere do I say anything like what you say.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#135

Post by zedshaw »

blueben wrote: post #106
jacquesm wrote: post #75

Earlier quoted context omitted.

So, if it's working why not throw a load of real world traffic at it in stead of this 'science' that you're performing here ? After all, that is where the rubber meets the road and it would be a very easy way to determine if your hunch is right or not. Epoll was specifically created with that sort of workloads in mind, your 'surprising' conclusion is not rooted in the fact that epoll is somehow behaving in a way that…

Because "real world traffic" is a bullshit test. Who's real world traffic? Mine? Yours? Google's? Yahoo's? Science is repeatable, which means you can replicate the same inputs to the test every time. The lab experiments involved in science are often highly sanitized and not "real world" at all. Determining what the results mean in the real world comes later. If you're going to complain about science, at least underst…

Exactly, there's no concept of confounding at all. You use "real world tests" (whatever the hell that is) when you have an actual specific setup to test. You use a small model experiment like this to test one specific thing like poll vs. epoll.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#136

Post by zedshaw »

jacquesm wrote: post #111
blueben wrote: post #106

Earlier quoted context omitted.

Because "real world traffic" is a bullshit test. Who's real world traffic? Mine? Yours? Google's? Yahoo's? Science is repeatable, which means you can replicate the same inputs to the test every time. The lab experiments involved in science are often highly sanitized and not "real world" at all. Determining what the results mean in the real world comes later. If you're going to complain about science, at least underst…

There are basically three sorts of traffic that I have experience with and would expect to be the major portion of whatever goes over the web: - underwater ajax requests - regular website content (images, dynamic html, css and other relatively small (say - media servers (filedumps, video servers, streaming audio servers) Each of those requires fairly specific tuning of the TCP stack to get the most out of it, so you'…

You know what I find troubling about your behavior on this thread? It's just so weirdly manipulative. I gotta think you have like 40% stock invested in Epoll, Inc. or something. You make wild claims about what I'm saying that aren't true, you imply that I know nothing of real world performance when I've written some pretty bad ass real world software. You reply to every single thread with a constant stream of FUD and nitpicking everything you can then blowing it out of proportion.

I mean, are you sure you didn't used to work for Microsoft and then got hired by Linus to work your FUD spreading magic?

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#137

Post by adamtj »

zedshaw wrote: post #129
blasdel wrote: post #109

Earlier quoted context omitted.

Yes, you measured the ideal ATR inflection point for poll vs. epoll in your synthetic microbenchmark. But you guessed wildly about what ATRs people see in the real world: http://news.ycombinator.com/item?id=1572292 http://news.ycombinator.com/item?id=1572418

Yeah, 'cause there's no way I'll be able to test a real web server that I actually wrote based on this small test. This is a small test to test one specific thing, doing more would confound the test. Confounding. Look it up. Incidentally, this is the same test everyone else uses, so if you thought it was bullshit why did you support it when people testing epoll with it were using it? Oh, because they used it to confi…

I think he was suggesting that you find the ATR on real-world production webservers, which has nothing to do with your test and would not confound it. Maybe in practice nobody sees an ATR > 30%, in which case your test, if correct, is irrelevant. What sort of loads produce an ATR > 60%? How common are they?

One problem with getting such numbers is that real world, the load is given, not the ATR, and the ATR for a given load may depend on implementation choices. Total connections is a function of server latency, and on a loaded system, latency can be a function of polling method. So switching methods may change your ATR.

On the other hand maybe ATR for a given load doesn't change significantly depending on implementation. Your test found what is best for a given ATR, but not necessarily for a given load, or how ATR depends on load for a given implementation. Depending on the results, you may want to add some hysteresis to superpoll.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#138

Post by tel »

zedshaw wrote: post #129
blasdel wrote: post #109

Earlier quoted context omitted.

Yes, you measured the ideal ATR inflection point for poll vs. epoll in your synthetic microbenchmark. But you guessed wildly about what ATRs people see in the real world: http://news.ycombinator.com/item?id=1572292 http://news.ycombinator.com/item?id=1572418

Yeah, 'cause there's no way I'll be able to test a real web server that I actually wrote based on this small test. This is a small test to test one specific thing, doing more would confound the test. Confounding. Look it up. Incidentally, this is the same test everyone else uses, so if you thought it was bullshit why did you support it when people testing epoll with it were using it? Oh, because they used it to confi…

I think there's less disagreement about the well-controlled result derived and more about whether

   a) Your controls were right, and
   b) What the most optimal decision is in light of this new information.
(a) is well known to be one of the most difficult parts of scientific reasoning and is almost always open to endless debate and improvement. In short, it's the question of whether ATR is a human-sensible metric. (b) however has an interesting direct answer: figure out the distribution of "live" ATRs on an interesting population of real servers and then, to borrow Eliezer's phrase, shut up and multiply.

If a lot of servers that you're targeting with M2 fall across that 60% divide (under circumstances similar to your controlled microbenchmark) then of course Superpoll is a good compromise.

Jacques is arguing a combination of (a) and (b). Perhaps ATR is not a sufficient metric to understand all interesting server loads. Moreover, perhaps many interesting servers live at really low or high ATRs all the time and so Superpoll must gracefully degrade to either poll or epoll.

In any case, driving for empirical data is noble, but possessing data is never sufficient to whitewall all detractors. It's really nice to have strong empirical support for the breakeven point between the two (ie. the ratio of their constant time components) via your benchmark, but science isn't just statistics.

(edit: I'll also add that pushing the pipetest microbenchmark past where people are usually making hyperbolic claims is a pretty big deal and a good catch.)

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#139

Post by nkurz »

zedshaw wrote: post #132
jacquesm wrote: post #113

Earlier quoted context omitted.

> I'm a scientist I've never come across a scientist that took criticism of their work the way you do and that responded in the way you do. Shouting down, deriding, insulting and in general being a jerk to those that don't agree with you because 'you're a scientist' is not the way of science.

I'm shouting you down because you're a FUD slinging troll. Very first thing you did was immediately reply to every branch of the comments with your agenda. I actually have no idea what your problem is, since I'm just presenting some information and working on my own software with it, but you've got some weird "epoll religion" you like to spread. So, consider me the Richard Dawkins of epoll.

Zed, you might reconsider your social strategy. Your article lays out a brilliant theoretical approach, but it's hard figure out how to apply it without more real-world data. Pragmatically, it seems like your goal should be to encourage others who are in a position to gather this data to do so, and to share it with you.

It looks like Jacques looks has a pretty good start at making these measurements: http://news.ycombinator.com/item?id=1573145. If the numbers he provides aren't helpful, or aren't complete, you might try encouraging him to fix them. Calling him a "FUD slinging troll" seems more likely to cause him to tune out and ignore you, to the detriment of us all.

Realize that you've been thinking about this problem for a while now, while others have just started their thought process. Your goal is to get them up to speed so they can move your argument further, but this won't be instantaneous. Treating them as potential allies during this formative stage might pay off. If you can hold off with the insults for a couple hours or even days, you might get better results than if you shout them down immediately. :)

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#140

Post by rortian »

zedshaw wrote: post #131
rortian wrote: post #103

Earlier quoted context omitted.

You may think you are. Far out. Why you think you can violate basic social norms, while others apparently support it, is beyond me.

Basic social norms also involve not saying ruthless comments you wouldn't say to my face.

Please, you are being ridiculous. I would tell you that you mostly produce self-righteous bile to your face or anywhere else. Do you say data or 'STFU' to people's face?

You're arrogant and insecure. You lash out at others who question you. It's pretty sad.

People make assertions all the time, as you do, that are difficult if not impossible to verify. Sometimes the data is too hard to produce. You've noted yourself that real-world server data would be hard to produce. You might say admit that and be more humble about the fact that you've analyzed a micro-benchmark.

Post reply on HN