Zed Shaw: "poll, epoll, science, and superpoll" with R • Hacker News Board

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#21

Post by axod » Tue, Aug 03, 2010, 1:15 PM UTC

jakevoytko wrote: post #12

axod wrote: post #9

Sounds like premature optimization to me. Is this really the bottleneck? Is the extra complexity and logic really going to be a net win?

A conclusion reached by measurement is not premature. This looks like an attempt to write a better server than the 80/20 rule allows. If he's wrong and only one polling method is useful in production, the live servers will pick the good one and nobody will suffer because he jumped to conclusions. Since he's written Mongrel, I trust that he has a reason to worry about polling that may not have appeared in the post

FWIW, I've written an async webserver which handles a few thousand concurrent users, and does thousands of HTTP reqs/second. I've never seen poll/epoll as a bottleneck, but I'm using Java NIO which really seems to work extremely well (I don't remember how/which it uses, I think epoll).

Maybe Java does some of this cool stuff already so perhaps I'm shielded from the pain of dealing with things directly.

In the past I've written Java NIO code that dealt with around 60,000 concurrent connections pretty well. The time spent doing poll seemed to be completely insignificant. CPU usage was negligible.

It'd be good to see some numbers though - for example:

For average mongrel application, 40% of CPU time is spent in poll / average of 30ms latency is due to poll etc.

But I'm skeptical those numbers are true. That was my point.

If you don't start with those numbers and measurements, optimizations like this, whilst interesting, may end up being of no real use to anyone.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#22

Post by jacquesm » Tue, Aug 03, 2010, 1:17 PM UTC

jakevoytko wrote: post #20

jacquesm wrote: post #13

Earlier quoted context omitted.

> A conclusion reached by measurement is not premature. That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program. By doing these tests in isolation instead of while running on a profiling kernel under production load it is very well possible that the bot…

"I need polling" => "Here are my options, which one is better?" => "They're good for different things" => "I'll pick the best one for the environment" is a reasonable design process. More so than some decisions that I make! Yes, there's a fixed time budget. But you're suggesting selectively ignoring evidence when designing a program, preferring random guessing and pattern matching to actual numbers. Should he have co…

The funny thing is that he's either going to pick the wrong solution for the workload or spend a lot of time on creating a hybrid which will work as good as epoll would without augmentation.

Zed is clearly out to change the world and I would very much like him to succeed but he just seems to be missing the obvious here, which is that idle connections are the ones to worry about (because they're very expensive!) so his benchmark at this point in time is useless.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#23

Post by axod » Tue, Aug 03, 2010, 1:24 PM UTC

jakevoytko wrote: post #20

jacquesm wrote: post #13

Earlier quoted context omitted.

> A conclusion reached by measurement is not premature. That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program. By doing these tests in isolation instead of while running on a profiling kernel under production load it is very well possible that the bot…

"I need polling" => "Here are my options, which one is better?" => "They're good for different things" => "I'll pick the best one for the environment" is a reasonable design process. More so than some decisions that I make! Yes, there's a fixed time budget. But you're suggesting selectively ignoring evidence when designing a program, preferring random guessing and pattern matching to actual numbers. Should he have co…

"I need a filesystem" => "Here are my options, which one is better?" => "They're good for different things"...

STOP

For most things, it doesn't matter. A filesystem is a filesystem.

You only need to make decisions like that when you properly measure and decide that X may be a bottleneck, or you need features that Y has but X doesn't.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#24

Post by axod » Tue, Aug 03, 2010, 1:29 PM UTC

statictype wrote: post #10

axod wrote: post #9

Sounds like premature optimization to me. Is this really the bottleneck? Is the extra complexity and logic really going to be a net win?

Looks like he's already written a large bulk of his server's code, so maybe this optimization isn't really premature :) You're probably right that when you actually use Mongrel2 as your app server your app-specific code higher up will be a larger bottleneck, but that's code that you have to deal with and this is code that he has to deal with so optimizing the hell out of it doesn't sound like a bad idea.

Lets say 95% of time is in your code, and 5% is in Mongrel2. Lets say that within Mongrel2, 10% of time is in this poll/epoll stuff.

That's 0.5% of your total time being spent here. So even if it's made twice as fast, your app will only speed up by say 100ms -> 99.75ms

Find the big things that matter and optimize them. Adding extra complexity to small things that don't matter is a recipe for more bugs and more issues.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#25

Post by pmjordan » Tue, Aug 03, 2010, 1:46 PM UTC

axod wrote: post #24

statictype wrote: post #10

Earlier quoted context omitted.

Looks like he's already written a large bulk of his server's code, so maybe this optimization isn't really premature :) You're probably right that when you actually use Mongrel2 as your app server your app-specific code higher up will be a larger bottleneck, but that's code that you have to deal with and this is code that he has to deal with so optimizing the hell out of it doesn't sound like a bad idea.

Lets say 95% of time is in your code, and 5% is in Mongrel2. Lets say that within Mongrel2, 10% of time is in this poll/epoll stuff. That's 0.5% of your total time being spent here. So even if it's made twice as fast, your app will only speed up by say 100ms -> 99.75ms Find the big things that matter and optimize them. Adding extra complexity to small things that don't matter is a recipe for more bugs and more issues…

I haven't actually used Mongrel2 yet, but I get the impression you can use it as a front-end for multiple physical servers, in which case the relative load of the M2 process on its server increases. Besides, unless Zed is writing your application and this is eating away at his time for that, it's not clear what exactly your complaint is.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#26

Post by kunley » Tue, Aug 03, 2010, 1:53 PM UTC

Cool experiment Mr Zed, but what about kqueue?

It seems superior to both *poll minions. Would be great if you proved/falsified this thesis as well.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#27

Post by frognibble » Tue, Aug 03, 2010, 1:54 PM UTC

The blog post does not say if the epoll code uses level triggering or edge triggering. It would be interesting to see the results for both modes. The smaller number of system calls required for edge triggering might make a difference in performance.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#28

Post by [deleted] » Tue, Aug 03, 2010, 1:57 PM UTC

axod wrote: post #24

statictype wrote: post #10

Earlier quoted context omitted.

Looks like he's already written a large bulk of his server's code, so maybe this optimization isn't really premature :) You're probably right that when you actually use Mongrel2 as your app server your app-specific code higher up will be a larger bottleneck, but that's code that you have to deal with and this is code that he has to deal with so optimizing the hell out of it doesn't sound like a bad idea.

Lets say 95% of time is in your code, and 5% is in Mongrel2. Lets say that within Mongrel2, 10% of time is in this poll/epoll stuff. That's 0.5% of your total time being spent here. So even if it's made twice as fast, your app will only speed up by say 100ms -> 99.75ms Find the big things that matter and optimize them. Adding extra complexity to small things that don't matter is a recipe for more bugs and more issues…

[deleted]

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#29

Post by rbanffy » Tue, Aug 03, 2010, 2:14 PM UTC

pmjordan wrote: post #25

axod wrote: post #24

Earlier quoted context omitted.

Lets say 95% of time is in your code, and 5% is in Mongrel2. Lets say that within Mongrel2, 10% of time is in this poll/epoll stuff. That's 0.5% of your total time being spent here. So even if it's made twice as fast, your app will only speed up by say 100ms -> 99.75ms Find the big things that matter and optimize them. Adding extra complexity to small things that don't matter is a recipe for more bugs and more issues…

I haven't actually used Mongrel2 yet, but I get the impression you can use it as a front-end for multiple physical servers, in which case the relative load of the M2 process on its server increases. Besides, unless Zed is writing your application and this is eating away at his time for that, it's not clear what exactly your complaint is.

> it's not clear what exactly your complaint is.

Any complexity introduced in the code increases its long term maintenance cost. I strongly suspect this is one of those cases where the performance gain will not justify the long-term effort of maintaining a more complex architecture.

I remember having a similar discussion circa 1992 about advantages and disadvantages of using ODBC versus native MS SQL/Sybase libraries. I instrumented the program I was writing and showed it spent 99+% of the time idling, 1-% of the time computing and, of that time, about 78% waiting for the database to return something. Using native libraries would yield a minuscule improvement at the cost of a huge headache.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#30

Post by terra_t » Tue, Aug 03, 2010, 2:19 PM UTC

jacquesm wrote: post #2

In real-life web serving situations, and not in benchmarks, the majority of the fds is not active. It's the slow guys that kill you. A client on a fast connection will come in and will pull the data as fast as the server can spit it out, keeping the process and the buffers occupied for the minimum amount of wall clock time and the number of times the 'poll' cycle is done is very small. But the slowpokes, the ones on…

Yeah, but there's a fetishization of "high concurrency" (being able to support a huge number of connections) rather than absolute performance.

For instance, you might have a system which has a latency of 1 second, and at a given workload, you have 10,000 connections. In the Java culture, people think you're a genius if you can increase those connections to 100,000 and increase the latency to 10 seconds.

End users, on the other hand, would be happier if you cut the latency to 0.1 seconds, but there are a lot of people who'll then think you're a loser who can only manage to handle 1000 concurrent connections.

Of course, getting that latency down is a holistic process that requires you to think about the client, the server, and what exactly goes over the wire.