Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zed Shaw: "poll, epoll, science, and superpoll" with R

sheddingbikes.com

Post reply on HN
Print view View original

31–40 of 149 posts

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#31

Post by jerf »

[deleted] wrote: post #19

[deleted]

You know, I don't really like this sort of comment. Someone goes to great effort to empirically verify something, something that is against common wisdom, then someone gets to just float in with 20/20 hindsight and say "Well, yeah, duh."

If it was so "yeah, duh", where's comment your about how blindingly obvious it was that common wisdom was wrong that dates from before somebody demonstrated it with concrete data and large test runs?

All you'd have to do to improve your comment is remove the last "Stop the presses" snark.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#32

Post by jacquesm »

terra_t wrote: post #30
jacquesm wrote: post #2

In real-life web serving situations, and not in benchmarks, the majority of the fds is not active. It's the slow guys that kill you. A client on a fast connection will come in and will pull the data as fast as the server can spit it out, keeping the process and the buffers occupied for the minimum amount of wall clock time and the number of times the 'poll' cycle is done is very small. But the slowpokes, the ones on…

Yeah, but there's a fetishization of "high concurrency" (being able to support a huge number of connections) rather than absolute performance. For instance, you might have a system which has a latency of 1 second, and at a given workload, you have 10,000 connections. In the Java culture, people think you're a genius if you can increase those connections to 100,000 and increase the latency to 10 seconds. End users, on…

If you could increase the number of connections to 100,000 you would indeed be a genius because when you bind to a network interface using IPV4 there is a hard limit of the short integer used to indicate the port number which automatically limits you to 65536 connections (actually a few less, usually you'll lose 3 for stdin,stdout and stderr (which you can close to reuse them) and one for the listen socket).

As far as I know the only way around this is to use multiple IPS (possibly aliases on the same interface) but that would still require a new process.

So even if your per-process limit for fds can be larger than 64K the network layer or the mapper that turns fds in to socket ids for the network stack to work with may impose a restriction. I don't know enough about the linux kernel to figure out what exactly causes this.

I use the 64K limit on some high throughput machines (mostly video and image servers), but when I go over that I need to start another process. Possibly there's a way around that but the expense of another process is fairly small so I haven't put in much time to see if I can work around that. Socket to fd mapping presumably takes in to account the address as well as the port so it shouldnt't be a problem but on the kernel of the machines where I have to resort to these tricks it appears to be a limit.

Maybe someone with more knowledge of the guts of the linux kernel can point out why this happens.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#33

Post by [deleted] »

jacquesm wrote: post #32
terra_t wrote: post #30

Earlier quoted context omitted.

Yeah, but there's a fetishization of "high concurrency" (being able to support a huge number of connections) rather than absolute performance. For instance, you might have a system which has a latency of 1 second, and at a given workload, you have 10,000 connections. In the Java culture, people think you're a genius if you can increase those connections to 100,000 and increase the latency to 10 seconds. End users, on…

If you could increase the number of connections to 100,000 you would indeed be a genius because when you bind to a network interface using IPV4 there is a hard limit of the short integer used to indicate the port number which automatically limits you to 65536 connections (actually a few less, usually you'll lose 3 for stdin,stdout and stderr (which you can close to reuse them) and one for the listen socket). As far a…

[deleted]

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#34

Post by jbeda »

jacquesm wrote: post #32
terra_t wrote: post #30

Earlier quoted context omitted.

Yeah, but there's a fetishization of "high concurrency" (being able to support a huge number of connections) rather than absolute performance. For instance, you might have a system which has a latency of 1 second, and at a given workload, you have 10,000 connections. In the Java culture, people think you're a genius if you can increase those connections to 100,000 and increase the latency to 10 seconds. End users, on…

If you could increase the number of connections to 100,000 you would indeed be a genius because when you bind to a network interface using IPV4 there is a hard limit of the short integer used to indicate the port number which automatically limits you to 65536 connections (actually a few less, usually you'll lose 3 for stdin,stdout and stderr (which you can close to reuse them) and one for the listen socket). As far a…

TCP connections are identified by the (src ip, src port, dest ip, dest port) tuple. The server only needs one port. So theoretically a server can handle 64k connections per client.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#35

Post by silentbicycle »

kunley wrote: post #26

Cool experiment Mr Zed, but what about kqueue? It seems superior to both *poll minions. Would be great if you proved/falsified this thesis as well.

kqueue is on OpenBSD and FreeBSD, while epoll is from Linux. (poll and select are on both)

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#36

Post by jacquesm »

jbeda wrote: post #34
jacquesm wrote: post #32

Earlier quoted context omitted.

If you could increase the number of connections to 100,000 you would indeed be a genius because when you bind to a network interface using IPV4 there is a hard limit of the short integer used to indicate the port number which automatically limits you to 65536 connections (actually a few less, usually you'll lose 3 for stdin,stdout and stderr (which you can close to reuse them) and one for the listen socket). As far a…

TCP connections are identified by the (src ip, src port, dest ip, dest port) tuple. The server only needs one port. So theoretically a server can handle 64k connections per client.

But, a server can have multiple IPS, so a server should be able to handle more than 64K connections from multiple clients without a problem. In practice there appears to be some kind of limit.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#37

Post by mfukar »

jacquesm wrote: post #11
pmjordan wrote: post #7

Earlier quoted context omitted.

Thanks for the detailed explanation. Sounds like I was at least on the right track. But if you ask me, the real solution is to have the kernel team fix their epoll implementation performance issues instead of forcing people to work around it with hybrid approaches. That does indeed sound like a better conclusion. Other than the stupid single-syscall-per-fd requirement, there's nothing in epoll's interface that would…

I highly doubt that in production it will make any difference at all. In the end it is not the poll/epoll overhead that determines your overall throughput. If you call poll/epoll more frequently than you should then it starts to add up but in reality one call per several thousand file system operations doing real IO is not going to make a big difference. Of course all the little bits help and I'm happy to see someone…

I keep seeing this...misconception? I don't know what to call it, over most comments. The article doesn't talk about optimizations. It talks about design decisions. Someone actually took the time to sit down, ponder what kind of workloads will be handled by his application, came across an interesting dilemma, measured and tested them both, and finally reached an educated conclusion.

That right there is how you correctly choose how to implement things, contrary to popular belief. Think first, write code later.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#38

Post by adamtj »

jacquesm wrote: post #36
jbeda wrote: post #34

Earlier quoted context omitted.

TCP connections are identified by the (src ip, src port, dest ip, dest port) tuple. The server only needs one port. So theoretically a server can handle 64k connections per client.

But, a server can have multiple IPS, so a server should be able to handle more than 64K connections from multiple clients without a problem. In practice there appears to be some kind of limit.

The server doesn't need multiple IPs to handle > 65535 connections. All the server connections to a given IP are to the same port. For a given client, the unique key for an http connection is (client-ip, PORT, server-ip, 80). The only number that can vary is PORT, and that's a value on the client. So, the client is limited to 65535 connections to the server. But, a second client could also have another 65K connections to the same server-ip:port.

edit: You may be limited by number of open sockets or file handles. It's likely a per-process limit. Google or some linux guru could help you track down what limit it actually is, but it's not the number of server ports available. It might be a number you could raise.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#39

Post by kunley »

silentbicycle wrote: post #35
kunley wrote: post #26

Cool experiment Mr Zed, but what about kqueue? It seems superior to both *poll minions. Would be great if you proved/falsified this thesis as well.

kqueue is on OpenBSD and FreeBSD, while epoll is from Linux. (poll and select are on both)

I'm aware of it (you forgot to mention that kqueue is on the OS X as well). So what?

There are probably hordes of people who will be willing to run Mongrel2 on *BSD platforms, precisely because of the performance reasons. And Zed is a famous tinkerer rather than a religious zealot, so very probably he could be interested in checking kqueue as well.

"Why not" is also a good reason for a hacker when he's lacking other reasons.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#40

Post by jacquesm »

adamtj wrote: post #38
jacquesm wrote: post #36

Earlier quoted context omitted.

But, a server can have multiple IPS, so a server should be able to handle more than 64K connections from multiple clients without a problem. In practice there appears to be some kind of limit.

The server doesn't need multiple IPs to handle > 65535 connections. All the server connections to a given IP are to the same port. For a given client, the unique key for an http connection is (client-ip, PORT, server-ip, 80). The only number that can vary is PORT, and that's a value on the client. So, the client is limited to 65535 connections to the server. But, a second client could also have another 65K connection…

Right, that makes perfect sense. But it really makes me wonder why I run in to that hard limit, I've tried just about everything to get around it and no matter what I do that seems to be the magic number.

I should go and do some testing to see what's causing this, you make me feel like the solution is right around the corner.

re. your edit, ulimit will happily raise the number > 64K, all the /proc/* settings seem to be ok so that's not it, it has to be some other layer in the stack that causes this. I'll definitely spend some time on this, it's been bugging me for a long time.

edit2: there seems to be a max_user_watches upper limit to what epoll will handle.

Post reply on HN