Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Zed Shaw: "poll, epoll, science, and superpoll" with R

sheddingbikes.com

Post reply on HN
Print view View original

101–110 of 149 posts

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#101

Post by jacquesm »

mfukar wrote: post #45
jacquesm wrote: post #13

Earlier quoted context omitted.

> A conclusion reached by measurement is not premature. That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program. By doing these tests in isolation instead of while running on a profiling kernel under production load it is very well possible that the bot…

If you have 10K sockets open then typically poll/epoll will return a large number of 'active' descriptors, ... That's half true - it doesn't hold for low ATR traffic (lots of hanging connections, clients that GET something, spend time elsewhere while in the meantime the browser keeps the connection alive). In short, there's nothing typical about it because, while those two kinds of loads have been studied extensively…

Large in an absolute sense, say 3K for a pool of 10K sockets. That's a sizeable number of active connections to deal with after a single system call. Typically for each of those fds you'll then do a read or a write. So the epoll/poll overhead is fairly small, with epoll coming out a bit faster than poll in that situation.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#102

Post by zedshaw »

KirinDave wrote: post #77

Is it just me, or did Zed not describe his testing methodology in any detail? I can't even find a reference to his OS configuration and version details that he's developing on, which seems to me like a critical detail.

There's the pipetest.c file that everyone uses (since 2002) linked off that blog post, but I got tired and went to sleep.

Today I'm crafting how I ran the tests and releasing all the code and asking everyone to test my results. I am completely assuming I am wrong so looking for other people to test it.

Incidentally, if you google for "pipetest.c" you'll it's kind of the gold standard for this comparison, so if that code is wrong, then the entire assumption that epoll is better needs to be redone.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#103

Post by rortian »

zedshaw wrote: post #95
rortian wrote: post #85

Earlier quoted context omitted.

If only you practiced what you preached. Imagine the amount of self-righteous bile that your servers wouldn't serve.

I am practicing what I preach, you stupid 20%-er troll. That's why the whole blog post is full of measurements, testing hypotheses, and the assumption that I might be wrong. Because unlike you, I actually go do shit rather than spout off in a comment thread.

You may think you are. Far out. Why you think you can violate basic social norms, while others apparently support it, is beyond me.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#104

Post by KirinDave »

zedshaw wrote: post #102
KirinDave wrote: post #77

Is it just me, or did Zed not describe his testing methodology in any detail? I can't even find a reference to his OS configuration and version details that he's developing on, which seems to me like a critical detail.

There's the pipetest.c file that everyone uses (since 2002) linked off that blog post, but I got tired and went to sleep. Today I'm crafting how I ran the tests and releasing all the code and asking everyone to test my results. I am completely assuming I am wrong so looking for other people to test it. Incidentally, if you google for "pipetest.c" you'll it's kind of the gold standard for this comparison, so if that c…

Okay. And I appreciate that, I'll look.

To make your process scientific, I'd like to suggest you add the following things to the post when you find it convenient:

1. A detailed explanation of your methodology, preferably with source code. This is so we can reproduce the tests. The ability to reproduce your work is a critical part of any process calling itself science.

2. A detailed list of the hardware you used & its deployment. (For reasons listed above).

3. Your raw data should be made available upon request so other people can work it as well.

P.S., aren't you concerned about I/O overhead with your superpoll proposal? It seems like the added resource allocation and the time spent in zeromq is going to eat up the small advantages you gain?

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#105

Post by blasdel »

zedshaw wrote: post #98
blasdel wrote: post #86

Earlier quoted context omitted.

> but where's the evidence what people see for active/total ratios in the real world? Yes Zed, where the fuck is it? You're claiming SCIENCE! based on your worst-case synthetic localhost benchmarks, and then turning around and wildly guessing as to real-world performance characteristics with internet latencies. Worse, your whole thesis hinges off of ATR but you made no effort to measure it anywhere, instead you're pa…

Wow here we are again, you not reading my article. I ran the same test that everyone else runs for poll vs. epoll, then used R to craft graphs and tested hypothesis. It was not a localhost test. So far all you've got is trolling HN comments. YOU WIN!

That you didn't say anything about the machines you ran it on was a huge red flag. It looks like jacquesm has you nailed on the localhost testing, I hadn't noticed the link to the code before.

I'll trust that you accurately measured the ATR boundary between poll and epoll in your specific synthetic benchmark. That's then completely undermined by your handwaving in this thread about what ATR looks like in the wild, and the lack of any way for us to relate your microbenchmark with the real world.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#106

Post by blueben »

jacquesm wrote: post #75
zedshaw wrote: post #64

Earlier quoted context omitted.

Again with this idea that Mongrel2 isn't working. You sir have no freaking idea what you're talking about. "That's just plain wrong. Premature optimisation does not refer to having to measure before you optimise, it refers to optimising things that in practice may have little or no effect on the actual performance of the program." No, that's just plain wrong. Premature optimisation is actually implementing something…

So, if it's working why not throw a load of real world traffic at it in stead of this 'science' that you're performing here ? After all, that is where the rubber meets the road and it would be a very easy way to determine if your hunch is right or not. Epoll was specifically created with that sort of workloads in mind, your 'surprising' conclusion is not rooted in the fact that epoll is somehow behaving in a way that…

Because "real world traffic" is a bullshit test. Who's real world traffic? Mine? Yours? Google's? Yahoo's? Science is repeatable, which means you can replicate the same inputs to the test every time. The lab experiments involved in science are often highly sanitized and not "real world" at all. Determining what the results mean in the real world comes later.

If you're going to complain about science, at least understand how it works.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#107

Post by jacquesm »

zedshaw wrote: post #98
blasdel wrote: post #86

Earlier quoted context omitted.

> but where's the evidence what people see for active/total ratios in the real world? Yes Zed, where the fuck is it? You're claiming SCIENCE! based on your worst-case synthetic localhost benchmarks, and then turning around and wildly guessing as to real-world performance characteristics with internet latencies. Worse, your whole thesis hinges off of ATR but you made no effort to measure it anywhere, instead you're pa…

Wow here we are again, you not reading my article. I ran the same test that everyone else runs for poll vs. epoll, then used R to craft graphs and tested hypothesis. It was not a localhost test. So far all you've got is trolling HN comments. YOU WIN!

Pipes, localhost, who is counting, as far as I'm concerned that's the same thing, making it seem as if for the purpose of this test that's a significant difference is simply conversational trickery.

If you have tested this on real live servers then there is no evidence of that in your posting, and to suggest that this:

http://dpaste.de/32o8/

is anything but a localhost test is simply bogus.

The only use case where you may be right that poll is advantageous as far as I can see is streaming media servers (video, audio, other large files), image servers are the ones with the worst active-to-total ratios, especially if the images are small. I should know, I only serve up a few billion of them every day. A few years ago or so I was stupid enough to think that video was hard, man was I ever wrong. Repeated connections to the same host, that's a much bigger killer than pumping bits.

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#108

Post by FooBarWidget »

kqueue wrote: post #88
FooBarWidget wrote: post #78

Earlier quoted context omitted.

Yes but the argument to poll is a pointer. The pointer would be copied but the kernel can still follow the pointer to userspace, right?

The pointer referred to by the process is not accessible by the kernel because when the user process was running, it had a different vm space than the kernel vm space. So if it just passes the pointer (without copying the pointer's data), then the kernel will point to a virtual address that won't exist until the user process gets swapped in again.

This sounds really strange to me. The kernel has full access to the page tables so can't it lookup things in userspace?

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#109

Post by blasdel »

zedshaw wrote: post #94
blasdel wrote: post #89

Earlier quoted context omitted.

How about you get measurements of ATR from real-world deployments instead of the wild conjectures you've laid in this thread? Your challenge applies even moreso to yourself: Measure it or GTFO.

Oh, you mean do what I'm already doing? Measuring and developing ideas then testing them? It helps if you're going to comment that you actually read the words I use, not the ones you have in your head that make you sound like you're super smart.

Yes, you measured the ideal ATR inflection point for poll vs. epoll in your synthetic microbenchmark.

But you guessed wildly about what ATRs people see in the real world: http://news.ycombinator.com/item?id=1572292 http://news.ycombinator.com/item?id=1572418

Re: Zed Shaw: "poll, epoll, science, and superpoll" with R

#110

Post by kqueue »

zedshaw wrote: post #71
kqueue wrote: post #46

Lets assume we have 20k opened FDs. In case of poll(), you have to transfer this array of FDs from the userland vm to the kernel vm each time you call poll(). Now compare this with epoll() (let's assume we are using EPOLLET trigger), when you only have to transfer the file descriptors once. You might say the copying won't matter, but it will matter when you have a lot of events coming on the 20k FDs which eventually…

Yep, that's what I thought too, that at least epoll would be as fast. Turns out it's not though, but then I could be wrong. Also, your assumption of EPOLLET is potentially wrong. I think (unproven) that the extra overhead and complexity of using edge trigger right makes EPOLLET pointless.

Sorry, I meant level-triggered. :) I think edge-triggered does add an extra overhead as you stated.
Post reply on HN