Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nxweb – Fast and Lightweight Web Server

nxweb.org

Post reply on HN
Print view View original

61–70 of 75 posts

Re: Nxweb – Fast and Lightweight Web Server

#61

Post by hueving »

zzzcpan wrote: post #55
JoeAltmaier wrote: post #50

Earlier quoted context omitted.

Re: gallons of gas. There's the old puzzle: your spouse gets 100MPG in that super-hybrid-mobile. The salesperson wants to upgrade you for $1000 to the super-duper-hybrid-mobile at 200MPG! Double the mileage! You suggest instead that you get the old truck serviced and replace the plugs, distributor and tailpipe. Estimated cost $1000, and should get you from 10MPG to 11MPG. Which is the better deal? Assuming you both d…

Web servers are not like that. Micro-optimizations only work for benchmarks and very specific load patterns that almost no people have.

Isn't it precisely like that? The point of the exercise is that even when you are getting really high mpg changes (e.g. 100 to 120), the best gain is improving the the really slow component of the pipeline (e.g. the truck from 10 to 12).

Re: Nxweb – Fast and Lightweight Web Server

#62

Post by rkeene2 »

jerf wrote: post #48
geocar wrote: post #32

Earlier quoted context omitted.

Most websites do not see more than 100 requests per second. In those cases you are correct: parsing and de-parsing is insignificant compared to the amount of energy the computer is using to heat the room. However in order to do a trillion requests per day you need around 30 machines using a custom web server, or 300 machines using Fastcgi: In this situation the cost is an order of magnitude.

Many people observe that as miles-per-gallon gets better and better, it begins to become a deceptive measurement in a way, because going from 10 to 20 mpg is a much, much larger change than going from 30 to 40, or even from 80 to 140. It seems people get a better sense of what's going on to measure gallons per mile. When you start doing that it becomes more clear that going from .0001 gallons per mile to .00001 gallo…

My web server, filed ( http://filed.rkeene.org/ ), is faster than nginx for serving static content by doing two things: 1. Only handling static files 2. Being extremely optimized for serving static content

It's very safe as far as I can tell having run it under AFL with no crashes with ASan on as well as having run it in production on the public Internet.

A few of the optimizations I do in "filed" could also be done in nginx, but most would cost too much.

A separate logging thread that is queued to helps a lot and was one of the main reasons for writing "filed" -- my ability to serve files was being slowed by my ability to write logs indicating that I had served something. The downside is that there may be a large queue of unwritten logs in the event of a kernel panic it other unexpected process termination.

Most requests don't even open the file they are serving because "filed" caches open file descriptors -- once the file has been opened it's kept open until cache entry is needed for a newer file.

There are no runtime allocations after startup except for log entries, leading to very consistent performance under loads.

Re: Nxweb – Fast and Lightweight Web Server

#63

Post by ktRolster »

geocar wrote: post #16
ktRolster wrote: post #6

A lot of ad-tech companies build ad-servers in C, because the latency is so crucial in that context.

Specifically it's garbage collection. e.g. node.js has no problem getting 20k/sec per core, but a stall at the wrong time kills every pipelined HTTP request that follows (until you tear down the connection and restart it).

Worth also mentioning that ad servers tend to have massive RAM requirements (again, for speed). GC in a 30GB JVM can take 10 minutes. To handle it, companies mark the boxes as 'inoperable' when they are in GC mode and remove them from the cluster until thy are ready to return.

All of this is motivation to rewrite everything in C.

Re: Nxweb – Fast and Lightweight Web Server

#65

Post by zzzcpan »

hueving wrote: post #61
zzzcpan wrote: post #55

Earlier quoted context omitted.

Web servers are not like that. Micro-optimizations only work for benchmarks and very specific load patterns that almost no people have.

Isn't it precisely like that? The point of the exercise is that even when you are getting really high mpg changes (e.g. 100 to 120), the best gain is improving the the really slow component of the pipeline (e.g. the truck from 10 to 12).

Well, no. Web server's role is more like a taxi drive home after a 12 hour flight. From that perspective MPGs don't matter at all.

Re: Nxweb – Fast and Lightweight Web Server

#66

Post by vegardx »

geocar wrote: post #57
zzzcpan wrote: post #53

Earlier quoted context omitted.

> 40k/sec http requests (no pipelining) per core is about as fast as it gets unless you move TCP into user space. I think it can be micro-optimized beyond that. With predictions to avoid unnecessary syscalls, with syscalls grouped together to make cpu more efficient for the rest of the time it spends in event loop, and if it's possible to modify kernel a bit - with batching syscalls together to make them very cheap.

At some point, your complexity gets bigger than simply coding a state machine that operates directly on the network buffers themselves: That is to say, I suspect that if micro-optimisations can double our performance, they will be more complicated than just writing a customised ring0 that implements HTTP directly inside the network driver. Here is how I'm looking at it: • 10Gb/sec network port • 4k max requests and r…

What kind of "main memory" are you talking about? Regular, consumer grade memory, will have a bandwidth at least ten times faster than your 10Gb/s network interface. Change the bit to a byte and you're a little closer.

Re: Nxweb – Fast and Lightweight Web Server

#67

Post by brobinson »

eva1984 wrote: post #27
pjmlp wrote: post #17

Earlier quoted context omitted.

Or they don't know any better. I took part in a few projects that replaced high throughput servers handling mobile network traffic from C++ to Java.

Can confirm. I work at two ad-tech related companies, one large one medium, both use Java from the beginning. And I know other company using Python/Golang as well. Didn't know C(not C++) is particular popular until today. Note that ad company are pretty business focused, add or remove features for big clients are pretty common, so development efficiency matters a lot.

We went from C++ to Go, and started to auto-lose any auctions whose bid requests were in progress during a GC cycle. Go 1.1/1.2 had atrociously long GC pauses (sometimes up to 500ms). You are correct that development efficiency was very important, so losing the auctions was justifiable.

Re: Nxweb – Fast and Lightweight Web Server

#68

Post by ktRolster »

pjmlp wrote: post #44
cmrdporcupine wrote: post #41

Earlier quoted context omitted.

Did you have tight and consistent latency requirements? GC pauses are a killer. They can be worked around, but it takes intensive tuning. The RTB bidder I wrote many moons ago at a startup was fast as hell, but had problems in the 95th percentile of requests meeting the latency targets, due to GC. The ad servers at the big ad tech heavy weights are in C++.

Of course, people aren't going to be happy when they packets get dropped or the network monitoring software wasn't able to provide a (soft) real time view of what was happening. Also note I wasn't doing this alone, it was a very big project a mobile operator.

Your experience isn't representative, so there were probably other confounding factors, like the C++ wasn't very well written, or you weren't dealing with petabytes of data or something like that.

Re: Nxweb – Fast and Lightweight Web Server

#69

Post by iampims »

brobinson wrote: post #67
eva1984 wrote: post #27

Earlier quoted context omitted.

Can confirm. I work at two ad-tech related companies, one large one medium, both use Java from the beginning. And I know other company using Python/Golang as well. Didn't know C(not C++) is particular popular until today. Note that ad company are pretty business focused, add or remove features for big clients are pretty common, so development efficiency matters a lot.

We went from C++ to Go, and started to auto-lose any auctions whose bid requests were in progress during a GC cycle. Go 1.1/1.2 had atrociously long GC pauses (sometimes up to 500ms). You are correct that development efficiency was very important, so losing the auctions was justifiable.

Are things different with Go 1.5/1.6 now that GC has start to be optimized?

Re: Nxweb – Fast and Lightweight Web Server

#70

Post by brobinson »

iampims wrote: post #69
brobinson wrote: post #67

Earlier quoted context omitted.

We went from C++ to Go, and started to auto-lose any auctions whose bid requests were in progress during a GC cycle. Go 1.1/1.2 had atrociously long GC pauses (sometimes up to 500ms). You are correct that development efficiency was very important, so losing the auctions was justifiable.

Are things different with Go 1.5/1.6 now that GC has start to be optimized?

Not sure, sorry. I haven't written any high-performance stuff in it since 2014.
Post reply on HN