Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Optimizing Nginx, Node.js and networking for heavy workloads

engineering.gosquared.com

Post reply on HN
Print view View original

11–20 of 26 posts

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#11

Post by gnw »

aroman wrote: post #10

Good read, but despite being a major proponent of Node.js and many of the ideals it seeks to embrace, I'm not sure I'm comfortable with calling Apache "archaic". It's not like IE6, which has objectively no redeeming value as a modern platform target -- it did back in the day for sure, and is still relevant in some spheres, but overall I don't think anyone (even Microsoft) would argue that IE6 is "archaic". But to cal…

You're perfectly correct, and Apache has and continues to serve as one of the most successful and widely deployed web servers to date. That said, in the context of more conveniently architecting for high volumes of traffic, Apache was conceived in a time of fundamentally different problems, and in that respect it can be viewed as a more antiquated option when scoping out the landscape of appropriate web server software.

I did not intend any pejorative connotations by calling it "archaic". I just wanted to emphasise that it has been eclipsed by newer software following different design paradigms better suited to this kind of problem.

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#13

Post by jiggy2011 »

aroman wrote: post #10

Good read, but despite being a major proponent of Node.js and many of the ideals it seeks to embrace, I'm not sure I'm comfortable with calling Apache "archaic". It's not like IE6, which has objectively no redeeming value as a modern platform target -- it did back in the day for sure, and is still relevant in some spheres, but overall I don't think anyone (even Microsoft) would argue that IE6 is "archaic". But to cal…

Precisely , I've used Apache for yeeeeeears and see exactly 0 ROI in switching to something else.

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#14

Post by benesch »

zerop wrote: post #12

should nginx be only used for serving static files. Does it have any advantage when used to serve plain data API. I want to expose REST api (django + uwsgi) over web, but not sure if should use nginx for it.

There's probably nothing inherently wrong or slow with running Django through nginx.

That said, one of the most common deployment strategies is gunicorn. It's better documented [1], and it's always good to separate your web app server from your static file server/CDN.

[1] https://docs.djangoproject.com/en/dev/howto/deployment/wsgi/...

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#15

Post by jiggy2011 »

zerop wrote: post #12

should nginx be only used for serving static files. Does it have any advantage when used to serve plain data API. I want to expose REST api (django + uwsgi) over web, but not sure if should use nginx for it.

I think this basically boils down to: "are you likely to have many users connected concurrently at any one time"

If your API basically involves a client connecting, quickly getting a small JSON/XML response and then disconnecting again you are probably absolutely fine with Apache unless you have truly enormous numbers of users.

OTOH if the socket is likely to be held open for a while, because maybe the API responses can take some time to be returned or the client is likely to hold the connection open in order to get a stream of data over time then you may get more mileage out of nginx.

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#17

Post by silenteh »

pygy_ wrote: post #7
silenteh wrote: post #4

enabling net.ipv4.tcp_tw_reuse net.ipv4.tcp_tw_recycle can create unexpected problems with NAT, so use it with caution.

Could you provide some details (or a link)?

When tcp_tw_reuse is enabled the kernel can decide to use the sockets in TIME_WAIT, before they expire or they are closed by the clients.

This is a problem though, because the connection could still be used by the client and therefore there could be some collisions regarding the TCP sequence numbers, specially on high traffic servers. The kernel can try to avoid this collision with a technique called PAWS (protection against wrapped sequence numbers: rfc1323). Unfortunately PAWS works only with tcp_timestamps enabled on both sides (client and server). tcp_timestamps has also an overhead and therefore it is normally disabled on servers with a high traffic, leading to potential problems.

About tcp_tw_recycle, when it is enabled, it forces the verification of this tcp_timestamp. So in case of NAT, multiple clients will send different tcp timestamp to the server, to the same mapped connection which points to the TIME_WAIT socket, and because the tcp timestamp are different then the packets will be dropped by the kernel. This is the reason why it is not a good thing to enable tcp_tw_recycle when you use a load balancer or in case of NAT.

A good practice is to enable tcp_tw_reuse (instead of tcp_tw_recycle), to make sure tcp_timestamp is enabled and to decrease the size of the tcp timestamp with tcp_timewait_len.

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#18

Post by lucian1900 »

aroman wrote: post #10

Good read, but despite being a major proponent of Node.js and many of the ideals it seeks to embrace, I'm not sure I'm comfortable with calling Apache "archaic". It's not like IE6, which has objectively no redeeming value as a modern platform target -- it did back in the day for sure, and is still relevant in some spheres, but overall I don't think anyone (even Microsoft) would argue that IE6 is "archaic". But to cal…

Also, now apache has an event worker and can even use two types of workers at the same time.

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#19

Post by cpleppert »

silenteh wrote: post #17
pygy_ wrote: post #7

Earlier quoted context omitted.

Could you provide some details (or a link)?

When tcp_tw_reuse is enabled the kernel can decide to use the sockets in TIME_WAIT, before they expire or they are closed by the clients. This is a problem though, because the connection could still be used by the client and therefore there could be some collisions regarding the TCP sequence numbers, specially on high traffic servers. The kernel can try to avoid this collision with a technique called PAWS (protection…

>>A good practice is to enable tcp_tw_reuse (instead of tcp_tw_recycle), to make sure tcp_timestamp is enabled and to decrease the size of the tcp timestamp with tcp_timewait_len.

Couple questions. what is tcp_timestamp? i assumes you are not referring to tcp_timestamps? What effect does tcp_timewait_len have on timestamps at all? Isn't it just the amount of time the connection closer holds on to TCBs?

Re: Optimizing Nginx, Node.js and networking for heavy workloads

#20

Post by cpleppert »

gnw wrote: post #9
silenteh wrote: post #4

enabling net.ipv4.tcp_tw_reuse net.ipv4.tcp_tw_recycle can create unexpected problems with NAT, so use it with caution.

I only suggested changing tcp_tw_reuse but you are right, especially tcp_tw_recycle can have adverse effects. According to this reference: http://www.speedguide.net/articles/linux-tweaking-121 "[tcp_tw_reuse] is generally a safer alternative to tcp_tw_recycle"

I believe the section about TCP_FIN_TIMEOUT is wrong. tcp_fin_timeout has nothing to do with the time wait state at all. TCP_TIMEWAIT_LEN is the value that holds onto the TCB
Post reply on HN