Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A HTTP Proxy Server in 20 Lines of node.js Code

catonmat.net

Post reply on HN
Print view View original

11–20 of 26 posts

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#11

Post by euroclydon »

gcampbell wrote: post #10
euroclydon wrote: post #7

Since this is near the top of the HN page, I'd like to ask a question about node.js: What is it suited for? I read the about section of the node.js homepage, and it says that "Node's goal is to provide an easy way to build scalable network programs." Okay, that sounds cool, and I like Javascript, so I want to see if I can fit it into any projects I'm working on, but then I go on to read: "Almost no function in Node d…

The important word in that sentence is "directly". There are plenty of ways to do I/O in node, but they generally involve event-based callbacks so that they don't block the thread.

Are there any special considerations for keeping the TCP connection open while the thread is waiting for the callback? I'm assuming no, since node.js doesn't seem to exist in order to facilitate long running requests.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#12

Post by calcnerd256 »

euroclydon wrote: post #11
gcampbell wrote: post #10

Earlier quoted context omitted.

The important word in that sentence is "directly". There are plenty of ways to do I/O in node, but they generally involve event-based callbacks so that they don't block the thread.

Are there any special considerations for keeping the TCP connection open while the thread is waiting for the callback? I'm assuming no, since node.js doesn't seem to exist in order to facilitate long running requests.

I'm not sure, but I'd guess closing over it in the callback would work, as long as the TCP timeout is longer than the database response (even then, you could probably touch the stream every so often while you wait, perhaps via nextTick or something).

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#13

Post by lftl »

euroclydon wrote: post #11
gcampbell wrote: post #10

Earlier quoted context omitted.

The important word in that sentence is "directly". There are plenty of ways to do I/O in node, but they generally involve event-based callbacks so that they don't block the thread.

Are there any special considerations for keeping the TCP connection open while the thread is waiting for the callback? I'm assuming no, since node.js doesn't seem to exist in order to facilitate long running requests.

I don't really have a direct answer to your question, but the somewhat official demo for node.js is a chat server that uses http long polling to push messages out, so node can definitely handle that.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#14

Post by simonw »

euroclydon wrote: post #11
gcampbell wrote: post #10

Earlier quoted context omitted.

The important word in that sentence is "directly". There are plenty of ways to do I/O in node, but they generally involve event-based callbacks so that they don't block the thread.

Are there any special considerations for keeping the TCP connection open while the thread is waiting for the callback? I'm assuming no, since node.js doesn't seem to exist in order to facilitate long running requests.

Node is designed specifically TO facilitate long-running requests. There is only one thread (it's a single-threaded, evented system). Everything is done with callbacks, so if you have a long running I/O operation you'll just add a callback which completes the current HTTP response when the long-running I/O has finished. Node is quite happy to keep thousands of such callbacks around at once for massive concurrency.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#15

Post by Sidnicious »

euroclydon wrote: post #11
gcampbell wrote: post #10

Earlier quoted context omitted.

The important word in that sentence is "directly". There are plenty of ways to do I/O in node, but they generally involve event-based callbacks so that they don't block the thread.

Are there any special considerations for keeping the TCP connection open while the thread is waiting for the callback? I'm assuming no, since node.js doesn't seem to exist in order to facilitate long running requests.

Since Node scripts are persistent (i.e. aren't created and destroyed for each request), it doesn't care how long or how many connections are open. It's ideal for long-running requests.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#16

Post by jimbokun »

silvestrov wrote: post #4

This is not a server in 20 lines of code, but server configuration in 20 lines (there is no parsing of the http protocols, but only calls into a lib that implements an http proxy). The examples could be implemented in fewer lines of Apache configuration. I would have been nice with some examples that shows what node.js can do, which can't be done (easily) with Apache.

At the top of the page it says "good coders code, great reuse."

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#17

Post by TimothyFitz »

silvestrov wrote: post #4

This is not a server in 20 lines of code, but server configuration in 20 lines (there is no parsing of the http protocols, but only calls into a lib that implements an http proxy). The examples could be implemented in fewer lines of Apache configuration. I would have been nice with some examples that shows what node.js can do, which can't be done (easily) with Apache.

Not quite accurate. The code he wrote does the "proxy" logic while libraries underneath it are doing the work of parsing HTTP. He's definitely not doing "calls into a lib that implements an http proxy."

More to the point, what he has is a proxy in 20 lines of code which will let him make, say, a custom load balancing http proxy in 40 lines of code.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#18

Post by fizx »

I benchmarked this a few weeks ago at around ~300req/s on my recent macbook, which was a little disappointing.

For kicks, here's my CoffeeScript/Node.js code I used. I did the benchmark with ab entirely on localhost. Normally that's not ideal, but it was enough to tell that this isn't as fast as I'd like.

http://gist.github.com/382372

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#19

Post by felixge »

fizx wrote: post #18

I benchmarked this a few weeks ago at around ~300req/s on my recent macbook, which was a little disappointing. For kicks, here's my CoffeeScript/Node.js code I used. I did the benchmark with ab entirely on localhost. Normally that's not ideal, but it was enough to tell that this isn't as fast as I'd like. http://gist.github.com/382372

You need to create a new client for each incoming request. In your code, you are actually queuing up all responses.

So your result is probably an order of magnitude off.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#20

Post by fizx »

felixge wrote: post #19
fizx wrote: post #18

I benchmarked this a few weeks ago at around ~300req/s on my recent macbook, which was a little disappointing. For kicks, here's my CoffeeScript/Node.js code I used. I did the benchmark with ab entirely on localhost. Normally that's not ideal, but it was enough to tell that this isn't as fast as I'd like. http://gist.github.com/382372

You need to create a new client for each incoming request. In your code, you are actually queuing up all responses. So your result is probably an order of magnitude off.

Good to know, thanks!

Edit: I feel like I tried that, and a couple other variations, and this was the fastest. But it's worth revisiting, and I'll definitely try it again.

Post reply on HN