Live data from Hacker News

A HTTP Proxy Server in 20 Lines of node.js Code

catonmat.net

21–26 of 26 posts

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#21
post #3

If anyone's looking for a project, a reusable HTTP proxying library for Node.js with easy hooks for adding pre- and post- checks and transforms would be incredibly useful. Being able to implement API rate limiting and authentication as a Node.js reverse proxy that wraps around an existing web application would be even easier with a well-tested, fully featured proxy library.

Does anyone have any idea how it would compare performance-wise to what's out there right now? vs Nginx for example.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#22
post #3

If anyone's looking for a project, a reusable HTTP proxying library for Node.js with easy hooks for adding pre- and post- checks and transforms would be incredibly useful. Being able to implement API rate limiting and authentication as a Node.js reverse proxy that wraps around an existing web application would be even easier with a well-tested, fully featured proxy library.

Does anyone have any idea how it would compare performance-wise to what's out there right now? vs Nginx for example.

Ry has some Nginx / Node comparison in his JSConf slides http://nodejs.org/jsconf2010.pdf

Summary: Node isn't quite as fast as NginX but it's close, and it's programable, and he's still tuning it.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#23
post #3

If anyone's looking for a project, a reusable HTTP proxying library for Node.js with easy hooks for adding pre- and post- checks and transforms would be incredibly useful. Being able to implement API rate limiting and authentication as a Node.js reverse proxy that wraps around an existing web application would be even easier with a well-tested, fully featured proxy library.

Does anyone have any idea how it would compare performance-wise to what's out there right now? vs Nginx for example.

EDIT: After looking at this http://nodejs.org/jsconf2010.pdf from sh1mmer's post, it looks like I need to be using a Buffer object to serve data over 64kb due to v8's garbage collector making a performance mess when using strings.

EDIT2: After looking at the code again, file reads come in as strings, so I don't actually see that Buffers would help, at least until the node file API starts passing around buffers instead of strings.

I can't speak to performance wrt to proxying, but I benchmarked its performance serving files recently[1].

Now, I didn't benchmark nginx, but as you can see, its pretty slow at that task, averaging ~10 MiB/s on fairly decent hardware, across a single gigabit switch.

One problem node has, is that it sucks dealing with strings containing non UTF-8 data:

From the node docs:

Node supports 3 string encodings. UTF-8 ('utf8'), ASCII ('ascii'), and Binary ('binary'). 'ascii' and 'binary' only look at the first 8 bits of the 16bit JavaScript string characters.

1. http://blog.andrewvc.com/nodejs-can-really-scale

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#24
post #9

Since this is near the top of the HN page, I'd like to ask a question about node.js: What is it suited for? I read the about section of the node.js homepage, and it says that "Node's goal is to provide an easy way to build scalable network programs." Okay, that sounds cool, and I like Javascript, so I want to see if I can fit it into any projects I'm working on, but then I go on to read: "Almost no function in Node d…

Database adapters for Node are beginning to show up - there's an async PostgreSQL one and I think there's one using libdrizzle (which should work for MySQL as well). Since Node speaks HTTP fluently it's great for talking to things like CouchDB, and there's also DBSlayer http://code.nytimes.com/projects/dbslayer which provides an HTTP API for running MySQL SQL queries. Node + Redis is an awesome combination. Don't for…

Thanks for the shout-out in the slides about the Redis client for Node.js.

http://github.com/fictorial/redis-node-client

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#25
post #20
post #19

Earlier quoted context omitted.

You need to create a new client for each incoming request. In your code, you are actually queuing up all responses. So your result is probably an order of magnitude off.

Good to know, thanks! Edit: I feel like I tried that, and a couple other variations, and this was the fastest. But it's worth revisiting, and I'll definitely try it again.

For posterity, after re-benchmarking, 1200-1300 req/s.

Re: A HTTP Proxy Server in 20 Lines of node.js Code

#26

Since this is near the top of the HN page, I'd like to ask a question about node.js: What is it suited for? I read the about section of the node.js homepage, and it says that "Node's goal is to provide an easy way to build scalable network programs." Okay, that sounds cool, and I like Javascript, so I want to see if I can fit it into any projects I'm working on, but then I go on to read: "Almost no function in Node d…

I suppose I should chime in and say that I am not exactly sure what Node.js's about page is talking about when it says 'I/O'. Could someone clue me in as to what exactly this means?
Post reply on HN