Live data from Hacker News

Vert.x (JVM async) vs Node.js Http benchmarks results

vertxproject.wordpress.com

61–70 of 122 posts

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#61
post #32

Earlier quoted context omitted.

Not really. It shows that this benchmark is crap (likely benchmarking disk io versus disk io + some caching). Read Isaac's comment for more detail. He sums it up pretty well. No profiling info, using a custom test, no analysis besides some pretty graphs. I have a hard time believing the JVM is really 10x faster than v8 for such a simple server.

Why would it surprise you that a statically typed language like Java on the highly optimized JVM is faster than Javascript? Any benchmark you care to look at should clearly shows JVM is much faster: http://shootout.alioth.debian.org/u64/benchmark.php?test=all...

It would surprise me because there just isn't that much time being spent in JavaScript on this test. Do the math.

If a program spends 2% of its time in V8, 10% of its time in the network stack, and 80-something% of its time reading a file from disk, then how can you even consider that you can make it 10 times more effective by optimizing away the 2%? Even if the JVM was 100 times faster than V8, then you would expect to get faster by a factor of just slightly less than 2%.

Ie, if you were seeing 1000 requests per second before, and you're spending 2% of your time parsing and running actual JavaScript, and you make the VM go to literally zero latency (which is impossible, but the asymptotic goal), then you'd expect each request to take 2% less time. So, they'd go from an avg of 1ms to 0.98ms. Congratulations. You've increased your 1000qps server to 1020.4 qps.

On the other hand, if you take the 80% of time spent reading the file over and over again, and optimize that down to zero (again, impossible, but the asymptote that we approach as it is reduced), then you would expect every request to take 80% less time. So, your 1ms response becomes a 0.2ms response, and your 1000 qps server is now a 50000 qps server.

So, no, if you respond to 10x as many requests, it's almost certainly either a bug in the benchmark, or some apples-to-oranges comparison of the work it's doing. I called out one obvious issue like this, that the author is using a deprecated API that's known to be slow. But even still, it's not THAT slow.

You can't summon speed out of the ether. All you can do is reduce latency, and you can only reduce the latency that exists. Even if your VM is faster, that only matters if your VM is actually a considerable portion of the work being done. The JVM and V8 are both fast enough to be almost negligible.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#62
post #11

Perhaps a dumb question: Are the headers the same? With so small files, the header layout is suddenly very important for your measurement. I've seen my share of web servers which are very different in their compliance.

It's not a dumb question at all.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#63

Earlier quoted context omitted.

Why would it surprise you that a statically typed language like Java on the highly optimized JVM is faster than Javascript? Any benchmark you care to look at should clearly shows JVM is much faster: http://shootout.alioth.debian.org/u64/benchmark.php?test=all...

It would surprise me because there just isn't that much time being spent in JavaScript on this test. Do the math. If a program spends 2% of its time in V8, 10% of its time in the network stack, and 80-something% of its time reading a file from disk, then how can you even consider that you can make it 10 times more effective by optimizing away the 2%? Even if the JVM was 100 times faster than V8, then you would expect…

Is it possible that his JVM is setup to execute with different ulimit settings than Node? The Node numbers look too suspiciously close to multiples of 1024 (e.g., default file descriptor limit).

EDIT: whoops, saw 4096 for the stream but it's 4896.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#64
post #49

Earlier quoted context omitted.

Seeing these numbers (although as the author states, the data needs to be taken with a grain of salt since he didn't try to set up a more "proper" test environment) really makes me want to take a serious look at vert.x, but what you mention here is my main concern. Node is a joy to use because of the NPM, and all the available libraries. I'd hate to lose that. That said, so long as it has a solid websocket api, and I…

> Node is a joy to use because of the NPM, and all the available libraries. I'd hate to lose that. Maven's not bad at all (from a consumption perspective--from a dev perspective, it's a bit of a pain in the ass), and it looks like pretty much any Java library that doesn't do anything too insane should work just fine. (Disclaimer: never used vert.x myself, but I did a quick scan of the code.)

Cool, thanks for the tip

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#65

Don't serve static files with node. Use an nginx reverse proxy for your html/js/css assets.

Came here to say this. While microbenchmarks are fun for all, in the real world you would have a completely different reason for choosing node.js that has nothing to do with this kind of performance. So just use the best tool for what is being benchmarked here: nginx.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#66
post #22
post #2

Interesting results. While it's not surprising to me that the pure java implementation was far and ahead better since: 1.) Java's much faster than Javascript, and 2.) Netty (which Vert.x is based on), has been doing high performance async IO for years before node even existed in extremely high performance environments. What is surprising however, is that javascript on the JVM w/ vert.x is faster than V8 with node. In…

Very good point. V8 does heavy code optimization and Node.js http sever code should be well optimized as well. And maybe the load balancer for the multicore Node test isn't optimized enough. Thus, these results feel a little shady. Anyway, Vert.x was the trigger that I am finally downloading the JVM (JDK) to try Vert.x (and maybe later Clojure). But there's still one major drawback—the non-existant ecosystem. I know…

It's funny, the path software technology takes. A Java clone of a fairly basic web engine (which is a fairly simple application type to begin with) convinces people to try the JVM which is the most performant managed runtime around and definitely one of the most advanced pieces of software ever created. I'm sometimes amazed that and people coming from easy-to-use web languages have never been exposed to it, and if they have they still have their doubts.

Dude, I've been developing real-time military applications and we've moved from C/C++ to Java and got performance gains (I'm not saying that C can't beat Java doing specific computations - of course it can - but when building a large application with millions of lines of code and a large team of developers, Java would be a safer bet than C++ if speed is your concern). In other words, no other environment can beat the JVM.

And, yeah, you should try Clojure. It's a cathartic experience.

EDIT: The only doubts I have about vert.x is that it can consistently beat "old" JVM servlet containers under heavy, real-world loads. That remains to be seen.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#67

Earlier quoted context omitted.

Why would it surprise you that a statically typed language like Java on the highly optimized JVM is faster than Javascript? Any benchmark you care to look at should clearly shows JVM is much faster: http://shootout.alioth.debian.org/u64/benchmark.php?test=all...

It would surprise me because there just isn't that much time being spent in JavaScript on this test. Do the math. If a program spends 2% of its time in V8, 10% of its time in the network stack, and 80-something% of its time reading a file from disk, then how can you even consider that you can make it 10 times more effective by optimizing away the 2%? Even if the JVM was 100 times faster than V8, then you would expect…

The flaw in your argument is that the server is spending 80% of its time reading a file from disk.

It's more than likely that it spends close to 0% of its time in disk access since its serving the same file, which will be cached by the OS in memory.

About the deprecated API. Earlier on I updated the results so they don't use that API, and I also added results for using streams. The results are slightly better but not by very much.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#68
post #59
post #56

Earlier quoted context omitted.

Could you clarify that? Are you saying that if the response is sent within the same second that the request came in that it contributes to the metric? Or would a response that is sent 30 seconds after the request came in contribute to the metric too?

It doesn't matter whether a request straddles a second or not in a throughput measuring benchmark when you saturate the system. A client would only count a request when its request call has returned. Runs it for N minutes, count up how many requests have completed, then divide the total with the time and you got req/sec. Besides the benchmark has run for a minute. I doubt each request lasts 30 seconds.

+1.

The system is in steady state, i.e. queues of requests/responses aren't growing. Therefore it doesn't actually matter if you count the requests or the responses.

Re: Vert.x (JVM async) vs Node.js Http benchmarks results

#69

Earlier quoted context omitted.

Yes indeed. You can mix and match Java, JavaScript, Ruby and Groovy in the same app. We hope to support more languages going ahead (e.g. Java, Scala, ...)

+1 for Scala Might be nice to also see a FAQ about the differences between vert.x and Play 2

Hopefully Scala won't be too long :)
Post reply on HN