Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Web Framework Benchmarks Round 4

techempower.com

Post reply on HN
Print view View original

211–220 of 358 posts

Re: Web Framework Benchmarks Round 4

#211

Post by rdtsc »

just2n wrote: post #169

Before looking at the benchmark results, I took a glance at the Node source and I expected it to perform worse than it did previously. It does almost universally. Not only haven't the glaring perf issues remained since round 1, it's added more. In the real world, when you look at a metric that says your req/s is a bottleneck, which is what this benchmark is loosely simulating, you'd fix it. You wouldn't just say "nop…

I am not sure if you had this in mind or not ( and I already wrote this in a comment above, so sorry for repeating ) but I was wondering about concurrency. That is primarily my concern as web frameworks show their mettle so to speak when a large swarm of parallel requests hammer it. What do they do then? Maybe sequentially one request at a time they are very fast but start barfing out socket error when concurrency increases only slightly. That is a worse case in general than something that perhaps is slower in sequential benchmark but stays up in the face of a concurrent onslaught of client requests.

Otherwise I can see how someone would assume a simplified and misleading heuristic "If I can process 1000 requests in 1 second. That means the server can handle 1000 requests/seconds. So if 1000 requests come in at once, they will all be processed in 1 second". Two thing can happen, it could processes it slower than one seconds, it could error out and die, or it could actually process it fast if it can scale across CPUs. That is where the gold is if you ask me... Anyway just my 2 cents.

Re: Web Framework Benchmarks Round 4

#212

Post by snaky »

mitchi wrote: post #18

http://openresty.org/ Looks interesting but at a quick glance it looks like you are programming in configuration files? I'm not sure I like the idea.

Yes. "Nginx programming" is not a joke for a last couple of years in HighLoad world. Before ngx_lua (OpenResty is nginx plus ngx_lua module basically) you had to do it in C.

Re: Web Framework Benchmarks Round 4

#213

Post by qw »

tlear wrote: post #59
exabrial wrote: post #32

Earlier quoted context omitted.

Well no matter what you use, you should always benchmark for that exact reason. Making things go fast takes a lot of performance engineering, and the JVM has millions of dollars worth of tuning put into it. BTW, if you like Node.js, you should probably look at Vert.x. I haven't used it, but it's a similar concept, it runs on the JVM, and it seems to spank Node.js.

When you benchmark Vert.x on a multicore system vs 1 node process. Now add multiple node processes to occupy the cores and store changes completely

The benchmark is using multiple Node processes.

Re: Web Framework Benchmarks Round 4

#214

Post by cunac »

raylu wrote: post #192
peeters wrote: post #75

Earlier quoted context omitted.

Java gets that wrap from originally being slow to execute, and also having a huge up-front cost to spin up a VM. The first isn't true any more: the Java VM competes with native code on most benchmarks, and due to its ability to perform runtime optimizations, can occasionally outperform native code. The second doesn't matter at all for web servers. The cost of starting up the web server is tertiary to uptime and perfo…

I agree with everything you've said here, but I'd like to add something about startup time. If it takes you 5s to start up your server, that's a lot of time you've added to each development iteration. Make a change, restart the server, wait 5s, see if it works/check debug output.

typically you don't stop/start server during development cycle, hot deployment works 99% of cases

Re: Web Framework Benchmarks Round 4

#215

Post by EGreg »

bhauer wrote: post #164
EGreg wrote: post #40

Earlier quoted context omitted.

In really scalable sites, you need sharding. Unless your database itself is doing the scaling (such as with Riak), you're going to sometimes hit multiple shards. With PHP and other languages that can't do async, you're going to have to query the DB sequentially, increasing latency proportionally to the number of shards you have to hit. With Node.js and other asynchronous apps, you don't. Disclaimer: mysqli does have…

Some of the fastest implementations you see in these tests are not asynchronous. With Servlet for example, a worker thread is chosen from Resin's thread pool and used to handle a request. The Servlet then executes 20 queries sequentially and returns the resulting list data structure. This is Servlet 3.0 but not using Servlet 3.0 async. Async isn't making the top performers fast. Being fast is making them fast.

I agree, but what about the special case of hitting multiple shards and aggregating the results? Shouldn't the non-blocking win over the blocking?

Re: Web Framework Benchmarks Round 4

#216

Post by oberhamsi »

hilem wrote: post #144
krg wrote: post #8

Earlier quoted context omitted.

Ruby 2.0.0-p0 http://www.techempower.com/benchmarks/#section=environment

though they're also using Rails 3.2.11, which not only isn't the latest 3.2.* release but doesn't take advantage of Ruby 2.0... it would be more interesting to see the latest Rails 3.2.* on Ruby 1.9.* along with the latest Rails 4 release candidate on Ruby 2.0.0-p0

I think he is too polite to say: "patch please"

Re: Web Framework Benchmarks Round 4

#217

Post by egeozcan »

notJim wrote: post #111
cmircea wrote: post #28

Earlier quoted context omitted.

Probably because all those big frameworks have to initialize everything for every request. Parsing, connections, configuration, you name it.

Raw php also has to initialize connections, and it does really well, so I don't think that's it. To me, that points to overhead from initializing the objects as the bottleneck. I would have thought that with APC, this wouldn't be a major issue, though, so I wonder if that's still not it.

I always hear this myth that php gets very slow when you write proper OO code. I neither write nor maintain any php code so I wouldn't know.

Re: Web Framework Benchmarks Round 4

#219

Post by candybar »

dragonwriter wrote: post #206
candybar wrote: post #188

Earlier quoted context omitted.

I keep hearing about this, but do people really rewrite performance-critical parts of their web apps in C? Even if it happens to be part of some third-party library? And maintain a fork? What if that performance-critical part is dependent on other parts in a non-trivial way? It seems that an unanticipated replacement of some core functionality with a C library may involve a major rewrite and most Ruby teams may not h…

> I keep hearing about this, but do people really rewrite performance-critical parts of their web apps in C? Certainly they do it for Ruby apps in general. I don't think its all that common for it to be a high-value proposition for web apps. > Even if it happens to be part of some third-party library? And maintain a fork? If its an open-source third-party library that tends to get used in a way that is performance-cr…

I guess my point is that it seems disingenuous to point out "you can write that bit in C" as a way to mitigate performance problems with Ruby, when in practice it's so costly compared to available alternatives (throw more hardware, write manually optimized Ruby, switch to a faster language/runtime) that almost no one does it. How much of Rails is written in C? It's like proposing compiler extensions/patches as a way of dealing with performance problems. And if you have a complex application that utilizes many of Ruby's idioms to deal with the complexity, it's extremely unlikely that you can simply replace parts of it with C libraries without reorganizing in such a way to increase complexity.
Post reply on HN