Live data from Hacker News

Web Framework Benchmarks Round 4

techempower.com

261–270 of 358 posts

Re: Web Framework Benchmarks Round 4

#261
post #111
post #28

Earlier quoted context omitted.

Probably because all those big frameworks have to initialize everything for every request. Parsing, connections, configuration, you name it.

Raw php also has to initialize connections, and it does really well, so I don't think that's it. To me, that points to overhead from initializing the objects as the bottleneck. I would have thought that with APC, this wouldn't be a major issue, though, so I wonder if that's still not it.

APC only caches the opcodes, so the interpreter doesn't have to parse your code. But the framework still has to set up itself for each request individually. Parse configuration, create objects, etc. It adds up quickly.

Re: Web Framework Benchmarks Round 4

#263
post #253

Wow... some of these tests are still pretty severely hobbled. Is there some reason that you use built in json serialization for some frameworks and not others? There is also a lot of heterogeneity in the implementation of the multiple queries test. For instance, even if I only look at... say... java frameworks, you seem to implement the exact same feature in very different ways between platforms. For instance, for se…

Hello bilbo0s. Thanks for taking the time to dig in and provide some feedback. As much as possible, we want each test to be representative of idiomatic production-grade usage of the framework or platform. Furthermore, we have solicited contributions from fans of frameworks and the frameworks' authors. A side objective is that the code double as an example of how best to use the framework or platform. All of this mean…

"..The multiple query test has a client-provided count of queries, so in most Java cases, we create a fixed-size array to hold the results fetched from the database. I wrote the Servlet and Gemini tests, so I can confirm that behavior in those tests..."

I agree, that approach would be best. I just was unsure why you didn't do it in Vert.x.

"...it is our understanding that idiomatic Vert.x usage encourages the use of asynchronous queries..."

Someone can correct me if I am wrong, but my understanding of Vert.x is that any query you send to the event bus is already asynchronous. There is no need for a developer to worry about threads at all when writing a vert.x handler. That handler will only ever be called from a single thread. So using a simple array is fine. Using the JsonArray is even better, because then it matches the Onion test idiomatically speaking. Which, I agree, is what you should be going for.

"...The Onion test was contributed by a reader and admittedly its compliance with the specification we've created is perhaps a bit dubious. We want a JSON serializer to process an in-memory object into JSON..."

Please don't misunderstand, the Onion test does what you want it to do. As well, it does it in the correct idiomatic fashion. That's exactly how I would write the Onion test. I was just wondering why the other tests went out of their way to decode Json and the reencode Json for each result. Onion only ever encodes to Json once, other tests are encoding and decoding multiple times. I only pointed out Vert.x because it was the most egregious. I mean in that case the answer from the persistor is already in Json. It is put in a non Json data structure... and then that data structure is encoded to Json??? Just seemed weird.

----

EDIT: Just verified that there is no need for thread safe code in a Vert.x handler. (Gotta say... that is pretty slick)

On a connected note... man ... these tests are a VERY good way to learn more about these different frameworks!

----

Re: Web Framework Benchmarks Round 4

#264
post #248

Wow... some of these tests are still pretty severely hobbled. Is there some reason that you use built in json serialization for some frameworks and not others? There is also a lot of heterogeneity in the implementation of the multiple queries test. For instance, even if I only look at... say... java frameworks, you seem to implement the exact same feature in very different ways between platforms. For instance, for se…

Contribute and we'll see what it does in round 5 ;)

Do you guys take Git Pulls???

Re: Web Framework Benchmarks Round 4

#265
post #114
post #95

It's very depressing to see Symfony2 at the bottom of these lists. Although there are quite a few performance optimizations that can be done to improve this, there are few excuses for such poor performance by default.

If you have improvements to the test (that would be realistic for a production deployment) then please submit a pull request. We definitely want to show each framework doing the best it can do. A few frameworks have a "stripped" version (just Django and Rails so far) to try to show the best that can be achieved when typical functionality is stripped out. Essentially optimizing for this test, which is interesting even…

[deleted]

Re: Web Framework Benchmarks Round 4

#266
post #232

Earlier quoted context omitted.

Looking into the code, the raw PHP is using PDO with persistent connections, and I'm not sure that's turned on in any of the frameworks. It looks like it's off by default in Redbean at least. No idea if this makes a big difference, I haven't had time to try it out.

Yes, it does look like none of the frameworks are using persistent connections, which would explain the horrible performance. To be fair though, it is still a valid measurement if the frameworks don't enable persistent connections by default.

> it is still a valid measurement if the frameworks don't enable persistent connections by default

I don't agree. Frameworks often prefer the "safe" option over "performance" by default. If you activate persistent connections by raw coding it [1] then you should also set an absolutely obvious database configuration flag like [2] in a framework.

---

[1] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

[2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...

Re: Web Framework Benchmarks Round 4

#267
post #230
post #150

Earlier quoted context omitted.

how easy is to use GO with other frameworks, let's say PHP? Would it be possible to write an application that uses PHP for some tasks, so you can benefit from the speed of GO and the the maturity of PHP?

You'll be better off writing raw PHP without classes than using Go. Go performances are raw Go. As soon as you add any framework to the party , the perfs are falling. see the gorilla test which is a Go framework.

Where are the Gorilla benchmarks? I only see raw Go for all 4 benchmarks.

Re: Web Framework Benchmarks Round 4

#268
post #36

Off all the top performers, Go seems to be the only sane choice to write a web app. Moreover it is at the sweet spot; expressive, flexible, simple, super performant, good community etc. I think it is convincing enough for me to give Go a serious look for our new app.

I'm very curious how C# would compare on the .NET and Mono stacks.

Re: Web Framework Benchmarks Round 4

#269
post #133

Earlier quoted context omitted.

I don't think it seems dumb. It would be a very useful comparison between platforms. Stackoverlfow runs on .Net and Windows servers and they say it's very performant. So why not compare with other frameworks on the same hardware.

Voidlogic is right that we are waiting to get a pull request that will include some .Net frameworks [1]. If you can help, it would be greatly appreciated. We do want to include .Net. We will test on Mono to start. We also want to test on .Net's native Windows platform. But we need to work on the testing platform we've built in order to automate a Windows server in the same way we presently automate a Linux server. [1…

Nancy[1] comes to mind.

[1]: http://nancyfx.org/

Post reply on HN