Web Framework Benchmarks • Hacker News Board

Re: Web Framework Benchmarks

#71

Post by chipotle_coyote » Thu, Jun 07, 2018, 2:58 PM UTC

mkirklions wrote: post #58

smoe wrote: post #36

Earlier quoted context omitted.

I agree with the overall sentiment. Plus almost all cases I run into performance issues with e.g. Python in general web development, that main problems were how the code was written, not what it was written in and being bottle-necked by the database. That said I disagree a bit with saying it is a a solved problem unless you are Google-scale. In developing countries if you are in a small company without big investors,…

Im somewhat traditional with my web, javascript and php. But are people really doing python and web development? I love python for my web crawlers, photoshop automation, and table formatting automation- But I never considered it a web contender. Any major sites running python? Whats the popular way people are doing it?

In addition to some other sites I've seen mentioned in respnose Reddit seems pretty major, and AFAIK it's still largely in Python. (They wrote their own Python framework, although they've been moving to Pyramid.) Yelp is also a Python shop, as is Disqus (which is built on Django).

Re: Web Framework Benchmarks

#72

Post by l1ambda » Thu, Jun 07, 2018, 2:59 PM UTC

onion2k wrote: post #15

These are fun benchmarks, but every single framework in the top 250 or so is more than fast enough for most apps. Even the slowest in the list manages 366/requests per second, which is probably enough to prove an idea before optimizing for speed. We're approaching the point where speed is essentially a solved problem unless you're at Google-scale.

It's not so much about immediate performance, but about headroom.

The blog post https://www.techempower.com/blog/ puts it better than I:

> I argue that if you raise the framework's performance ceiling, application developers get the headroom—which is a type of luxury—to develop their application more freely (rapidly, brute-force, carefully, carelessly, or somewhere in between). In large part, they can defer the mental burden of worrying about performance, and in some cases can defer that concern forever. Developers on slower platforms often have so thoroughly internalized the limitations of their platform that they don't even recognize the resulting pathologies: Slow platforms yield premature architectural complexity as the weapons of “high-scale” such as message queues, caches, job queues, worker clusters, and beyond are introduced at load levels that simply should not warrant the complexity.

Re: Web Framework Benchmarks

#73

Post by chipotle_coyote » Thu, Jun 07, 2018, 3:00 PM UTC

ddorian43 wrote: post #65

mkirklions wrote: post #61

Earlier quoted context omitted.

I'll tell you what. When I'm a fortune 10,000 company, we can afford to rewrite the web app.

But do they rewrite ? Twitter did. Facebook didn't.

Facebook wrote a new PHP VM and forked the language, so while it may not technically be rewriting...

Re: Web Framework Benchmarks

#74

Post by drunner » Thu, Jun 07, 2018, 3:01 PM UTC

bhauer wrote: post #29

tomaha wrote: post #27

Earlier quoted context omitted.

Do you have any plans to add memory usage to the benchmarks? It adds an extra dimension and sometimes even shows if there's a problem with a specific implementation (e.g. in the benchmark game it's often a hint for a program that can be optimized if the ratio compared to other languages is off).

Yes! In fact, I just responded to a similar question over at the Rust Subreddit thread about the same topic. We do in fact capture dstat data while executing the tests but as of today do not render these in any way. You can find raw CSV output from dstat at our logs server. For example at [1] are stats for Grizzly while measuring the "json" test type. I will create an issue at the project's GitHub repo to begin a con…

If I could make a recommendation that would humor me personally, you could publicize the stats required to run the instance on an B1S or Fv2 Azure instance.

I think it would be really interesting to see which frameworks are so costly upfront that they would impede a developer from prototyping with them on the cheap.

Re: Web Framework Benchmarks

#75

Post by jdietrich » Thu, Jun 07, 2018, 3:01 PM UTC

mkirklions wrote: post #58

smoe wrote: post #36

Earlier quoted context omitted.

I agree with the overall sentiment. Plus almost all cases I run into performance issues with e.g. Python in general web development, that main problems were how the code was written, not what it was written in and being bottle-necked by the database. That said I disagree a bit with saying it is a a solved problem unless you are Google-scale. In developing countries if you are in a small company without big investors,…

Im somewhat traditional with my web, javascript and php. But are people really doing python and web development? I love python for my web crawlers, photoshop automation, and table formatting automation- But I never considered it a web contender. Any major sites running python? Whats the popular way people are doing it?

In the dim and distant days of 2010ish, Python was quite widely used for back-end development, typically using Django or Flask. Instagram, Disqus and Pinterest were all originally built on Django, IIRC.

Re: Web Framework Benchmarks

#76

Post by beal » Thu, Jun 07, 2018, 3:04 PM UTC

There is a lot of effort being spent measuring what is very rarely a performance bottleneck in web applications.

Re: Web Framework Benchmarks

#77

Post by laumars » Thu, Jun 07, 2018, 3:07 PM UTC

pjmlp wrote: post #60

laumars wrote: post #45

Earlier quoted context omitted.

Honest question: What sort of embedded hardware deals with millions of requests a second? I wouldn't have thought that kind of utility was the same domain as "embedded" (in the stricter sense of the term). Granted you could expand the term to cover hardware load balancers and the sort, but those wouldn't really be running websites so much as routing and caching them. Aside from that, you have management interfaces fo…

It is not millions of requests a second, rather what you can put into something with 512 KB or less while providing a minimal Web management interface. For example, https://www.hcc-embedded.com/embedded-systems-software-produ...

Ahh yes. I do have some familiarity with HCC - albeit limited to a hobby project rather than anything professional. It's interesting stuff but I'd argue that's a completely different target industry to the ones that would use any of the web frameworks in the discussed referenced benchmark. So not really on point to the question raised by the GP with regards to C vs the more rapid development frameworks like node.

Not that I'm trying to dismiss your point either because if I was given a choice between node and C, I might personally choose C just because of my own personal prejudices against Javascript. :D

Re: Web Framework Benchmarks

#78

Post by blunte » Thu, Jun 07, 2018, 3:12 PM UTC

I'm not familiar with this site and benchmarks, but from reading comments it seems to be respected.

Given that, I'm confused why this is called "web framework" benchmarks. It looks to me like it is comparing some actual frameworks (which rank very poorly) against minimalist, task-focused http servers (which rank highly).

Re: Web Framework Benchmarks

#79

Post by pier25 » Thu, Jun 07, 2018, 3:12 PM UTC

onion2k wrote: post #15

These are fun benchmarks, but every single framework in the top 250 or so is more than fast enough for most apps. Even the slowest in the list manages 366/requests per second, which is probably enough to prove an idea before optimizing for speed. We're approaching the point where speed is essentially a solved problem unless you're at Google-scale.

I agree with your general point, but there are many more companies and products that do require high performance beyond Google.

Re: Web Framework Benchmarks

#80

Post by bhauer » Thu, Jun 07, 2018, 3:13 PM UTC

drunner wrote: post #74

bhauer wrote: post #29

Earlier quoted context omitted.

Yes! In fact, I just responded to a similar question over at the Rust Subreddit thread about the same topic. We do in fact capture dstat data while executing the tests but as of today do not render these in any way. You can find raw CSV output from dstat at our logs server. For example at [1] are stats for Grizzly while measuring the "json" test type. I will create an issue at the project's GitHub repo to begin a con…

If I could make a recommendation that would humor me personally, you could publicize the stats required to run the instance on an B1S or Fv2 Azure instance. I think it would be really interesting to see which frameworks are so costly upfront that they would impede a developer from prototyping with them on the cheap.

I like that idea. I'll see what we can do!