12 requests per second: A realistic look at Python web frameworks

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?

Re: 12 requests per second: A realistic look at Python web frameworks

#5

Post by ramraj07 » Fri, Feb 19, 2021, 3:11 AM UTC

Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html

If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute(). Makes a huge difference. Next optimization I do is create stored procedures on the database. Only then I start thinking about changing the framework itself.

For folks like me who want to get prototypes off the ground in hours, flask and fastapi are godsend, and if that means I have to worry about serving thousands of requests a second soon thats a happy problem for sure.

Re: 12 requests per second: A realistic look at Python web frameworks

#6

Post by Jarred » Fri, Feb 19, 2021, 3:13 AM UTC

est wrote: post #3

Might as well refer to TechEmpower benchmarks. https://www.techempower.com/benchmarks/

just.js (#9) looks interesting

https://github.com/just-js/just

It seems to be a much tinier JavaScript runtime than Node.js (still using v8), but linux only

The benchmark is probably unrealistically optimized code but even so, it implies Node.js itself has a large performance overhead

Re: 12 requests per second: A realistic look at Python web frameworks

#7

Post by tnash » Fri, Feb 19, 2021, 3:19 AM UTC

Use of ORMs is often a performance choke point. Raw DB queries are often much, much faster. Almost always, the more you abstract, the worse you perform. It's great as a developer but not so great as a user.

Re: 12 requests per second: A realistic look at Python web frameworks

#8

Post by rovr138 » Fri, Feb 19, 2021, 3:20 AM UTC

Laminary wrote: post #4

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?

Where is SSL the bottleneck? Wondering if terminating earlier and just relying on HTTP after would help.

Re: 12 requests per second: A realistic look at Python web frameworks

#9

Post by kijin » Fri, Feb 19, 2021, 3:21 AM UTC

Laminary wrote: post #4

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?

HTTP/1.1 Keep-Alive helps a lot. HTTP/2 is even better. You only do the SSL handshake once.

Re: 12 requests per second: A realistic look at Python web frameworks

#10

Post by imperio59 » Fri, Feb 19, 2021, 3:24 AM UTC

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive).

Sure, the framework matters at very large scale and the benefits from optimizing the framework become large when you're doing millions of requests a second over many thousands of servers because it can help reduce baseline cost of running the service.

But I agree with the author's main point which seems to be that framework performance is pretty meaningless when comparing frameworks if you're just starting on a new project. Focus on making a product people wanna actually use first. If you're lucky enough to get to scale you can work about optimizing it then.