Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12 requests per second: A realistic look at Python web frameworks

suade.org

Post reply on HN
Print view View original

11–20 of 239 posts

Re: 12 requests per second: A realistic look at Python web frameworks

#11

Post by gwittel »

Laminary wrote: post #4

In my benchmark testing, SSL appears to be the bottleneck; e.g., Apache vs. Nginx does not really matter. I assume the benchmarks above 10,000 RPS are not using SSL and regular HTTP? How are people doing benchmarks at 10k-100k RPS?

SSL handshaking is definitely a bottleneck. I can’t speak for the benchmarks but with SSL typically this scales via using persistent connections (keep alive), and SSL session resumption (much lower cost to setup vs a full handshake).

In the end you want to measure perf with and without SSL so you can identify the real bottleneck. Otherwise you might just benchmark your SSL implementations handshake performance and not what you really want.

As an example, I found that Kubernetes nginx-ingress can’t cache SSL sessions on the upstream side (nginx to your app). So request bursts can really hurt your application unless your pool has enough open connections to handle the burst (keep alive and keep alive requests). Without benchmarking my app in different ways I wouldn’t have figured this out as easily and might have just assumed my app was slow.

Re: 12 requests per second: A realistic look at Python web frameworks

#12

Post by nwsm »

ramraj07 wrote: post #5

Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute( ). Makes a huge difference. Next optimization I do is create stored proc…

For me, FastAPI is the sweet spot between performance and quick development.

Re: 12 requests per second: A realistic look at Python web frameworks

#13

Post by maxpert »

The fact that you are using offset of 50000 and complaining it slows everything down says a lot about the benchmarks. Top it all with ORM query with prefetch all, GIL, and shared CPU (I am guessing) that you used to run benchmark on. You see where this is headed?

Re: 12 requests per second: A realistic look at Python web frameworks

#14

Post by mlthoughts2018 »

I think it’s been bog standard practice to run flask via uwsgi or gunicorn with async workers and use multiple process based workers per deployed server unit (eg per pod in Kubernetes).

What matters is that the cumulative latency & throughput solve your problem, not how fast you can make one singular async worker thread.

I figure most people running complex web services in production would just do an eye roll at this post. Nobody's going to switch to PyPy for any of this.

My team at work runs several complex ML workloads, and we use the exact same container pattern for every service running gunicorn to spawn X async workers per pod and then scale pods per service to meet throughput requirements. Sometimes we also just post complex image processing workloads to a queue and batch them to GPU processor workers. In all these use cases, super low effort “just toss it in gunicorn running flask” has worked without issue for services supporting up to peak load of thousands to hundreds of thousands of requests per second.

Re: 12 requests per second: A realistic look at Python web frameworks

#16

Post by Izkata »

imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

> (usually people writing dumb code without realizing it's expensive)

Some years ago, one morning I gave a co-worker a recommendation on how to improve a loop that was unnecessarily hitting database through the Django ORM. He committed the fix that afternoon. Barely an hour later I accidentally reintroduced the exact same slowdown in the exact same loop when adding a different piece of data to it.

Soooo yeah, ORMs can be so simplistic it's too easy to do by accident even if you know exactly what's going on under the hood.

Re: 12 requests per second: A realistic look at Python web frameworks

#17

Post by divbzero »

imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.

Re: 12 requests per second: A realistic look at Python web frameworks

#18

Post by jsmeaton »

divbzero wrote: post #17
imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.

https://github.com/charettes/django-seal Can help with these kinds of issues though I haven’t yet used it myself (I should!)

Re: 12 requests per second: A realistic look at Python web frameworks

#19

Post by divbzero »

gchamonlive wrote: post #2

I have great experiences with Falcon for backend REST APIs, and it is supposed to be great in terms of requests per second. How does it compare to Sanic?

> I have great experiences with Falcon

Then keep using Falcon! OP’s core thesis is that developer comfort matters more when choosing an API framework because the performance bottleneck is usually found elsewhere. The API itself should be fast enough with some combination of Gunicorn, gevent, PyPy, and horizontal scaling.

Re: 12 requests per second: A realistic look at Python web frameworks

#20

Post by devnonymous »

imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

> My experience doing perf optimizations... without realizing it's expensive).

Completely agree and this has been my experience ae well. To this I'll also add, inadequate thought put into data modelling. One would have to think lesser about query performance or cost of overfetching if data is modelled around the needs of the system it would serve instead of just modelling real life entries and their relationships as is, straight onto the database.

Post reply on HN