Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12 requests per second: A realistic look at Python web frameworks

suade.org

Post reply on HN
Print view View original

191–200 of 239 posts

Re: 12 requests per second: A realistic look at Python web frameworks

#191

Post by cztomsik »

airstrike wrote: post #177
stevenjohns wrote: post #39

Earlier quoted context omitted.

There is lots of truth to this. Some ORMs like Django perform joins in very unsuspecting ways. A simple example is, say, foreign keys. Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. I've spent time optimising things like text searches down from 20…

> ...in very unsuspecting ways > Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. But that's not really unsuspecting. `book.user` is asking for the user table, `book.user_id` is not. Those two things are not identical even though they return the sam…

IIRC spark can optimize it away, and in OO there shouldn't be user_id

Re: 12 requests per second: A realistic look at Python web frameworks

#192

Post by jononor »

polyrand wrote: post #140

Related to ORMs/queries/performance, I have found the following combination really good: * aiosql[0] to write raw SQL queries and having them available as python functions (discussed in [1]) * asyncpg[2] if you are using Postgres * Map asyncpg/aiosql results to Pydantic[3] models * FastAPI[4] Pydantic models become the "source of truth" inside the app, they are designed as a copy of the DB schema, then functions rece…

Is there any automatic glue between SQL and Pydantic, or is each mapping hand-written?

Re: 12 requests per second: A realistic look at Python web frameworks

#193

Post by airstrike »

cztomsik wrote: post #191
airstrike wrote: post #177

Earlier quoted context omitted.

> ...in very unsuspecting ways > Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. But that's not really unsuspecting. `book.user` is asking for the user table, `book.user_id` is not. Those two things are not identical even though they return the sam…

IIRC spark can optimize it away, and in OO there shouldn't be user_id

> in OO there shouldn't be user_id

I don't think that's right. There is a user_id column in the book table, so why shouldn't there be book.user_id?

Re: 12 requests per second: A realistic look at Python web frameworks

#194

Post by kaba0 »

jtdev wrote: post #147
kaba0 wrote: post #118

Earlier quoted context omitted.

Yeah, ORMs are specifically made for insert-heavy operations or very basic mapping of rows to objects. Analytical queries and the rest should be done with SQL.

I keep seeing some variation of: “ORMs are really intended for x” or “ORMs work best when y is true”. At some point we should take an honest look to determine if ORMs provide enough benefit to justify their existence. “I can talk to the database without knowing SQL” doesn’t cut it.

Yes they do? They are basically available in every language, in some ecosystems there are multiple alternatives as well and there would not be that big of a market unless it is useful.

Re: 12 requests per second: A realistic look at Python web frameworks

#195

Post by magicalhippo »

zzzeek wrote: post #174
gregmac wrote: post #162

Earlier quoted context omitted.

The most insidious part about misusing ORMs is it's often not visible for a while. Modern DBMSs on modern hardware are crazy fast , so when you have only a few tens or hundreds of thousand rows in your table, those inefficient and pointless ORM queries are just not noticeable because you still get sub-second response times. As your database grows, the site begins to gets slower and slower, but it's hard to distinguis…

To be fair this is a problem inherent to databases in general. You can have hand written queries that perform badly due to structure or query frequency as well which are not apparent until the dataset grows. The ORM should make it easier to rectify such situations (eg drop in an eager loading directive) vs having to restructure hand-written routines for similar effects.

Indeed, even with query analyzer you might see say table scans instead of index scan just because the DB realizes just scanning the 100 rows you got is faster than trying to use an index.

So without a large number of rows it can be hard to know what it will actually do.

Re: 12 requests per second: A realistic look at Python web frameworks

#196

Post by Terretta »

Not that it matters any more, but a colleague mentions Flask was originally a joke of what not to do:

https://lucumr.pocoo.org/2010/4/3/april-1st-post-mortem/

Flask author reflects on that here:

http://mitsuhiko.pocoo.org/flask-pycon-2011.pdf

Quite relevant to the conclusion in the article.

Re: 12 requests per second: A realistic look at Python web frameworks

#197

Post by jerf »

imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

I'm increasingly banging on the drum that web frameworks shouldn't be measured in requests per second but seconds per request. It sounds really impressive to go from 100,000 to 500,000 requests per second. It's somewhat less impresssive if you consider that's going from 10 microseconds to 2 microseconds... if you consider that your real, non-benchmark handler is likely in the dozens of milliseconds.

I've got a couple of web handlers that after quite a bit of work I can legitimately claim will run on the microsecond timeframe... but they're the exception. Generally even a single DB hit across the network, even on the same system, is going to blow right past the web framework's time you're using.

On that note, using this sort of metric, japronto's claimed results smell funny. Even with a 4GHz processor, getting 1,214,440 requests per second on a single core is ~3300 cycles per request. That's less than one cycle per byte in the HTTP request for a reasonable request (with no blocking on any sort of memory request), and that's not counting the TCP itself, any response, or the overhead of switching back and forth between C and Python. I can't see how this is possible without a huge degree of corner cutting; just validating that what you've received is a legal HTTP request, correctly encoded, decoding the fields, etc. is going to eat into that pretty fast, even with all the SSE instructions you may be able to throw at it. (And to emphasize, I'm not saying this is "impossible", just that it requires a lot of corner cutting. I've also got a "web server" out in the wild that handles "web requests" blazingly fast... because it basically ignores the entire web request and shovels out a hard-coded response. Very fast. Not a very good web server.)

Re: 12 requests per second: A realistic look at Python web frameworks

#198

Post by [deleted] »

stiray wrote: post #93
divbzero wrote: post #17

Earlier quoted context omitted.

I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.

I agree with you. ORMs are at the end "converting" complexity of SQL capabilities into objects. It is perfectly normal that in best situation they will get at least as complex as SQLs but with features hidden behind annotations or be less efficient. SQL is readable (at least far more than 20m lasigna of boilerplate objects decorated with tons of annotations googled from internet where no one really know what they do)…

[deleted]

Re: 12 requests per second: A realistic look at Python web frameworks

#199

Post by closeparen »

imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

With Symfony back in the day, you could turn on a little toolbar that would show in the top of your rendered HTML. You could see how many SQL queries were run, what the statements were, how long they took. I think there was some other tracing information as well. This strikes me as a basic and necessary kind of testing to do when developing with an ORM. Perhaps more systematically you could get ORM business into Zipkin or Jaeger, and then have some kind of staging vs. prod or canary vs. prod statistical comparison to see if you’re about to release something dumb. Or maybe simpler to keep generated SQL in unit test assertions. You wouldn’t have to write it, but you would have to read and update it on changes, so you could notice if you were winding up with N+1 queries or a ridiculous join.

Re: 12 requests per second: A realistic look at Python web frameworks

#200

Post by ancount »

I inherited a flask queue worker, and it suffers from some major problems (like 12 req/second when it's not discarding items from the queue). I am primarily a javascript programmer so I'm a little bit out of my element.

I am tempted to refactor the worker to use async features, and that would require factoring out uWSGI, which is fine, I only added it last week. The article states that Vibora is a drop in replacement for flask, but I guess I'm a bit skeptical, as I can't find much information outside of Vibora having a similar api. For a web service with basically one endpoint, I could refactor to another implementation fairly easily, I'm just looking for the right direction.

I thought maybe I should refactor the arch to either batch requests to the worker, or to use async. Anyone have a feeling where I should go? I am just getting started researching this, but any advice would be appreciated.

Edit: at least quart has a migration page.. probably will just try it out, what can I lose? https://pgjones.gitlab.io/quart/how_to_guides/flask_migratio...

Second edit: Also might try out polyrand's stack in the comments.

Post reply on HN