Live data from Hacker News

12 requests per second: A realistic look at Python web frameworks

suade.org

151–160 of 239 posts

Re: 12 requests per second: A realistic look at Python web frameworks

#151
post #113

I always assumed Python could scale because of Reddit: https://github.com/reddit-archive/reddit Not quite sure if their current site's code is opensource... anyone know?

And because of Instagram: https://instagram-engineering.com/python-at-scale-strict-mod...

Re: 12 requests per second: A realistic look at Python web frameworks

#152

Earlier quoted context omitted.

My guess is accessing a related field within a loop causing a database request per iteration, e.g. ``` [book.author.name for book in Book.objects.all()] ```

Maybe I spent too much time with Django already, but if I see anyone doing anything but Book.objects.values_list('author__name', flat=True) for this type of expression, I would mark it as a 3x WTF? in the code review.

I’d also allow prefetch_related if you’re using more than the most trivial data - no point in duplicating logic you have in your models if you have a method which generates something like a name, URL, etc. based on multiple fields.

Re: 12 requests per second: A realistic look at Python web frameworks

#153
It's a bit of a step back in time reading things like this.

This is stateless HTTP requests hitting a relational database. How is this dead horse still being beaten? The patterns for load balancing, horizontal scalability, caching in this space well documented.

What are we gaining still profiling Django, Flask and Ruby on Rails in 2021.

Re: 12 requests per second: A realistic look at Python web frameworks

#154
post #66

Earlier quoted context omitted.

I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.

if that's your bag then you can still totally do that with an ORM, hibernate for example lets you just write a whole query in raw SQL while still getting all the benefits of eliding a bunch of boilerplate field copying, having "active" objects with ORM-level update/transaction management, etc. plus it means you don't have to write all the dumb "select * from books where bookName = :bookName" code that obviously can b…

I have never seen anyone, even the ORM "experts" in my jobs, save time with an ORM over what a junior engineer couldn't do faster with plain SQL.

The reason is that sure, for your first 10 basic select queries, the ORM saved you half an hour. Then you got to that complicated join and had to resort to looking up archane syntaxes and prototyping attribute quirks for an hour, when the junior guy got the whole thing written in 15 minutes of trial/stackoverflow/error in a SQL prompt.

Then, even when the "expert" did get it working, guess who is going to be the one looking at it again when trying to figure out production support issues? The junior guy, who is now clueless and has to spend two hours to figure out what this crazy ORM mess does here. The better alternative was just to have the SQL there ready to go so it is well understood and can be ran against the production database or a test database to reproduce the issue. No questions whatsoever.

I have seen this over and over again, more than a statistically relevant number of times.

Re: 12 requests per second: A realistic look at Python web frameworks

#155
post #113

I always assumed Python could scale because of Reddit: https://github.com/reddit-archive/reddit Not quite sure if their current site's code is opensource... anyone know?

Any language can scale. Pretty much most languages there can handle scale. It's usually bad algorithms or external services be it API & DB used poorly that impacts performance.

Re: 12 requests per second: A realistic look at Python web frameworks

#156
post #153

It's a bit of a step back in time reading things like this. This is stateless HTTP requests hitting a relational database. How is this dead horse still being beaten? The patterns for load balancing, horizontal scalability, caching in this space well documented. What are we gaining still profiling Django, Flask and Ruby on Rails in 2021.

I suppose every app you work on runs the same query repeatedly? Yes, load balancing makes sense, but the author is specifically looking at requests per thread, i.e., wouldn't it be great (and more cost effective) to get as much throughput from a single thread as possible?

Re: 12 requests per second: A realistic look at Python web frameworks

#157

Good article, but I can't help but notice a gaping hole in the benchmark -- why was there no attempt to run gunicorn in multi-threaded mode? The article has a link to https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a... , but failed to mention the key takeaway from the article: > threaded code got the job done much faster than asyncio in every case

The article explicitly says they were testing single threaded use cases.

Re: 12 requests per second: A realistic look at Python web frameworks

#158
post #43
post #25

Earlier quoted context omitted.

for me the value of an ORM is not so much query synthesis as serialisation of object fields to db columns and vice versa

Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.

"just generates an object type for a given SQL query's result rows"

For a similar purpose in Python I generally use psycopg with namedtuple resultsets; the namedtuple 'records' do what I need for the returned data and are reasonably efficient.

Re: 12 requests per second: A realistic look at Python web frameworks

#159

Good article, but I can't help but notice a gaping hole in the benchmark -- why was there no attempt to run gunicorn in multi-threaded mode? The article has a link to https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a... , but failed to mention the key takeaway from the article: > threaded code got the job done much faster than asyncio in every case

The article explicitly says they were testing single threaded use cases.

But why? The results would be a lot more useful if they include a single-process multi-threaded setup in the benchmark.

Re: 12 requests per second: A realistic look at Python web frameworks

#160
post #141
post #123

Earlier quoted context omitted.

Yes, things like setString(1, “asd”) and conversely getInt(2) are so beautiful and will never introduce mistakes /s ORMs are there for mapping objects and well, relations. Most ORMs provide additional features, but at the core they can, and for more complicated queries they should be used with native SQL queries. They are made for OLTP not for OLAP

Like there wouldn't be anything in between /s There are "simple ORMs" that only map results of SQL queries to objects. They do not provide a magic query API - which is the source of most problems. I don't do Python, but for .NET there is Dapper https://github.com/StackExchange/Dapper , you can have a look what I mean. You write the SQL query, explicitly execute it, the library maps the results of that query into obje…

+1 for Dapper. It does have some limited "magic" query building features, but only for straightforward CRUD operations that don't involve joins. And that's a Good Thing (TM).
Post reply on HN