Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12 requests per second: A realistic look at Python web frameworks

suade.org

Post reply on HN
Print view View original

131–140 of 239 posts

Re: 12 requests per second: A realistic look at Python web frameworks

#131

Post by arcturus17 »

kavalg wrote: post #94
cryptos wrote: post #58

Why Python at all? About 10 years ago I liked Python a lot (and still like it in principle) and felt very productive compared to, say, Java. Java was full of inconvenience, XML, bloated frameworks and all that. But today you can use Kotlin, that is in my opinion even nicer than Python, with performant frameworks (e. g. Quarkus or Ktor) on the super fast JVM. I don't want to start a language war, but maybe Python is n…

I can code comfortably in Python, Java, JavaScript and to some extent C/C++. In the last 4 years I have been using mostly Python for various reasons (Machine Learning, OS automation, web scraping ...). Compared to the other languages, Python feels lighter and faster to write to the extent that it rarely interrupts my flow of thoughts. Now and then I have to code in JavaScript (frontend), C/C++ (embedded / low level o…

I agree with your general take on developer productivity, but I don't feel that modern JS is significantly messier than Python, at least not to a level where it significantly impacts productivity (I'd rather avoid a debate on the abyssal depths of the language, eg, type coercion)

I feel about the same amount of grievances with both. For instance I dislike Python's async and functional semantics (list(map(lambda n...). But it has a much better standard library overall, a lot of the scripting syntax semantics (file opening, requests, etc.) feel cleaner, etc.

I'm more versed in JS but as the knowledge curves converge in months or years to come I don't think I'll be significantly more productive in Python.

I know you were expressing an entirely subjective opinion, probably contingent on the amount of day-to-day practice you have with each language (you say you code JS in the front-end now and then... I write a lot of it), but I still wanted to offer my counterargument.

Re: 12 requests per second: A realistic look at Python web frameworks

#132

Post by dfgdghdf »

I would question choosing Python for large server projects because the performance ceiling is so low. At least with the "middle tier" performance languages such as Java / C# you are unlikely to require a complete language switch as the project scales.

Re: 12 requests per second: A realistic look at Python web frameworks

#133

Post by qeternity »

twsted wrote: post #82
qeternity wrote: post #74

As a Django shop, we’ve always hoped PyPy would one day be suitable for our production deployments but in the end with various issues we were never able to make the switch. And then Pyston was re-released...and changed everything. It was drop in compatible for us and we saw a 50% drop in latencies. Source availability aside, I suggest anyone running CPython in prod take a look.

Can you tell us more about that? I remember pyston v1 from Dropbox. You are speaking about v2, which is a binary package (closed-source at the moment)?

Yeah it's a closed source binary.

We're very happy with it. Great compatibility, no horrendous warm up times, and very meaningful speedups.

Not much more to say. It's the same, just a bit faster.

Re: 12 requests per second: A realistic look at Python web frameworks

#134

Post by lacksconfidence »

m12k wrote: post #122
lacksconfidence wrote: post #114

Earlier quoted context omitted.

No, that is clearly a bug. The ORM already has the value of book.id, that's how it knows how to fetch the right book. Performing extra queries is just poor implementation.

The programmer also already had the value in book.user_id but still chose to ask the ORM to fetch all of .user so they could get .id from there instead. And they might then afterward call .name on it as well, and there would be no further queries, because the ORM has already been asked to fetch all fields of .user - so it might in fact have been sensible to fetch all of .user if so. The query builder cannot know whet…

Perhaps i'm a bit odd, but when I'm going to lean on an ORM to do things I expect it to actually do them. I expect that foo.user_id does not exist, because that representation has been transformed into an object. foo.user.id should be the only viable reference to the id. foo.user.id should return the value it already knows, any other property access i would expect will do the equiv of `select * from ...` if the object has not previously been populated.

Now perhaps some ORM's prefer to be thinner, to provide more footguns via a leaky abstraction that mixes implementation details with the object mapping. I don't think those are good implementaations.

Re: 12 requests per second: A realistic look at Python web frameworks

#135

Post by BiteCode_dev »

sdevonoes wrote: post #126

Interesting. I never heard of Japronto before. For the people working with Python: why Flask instead of Japronto?

The ecosystem.

You got loads of libraries built around flask, and a lot of doc and tutorials, not to mention how much battle tested it is.

Those can matter much more than hypothetical perfs in a synthetic benchmark.

Re: 12 requests per second: A realistic look at Python web frameworks

#136

Post by BiteCode_dev »

arcturus17 wrote: post #124
nodamage wrote: post #86

Earlier quoted context omitted.

That 7 million requests per second is achieved by writing a hard coded plain text HTTP response string directly to the client... it is so far disconnected from any real world use case that the number is basically meaningless.

It's still 8th in the composite benchmark. And the criticism you're leveling would affect the entire benchmark design, rather than a particular framework score, no? [1] https://www.techempower.com/benchmarks/#section=data-r20&hw=...

Yes but getting a good ranking at a useless benchmark if not very meaningful. It's cool, but I wouldn't use it to make any serious decision.

Re: 12 requests per second: A realistic look at Python web frameworks

#137

Post by BiteCode_dev »

arcturus17 wrote: post #131
kavalg wrote: post #94

Earlier quoted context omitted.

I can code comfortably in Python, Java, JavaScript and to some extent C/C++. In the last 4 years I have been using mostly Python for various reasons (Machine Learning, OS automation, web scraping ...). Compared to the other languages, Python feels lighter and faster to write to the extent that it rarely interrupts my flow of thoughts. Now and then I have to code in JavaScript (frontend), C/C++ (embedded / low level o…

I agree with your general take on developer productivity, but I don't feel that modern JS is significantly messier than Python, at least not to a level where it significantly impacts productivity (I'd rather avoid a debate on the abyssal depths of the language, eg, type coercion) I feel about the same amount of grievances with both. For instance I dislike Python's async and functional semantics ( list(map(lambda n...…

> list(map(lambda n...

List comprehensions are much better for this. Functional doesn't mean you have to use a function call. If you can use the paradigm with literal syntax, just do so.

Re: 12 requests per second: A realistic look at Python web frameworks

#138

Post by bottled_poe »

lacksconfidence wrote: post #134
m12k wrote: post #122

Earlier quoted context omitted.

The programmer also already had the value in book.user_id but still chose to ask the ORM to fetch all of .user so they could get .id from there instead. And they might then afterward call .name on it as well, and there would be no further queries, because the ORM has already been asked to fetch all fields of .user - so it might in fact have been sensible to fetch all of .user if so. The query builder cannot know whet…

Perhaps i'm a bit odd, but when I'm going to lean on an ORM to do things I expect it to actually do them. I expect that foo.user_id does not exist, because that representation has been transformed into an object. foo.user.id should be the only viable reference to the id. foo.user.id should return the value it already knows, any other property access i would expect will do the equiv of `select * from ...` if the objec…

I’m sure there are valid engineering reasons to do it this way. One that comes to mind is memory footprint in allocating the objects associate with foreign key references.

Re: 12 requests per second: A realistic look at Python web frameworks

#139

Post by smoe »

vosper wrote: post #23
ramraj07 wrote: post #5

Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute( ). Makes a huge difference. Next optimization I do is create stored proc…

You can also use SQLAlchemy Core, which is an intermediate between the full-blown ORM and running actual strings of SQL. I've had a great experience with Core - I can easily have it output essentially the exact SQL I'd write by hand, but I get many benefits (like the ability to compose queries) that are nicer than dealing with raw SQL.

This is what I like most about SQLAlchemy, I can use the ORM most of the time, but in cases where it needs optimizations or aggregation queries I can drop down or refactor to core, while still using the same model/table definitions, ability to combine or filter queries dynamically and leverage IDE features like refactoring.

Re: 12 requests per second: A realistic look at Python web frameworks

#140

Post by polyrand »

Related to ORMs/queries/performance, I have found the following combination really good:

* aiosql[0] to write raw SQL queries and having them available as python functions (discussed in [1])

* asyncpg[2] if you are using Postgres

* Map asyncpg/aiosql results to Pydantic[3] models

* FastAPI[4]

Pydantic models become the "source of truth" inside the app, they are designed as a copy of the DB schema, then functions receive and return Pydantic models in most cases.

This stack also makes me think better about my queries and the DB design. I try to make sure each endpoint makes only a couple of queries. Each query may have multiple CTEs, but it's still only a single round-trip. That also makes you think about what to prefetch or not, maybe I want to also get the data to return if the request is OK and avoid another query.

[0] https://github.com/nackjicholson/aiosql [1] https://news.ycombinator.com/item?id=24130712 [2] https://github.com/MagicStack/asyncpg [3] https://pydantic-docs.helpmanual.io/ [4] https://fastapi.tiangolo.com/

Post reply on HN