Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

12 requests per second: A realistic look at Python web frameworks

suade.org

Post reply on HN
Print view View original

231–239 of 239 posts

Re: 12 requests per second: A realistic look at Python web frameworks

#231

Post by akra »

arcturus17 wrote: post #128
EvilEy3 wrote: post #97

Earlier quoted context omitted.

Go being low-level has nothing to do with performance. They deliberately keep feature set on this level, they're not making some kind of trade-off. Also, high performance C# is so low level that you might as well write C++. Or you think you will use EF, LINQ and have 7 millions rps?

> Or you think you will use EF, LINQ and have 7 millions rps? I'm imagining not, but it's still comparatively faster than most, if not all, "full-batteries frameworks", right? I'm assuming the use of ORMs and such is more or less uniform in the comparison (eg, if they don't use EF for .net, they don't use Django ORM either) The overall performance of .net across these benchmarks really catches my attention like it do…

A framework that you can stay high level if you want and optimise when the rare but exceptional case arises on a particular route without splitting the process/resorting to C++ interop (with its own performance issues), or the risk of needing to rewrite due to performance to me is the big selling point. In a previous life I got some significant latency benefits by doing just that. I think its probably only gotten better with .NET 5.

Re: 12 requests per second: A realistic look at Python web frameworks

#232

Post by oliwarner »

The important thing to remember is that unless you're running a massive service, requests per second is less important than seconds per request.

Getting an API hit from 300ms to 70ms, and proper frontend caching is far more valuable than concurrency (if you can afford to throw servers at it) because it actually affects user performance.

Re: 12 requests per second: A realistic look at Python web frameworks

#233

Post by BiteCode_dev »

polyrand wrote: post #221
BiteCode_dev wrote: post #163

Earlier quoted context omitted.

I always though it would be a nice alternative to an ORM: having a tool that take a marshmallow/pydantic/whatever model, optionally passing it additional db specific options, then it generates a bunch of sql files you can call with aiosql. The whole things would then let you optionally get the result wrapped in a model if you need to, with ORM like helpers for common CRUD things. That would have the benefit of the st…

Absolutely! I think it should be doable, FastAPI and other libraries do heavy Pydantic model inspection, and some do automatic code generation. I should explore this a bit more.

It also has some nice properties:

- because you have all your queries written in the same place, when you create a db migration, you know where to look for code to update, and can even perform some automatically.

- sql queries could be turned into a stored procedure with a simple marking, without changing any code.

Of course the downside is that dynamically exploring data is not as easy as with an ORM.

Re: 12 requests per second: A realistic look at Python web frameworks

#234

Post by paulmd »

deergomoo wrote: post #165
arcbyte wrote: post #154

Earlier quoted context omitted.

I have never seen anyone, even the ORM "experts" in my jobs, save time with an ORM over what a junior engineer couldn't do faster with plain SQL. The reason is that sure, for your first 10 basic select queries, the ORM saved you half an hour. Then you got to that complicated join and had to resort to looking up archane syntaxes and prototyping attribute quirks for an hour, when the junior guy got the whole thing writ…

> The reason is that sure, for your first 10 basic select queries, the ORM saved you half an hour Maybe we work in very different fields, but "basic select queries" makes up 90% of what I need to fetch from the database. If I'm working on a forum and I want to load user 123, with all their posts, all the awards each post has, and the count of friends the user has, with Eloquent (Laravel's ORM), I could do: `User::wit…

I do see the larger point though that ORM usage could still translate into problems elsewhere when you don't pull data you should, or you pull data you shouldn't, or pull data in a way you shouldn't.

But the thing is, a lot of things aren't really on the hot path and optimizing them isn't worth a ton of time. Like yeah so what if this web request that only gets used 2% of the time makes 5 extra database requests that it shouldn't, on a single data item. Not gonna tank the overall program.

I guess I'd accept that it's important to be aware of what you're pulling, regardless of whether that's automagically when a proxy object sees it needs to be lazy loaded, or explicitly in a query. Throwing junior developers on performance-critical paths is going to be a problem anywhere and on any DB access layer.

Re: 12 requests per second: A realistic look at Python web frameworks

#235

Post by aabbcc1241 »

jtolmar wrote: post #43
zem wrote: post #25

Earlier quoted context omitted.

for me the value of an ORM is not so much query synthesis as serialisation of object fields to db columns and vice versa

Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.

Just hack that for Typescript over this weekend (: https://news.ycombinator.com/item?id=26211286

Re: 12 requests per second: A realistic look at Python web frameworks

#236

Post by cztomsik »

airstrike wrote: post #193
cztomsik wrote: post #191

Earlier quoted context omitted.

IIRC spark can optimize it away, and in OO there shouldn't be user_id

> in OO there shouldn't be user_id I don't think that's right. There is a user_id column in the book table, so why shouldn't there be book.user_id?

depends if you're doing active record (which I am not considering as typical ORM) or data mapper like hibernate where your entity is POJO with id and annotations but typically no foreign keys (just references to other model classes)

hibernate is then "the magic "environment where it just works"

I realize OO is out of fashion now but it's still true and it still works and I've been in a lots of projects where ORM was useful

Re: 12 requests per second: A realistic look at Python web frameworks

#237

Post by jtolmar »

grosskur wrote: post #61
jtolmar wrote: post #43

Earlier quoted context omitted.

Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.

sqlc can do this: https://sqlc.dev/

This is the nicest one of these from the ones posted, I think it's the model to copy.

Re: 12 requests per second: A realistic look at Python web frameworks

#238

Post by jtolmar »

aabbcc1241 wrote: post #235
jtolmar wrote: post #43

Earlier quoted context omitted.

Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.

Just hack that for Typescript over this weekend (: https://news.ycombinator.com/item?id=26211286

Wow, that was fast! I'm impressed.

If you want to keep going forward with it, I'd recommend working on making it clearer what the examples do, and looking at sqlc for Go for inspiration. Of the existing versions of this people replied with, that one had the clearest API, or at least clearest explanation on its landing page.

Re: 12 requests per second: A realistic look at Python web frameworks

#239

Post by wirrbel »

divbzero wrote: post #17
imperio59 wrote: post #10

My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…

I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.

SQLAlchemy expression language is cool stuff. not an ORM but also no string-templating ...
Post reply on HN