61–70 of 239 posts
Earlier quoted context omitted.
for me the value of an ORM is not so much query synthesis as serialisation of object fields to db columns and vice versa
Same here. I'd love to have a tool that just generates an object type for a given SQL query's result rows, and a function signature for its query parameters.
Might as well refer to TechEmpower benchmarks. https://www.techempower.com/benchmarks/
just.js (#9) looks interesting https://github.com/just-js/just It seems to be a much tinier JavaScript runtime than Node.js (still using v8), but linux only The benchmark is probably unrealistically optimized code but even so, it implies Node.js itself has a large performance overhead
Earlier quoted context omitted.
If you're not paying attention it would look that way, yes. If you pay attention you'll notice that just.js is using postgres as their DB, while all of the node benchmarks are handicapped by either using mongoose/mongodb or MySQL. There is no node benchmark with postgres, but all of the fastest benchmarks used it. "lithium" is a good example to show how much of an impact switching to postgres has. All of the 4 lithiu…
Somehow just.js is faster than Rust and C++ in 20-queries benchmark, they're all using postgres. https://www.techempower.com/benchmarks/#section=data-r20&hw=...
> [Multiple Queries] This is the first test where Just(js) has quite a big lead. This is likely due to the fact it is using a custom postgres client written in Javascript and taking full advantage of pipelining of requests. It also avoids sending a Sync/Commit on every query. As far as I am aware this is within the rules but will be happy to make changes to sync on every query if it is not.
https://just.billywhizz.io/blog/on-javascript-performance-01...
My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…
There is lots of truth to this. Some ORMs like Django perform joins in very unsuspecting ways. A simple example is, say, foreign keys. Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. I've spent time optimising things like text searches down from 20…
Hibernate (on Java) at least optimizes this specific use-case. At first, accessing a lazy-loaded property-object will give you a "proxy" and you can access the ID without incurring a database load (since it knows that anyway). And when doing a query, the object won't be joined when requesting book.user.id unless it needs to be (like you have some other WHERE clause that requires an actual join on that row).
https://www.techempower.com/benchmarks/#section=test&runid=8...
7.000.000 requests per second
Even GO can only achieve 4.500.000 million requests per secnod being a low-level language, in opposite to high-level C#.
My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…
I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.
plus it means you don't have to write all the dumb "select * from books where bookName = :bookName" code that obviously can be handled trivially by the ORM. You just use SQL the places where it makes sense.
the ORM hate always strikes me as a little misplaced because of this - you can always write SQL where it's appropriate, in any decent ORM. And you can write bad queries in SQL too. Obviously very complex queries are maybe better reserved for raw SQL but it seems like a lot of the hate comes from maybe less experienced engineers getting in over their head with complex work items because ORMs "make it easy", and that is going to happen with raw SQL too if you throw those same engineers at those work items.
ORMs aren't inherently that heavyweight, I see people complaining here about SQLAlchemy and as a Java developer I don't have performance concerns about hibernate. That sounds to me like a Python problem and a "this specific ORM isn't performant" problem, not ORMs being bad as a whole. And if you really want a "just load the data for me and do nothing else that incurs a performance hit" approach then you can use stateless objects and it's just a wrapper around the DB to load and transform the data for you and/or do a raw, whole-object update back to the DB.
My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…
There is lots of truth to this. Some ORMs like Django perform joins in very unsuspecting ways. A simple example is, say, foreign keys. Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. I've spent time optimising things like text searches down from 20…
Hmm.. Sounds like a bug. Why is this not the same value for a foreign key?
When you start hitting bottlenecks in your python web framework its probably time to switch to a faster language, not another framework in python. You're probably done rapid prototyping by this point anyway.
I don't think there is much gains to rewrite everything in a faster language. Unless they are a very very successful company with billions of customers, it's often cheaper to scale horizontally.
Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute( ). Makes a huge difference. Next optimization I do is create stored proc…
That sounds to me like a Python problem and a "this specific ORM isn't performant" problem, not ORMs being bad as a whole. Python has never been the fastest language (it's far slower than, say, Java) and the GIL really prevents applications from scaling well without multiple instances.
And if you really want a "just load the data for me and do nothing else that incurs a performance hit" approach then you can use stateless objects and the ORM truly becomes just a wrapper around the DB to load and transform the data into an object for you and/or do a raw, whole-object update back to the DB.
Long story short, fastapi was much much faster than anything else for us. It also felt a bit like flask. The integration with pydantic for validating dataclasses on the fly was also great.