111–120 of 239 posts
Earlier quoted context omitted.
Funnily enough, I recently optimized some code along these lines. The way I sped it up was to call `.values()` on the query, which serializes the data into a dict and prevented me from accidentally making subsueqent calls. PS: Indent by 4 spaces for code formatting.
s/ident/indent/
My experience doing perf optimizations in real world systems with many many people writing code to the same app is a lot of inefficiencies happen due to over fetching data, inefficiencies caused by naively using the ORM without understanding the underlying cost of the query, and lack of actual profiling to find where the actual bottlenecks are (usually people writing dumb code without realizing it's expensive). Sure,…
There is lots of truth to this. Some ORMs like Django perform joins in very unsuspecting ways. A simple example is, say, foreign keys. Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. I've spent time optimising things like text searches down from 20…
https://docs.djangoproject.com/en/dev/topics/testing/tools/#...
Not quite sure if their current site's code is opensource... anyone know?
Earlier quoted context omitted.
> A simple example is, say, foreign keys. Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. Hmm.. Sounds like a bug. Why is this not the same value for a foreign key?
Well, it is the same value, but the ORM doesn't handle book.user.id any differently than it does book.user.name where it isn't the same value, and thus the only option is to fetch the second table. So it's not a bug, it's really just the ORM being consistent in how it handles queries, thus missing out on a possible optimization in this special case, where a simpler query could have given the same result.
Earlier quoted context omitted.
> A simple example is, say, foreign keys. Trying to access the foreign key of an object by doing `book.user.id` does an additional query for the user table to get the ID. It's less known that the id is immediately available by just doing `book.user_id` instead. Hmm.. Sounds like a bug. Why is this not the same value for a foreign key?
Because when you are acessing .user you are asking for all its properties, not just the id. Django does provide relatively easy ways to get over the N+1 issue, though. If you do Book.objects.select_related('user'), only one query is made.
C#/ASP.NET is the fastest web framework now: https://www.techempower.com/benchmarks/#section=test&runid=8... 7.000.000 requests per second Even GO can only achieve 4.500.000 million requests per secnod being a low-level language, in opposite to high-level C#.
C# is only at number 3 in your list. Both Java and Rust are above it in the list. It's also a very "artificial" benchmark and real world code will give different results (if you have static content, just put it in a CDN and don't worry) Other benchmarks from the same site: - JSON Serialization: C# is number 34 - Single query: C# is number 23 - Fortunes: C# is number 7
https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
There's no routing or templating, it just writes a bunch of strings. No one would build an actual web app this way.
The only C# benchmarks that are remotely realistic are the mvc variants, starting with aspcore-mvc-ado-pg at number 79.
Earlier quoted context omitted.
Well, it is the same value, but the ORM doesn't handle book.user.id any differently than it does book.user.name where it isn't the same value, and thus the only option is to fetch the second table. So it's not a bug, it's really just the ORM being consistent in how it handles queries, thus missing out on a possible optimization in this special case, where a simpler query could have given the same result.
No, that is clearly a bug. The ORM already has the value of book.id, that's how it knows how to fetch the right book. Performing extra queries is just poor implementation.
If it's a bug, sounds like a "wontfix" to me.
Earlier quoted context omitted.
I increasingly lean towards plain SQL over ORMs. It requires greater familiarity with SQL but I prefer that over greater familiarity with ORM-specific syntax that doesn’t translate across frameworks or languages. In addition, you can prototype new queries and profile existing queries in the database and copy-paste directly into your code.
IMO both are required. I'm lucky in that I did a lot of plain SQL early on, and found ORMs later, but I think ORMs do cut out a lot of time for quick-and-dirty queries that end up not being the bottleneck. The problem arises once you find a bottleneck, you won't know how to optimize it if you haven't done a bit of SQL mucking about earlier. Also, the big thing is you won't know how to translate to other ORMs if you d…
Earlier quoted context omitted.
if that's your bag then you can still totally do that with an ORM, hibernate for example lets you just write a whole query in raw SQL while still getting all the benefits of eliding a bunch of boilerplate field copying, having "active" objects with ORM-level update/transaction management, etc. plus it means you don't have to write all the dumb "select * from books where bookName = :bookName" code that obviously can b…
We're gradually removing our ORM usage to avoid active objects. They look too much like VSTs, but subtly make foreign network calls when you least expect it, which makes it very hard to write effective, isolated tests or replace the database layer by something else, like a service call or a cache lookup.
Don't forget that you're paying a huge price using the sqlalchemy orm - https://docs.sqlalchemy.org/en/13/faq/performance.html If I know an endpoint is going to be hit hard, I forgo trying to use the ORM (except to maybe get the table name from the model obj so some soul can trace it's usage here in the future) and directly do an engine.execute( ). Makes a huge difference. Next optimization I do is create stored proc…
ORMs aren't inherently that heavyweight, as a Java developer I don't have performance concerns about hibernate. That sounds to me like a Python problem and a "this specific ORM isn't performant" problem, not ORMs being bad as a whole. Python has never been the fastest language (it's far slower than, say, Java) and the GIL really prevents applications from scaling well without multiple instances. And if you really wan…