Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Taming the beast that is the Django ORM – An introduction

davidhang.com

Post reply on HN
Print view View original

41–50 of 144 posts

Re: Taming the beast that is the Django ORM – An introduction

#41

Post by santiagobasulto »

I think Django's ORM is just AMAZING. And as with every other tool, it has to be used wisely.

First, it let's you get started quickly and prototype. You can write unit tests, make sure everything is working as expected, then count queries and make sure you're being efficient with your SQL engine.

Second, and even more importantly, it's crucial in the definition of the app and the Schema. Thinking in high level "classes and objects" helps with the abstraction and the design of the apps. Even if you then default to raw SQU queries, thinking and building your model with class abstractions is huge.

Finally, there are some "tiny details" (but in my eyes, very important) that everybody oversees:

* Migrations: the way Django has designed migrations is just marvelous. We've migrated tons of data and changed the structure of our DB multiple times without any issues.

* Troubleshooting and "reporting": the ability to fire a quick shell, load a few models in Django's REPL, and do a few `.filters()` is for me key. On top of that, we add a Jupyter server connected to our read replica and we can do all sorts of reporting VERY QUICKLY. Not everybody needs a Data Lake :)

PS: We've ran Django sites accessed by millions of people per month. And we never had an issue with the ORM. Yes, sometimes I have to tell more junior devs that the gymnastics they're doing with `select_related` and `prefetch_related` can be more easily and effectively resolved with a query. But that's it. I'd say less than 1% of all the queries in our codebase have to be migrated to raw SQL.

Re: Taming the beast that is the Django ORM – An introduction

#42

Post by santiagobasulto »

santiagobasulto wrote: post #41

I think Django's ORM is just AMAZING. And as with every other tool, it has to be used wisely. First, it let's you get started quickly and prototype. You can write unit tests, make sure everything is working as expected, then count queries and make sure you're being efficient with your SQL engine. Second, and even more importantly, it's crucial in the definition of the app and the Schema. Thinking in high level "class…

One common pitfall with Django ORM's is that it makes it very easy the use of inheritance in models. But as we know, the "impedance mismatch" between OOP and the Relational model is a problem.

It has happened that a dev in our team was populating an endpoint that used inheritance and when looking at the number of queries we were over 100.

But the solution in those cases is just OUTER JOINs and CASE. Especially since we use Postgres in the backend and it works so well.

So yes, there are some pitfalls, but they're greatly overshadowed by the advantages of the ORM.

Re: Taming the beast that is the Django ORM – An introduction

#43

Post by switch007 »

nprateem wrote: post #29

For me the killer feature of django is the auto-generated admin UI. I initially started my last project using Spring boot, but I was astonished to find there was no equivalent. I don't know how people build websites in any speed without such a tool. I guess they just waste time duplicating effort on an admin UI or pay for one of those tools that can generate one from an API (meaning they have to also build an API). I…

It can be quite difficult to do anything mildly advanced with the admin UI, IIRC

And it encourages CRUD thinking instead of thinking about business processes and user experience

It's great for tiny/personal projects but in an organisation it can be a trap IMO

Re: Taming the beast that is the Django ORM – An introduction

#44

Post by ruthmarx »

rowanseymour wrote: post #31

I adore the Django ORM but as listed under cons... it makes it very hard to avoid accidental N+1 queries, and they don't seem interested in addressing this ( https://code.djangoproject.com/ticket/30874 ). Yes lazy loading is neat when you're messing around on the shell, but you should absolutely not be leaning on it in production at any kind of scale. So instead you have to use unit tests to hopefully catch any N+1 q…

> they don't seem interested in addressing this (https://code.djangoproject.com/ticket/30874).

Actually it seems they are: https://code.djangoproject.com/ticket/22492#comment:11

Re: Taming the beast that is the Django ORM – An introduction

#45

Post by ruthmarx »

nprateem wrote: post #24

This is largely academic now. LLMs do a good job of writing highly complex queries with the django ORM. All you need is the django toolbar so you can check their efficiency, then keep telling it to make them more efficient.

LLMs are not at a point where we should be treating them as a solution to any software engineering issue, period.

Re: Taming the beast that is the Django ORM – An introduction

#46

Post by andreareina »

wruza wrote: post #32
rowanseymour wrote: post #31

I adore the Django ORM but as listed under cons... it makes it very hard to avoid accidental N+1 queries, and they don't seem interested in addressing this ( https://code.djangoproject.com/ticket/30874 ). Yes lazy loading is neat when you're messing around on the shell, but you should absolutely not be leaning on it in production at any kind of scale. So instead you have to use unit tests to hopefully catch any N+1 q…

What would be the best way?

Static lint wouldn't be a bad way

Re: Taming the beast that is the Django ORM – An introduction

#47

Post by AmericanChopper »

rowanseymour wrote: post #39
abc-1 wrote: post #33

ORMs do a great job at making easy things even easier and hard things a lot harder. If that sounds like a bad deal to you- it’s because it is!

The Django ORM makes migrating database state (a hard problem) super easy. It also makes GROUP BY queries (a relatively easy problem) oddly difficult. Use an ORM where it helps, use SQL where it doesn't.

I personally don’t think DB migrations are that difficult, and I’d say they can be done a lot safer without an ORM. I’ve also never found an ORM that was easier to learn and overall easier to use than SQL. I’ve never understood how ORMs became so popular, I don’t get how somebody can look at how opaque and complicated they are and conclude that learning an ORM is easier than just learning SQL.

Re: Taming the beast that is the Django ORM – An introduction

#48

Post by bruce343434 »

santiagobasulto wrote: post #41

I think Django's ORM is just AMAZING. And as with every other tool, it has to be used wisely. First, it let's you get started quickly and prototype. You can write unit tests, make sure everything is working as expected, then count queries and make sure you're being efficient with your SQL engine. Second, and even more importantly, it's crucial in the definition of the app and the Schema. Thinking in high level "class…

We've had serious trouble with migration merges when two people work on different parts of the code, yet in the same module, and they both generate a "migration nr. 6" on their feature branches.

Re: Taming the beast that is the Django ORM – An introduction

#49

Post by baq »

bitexploder wrote: post #26
choilive wrote: post #7

Earlier quoted context omitted.

Django's ORM is acceptable - but I think Rails/ActiveRecord is superior albeit certainly more opinionated. Most likely just my personal bias speaking because Rails was the first web "framework" I cut my teeth on.

SQLAlchemy is the best stand alone ORM I have found.

sqlalchemy is easily one of the best ORMs ever made for any language. Hats off to zzzeek.

Currently working in typescript and writing SQL queries out by hand since I just don’t trust anything to do the right thing wrt units of work.

Re: Taming the beast that is the Django ORM – An introduction

#50

Post by ninetyninenine »

orms are exercises in OCD.

Databases are the bottleneck your classic website. We choose to query these databases in a extremely high level language called SQL. This language is so high level that you need to come up with tricks and query analyzers in order to hack the high level query into something performant.

A better abstraction would be one that's a bit more similar to a standard programming language with an std that has query related operations/optimizers that can be composed so programmers can compose and choose query operations and avoid optimization issues that are hidden by high level languages like SQL.

We are unfortunately, sort of stuck with SQL (there are other options, but SQL remains popular because years of development has made it pretty good in spite of the fact that it's a poor initial design choice). This is already a historical mistake that we have to live with. Same with javascript (which has been paved over with typescript), same with CSS, etc. The web is full of this stuff. It's fine. My main problem is the ORM.

The ORM is just another layer of indirection. You already have a high level language you're dealing with, now you want to put Another High level language on top of it? ORMs are basically high level languages that compile into SQL.

What is the point? The ORM isn't actually making things easier because SQL is pretty much as high level of a language you can get outside of having an LLM translating direct english.

The point is OCD. Programmers find it jarring to work with SQL strings inside their beautiful language so they want to chop up a SQL string into web app language primitives that they can compose together. Query builders operate on the same concept but are better because they aren't as high level.

This is basically the main reason why Many programmers experience the strange counter intuitive phenomena about why ORMs actually makes things harder. You have to Hack SQL to optimize it. Now you have to hack another ORM on top of it in order to get it to compile it into the hacked query.

Post reply on HN