Building a higher-level query API: the right way to use Django's ORM
1–10 of 30 posts
Re: Building a higher-level query API: the right way to use Django's ORM
#2Correct. Another benefit: easier to implement cache between the view and the DB down the road.
Re: Building a higher-level query API: the right way to use Django's ORM
#3I'm a freelancer and I've been using this approach since ever. I wish more developers did too because it's really frustrating when they don't and you need to maintain their code.
Re: Building a higher-level query API: the right way to use Django's ORM
#4I see what he's trying to do but this just seems like a ton of extra "boilerplate" code when you're trying to make an app.
I'd rather spend extra time tracking down where I've used the is_done field if I later change it to a status, than spend all this time writing a manager for every query I do. Unit tests help with catching it if you've missed somewhere.
On the other hand, if you have an extremely complicated query, then this might make sense. Maybe he chose that example for illustrative purposes but it's not really the kind of thing he's recommending you use this for?
Re: Building a higher-level query API: the right way to use Django's ORM
#5The main point raised by the article is spot-on, and I'm ashamed to say that I had never recognised it as an issue before reading it. It applies even more strongly for more complex lookups (possibly involving Q objects), which I've always felt would find a better home in models.py than in views.py. And I too cringe every time I come across the django.db.models.manager source code.
Some thoughts:
The approach goes slightly against the commandment of "there should just be one way of doing it". It's probably best to apply it sparingly, only where there are very clear readability and/or DRYness improvements to be had.
The `PassThroughManager.for_queryset_class(TodoQuerySet)()` bit is a bit intimidating, especially to someone not familiar with django-model-utils. I'd probably take the time to write a `manager_with_queryset(queryset, manager=models.Manager)` function to make things more readable.
When you're trying to convince someone that the way they currently code isn't optimal, it really helps if the code you use to illustrate the current way looks like something your reader is likely to write. The strangely formatted filter chain at the start is thus really off-putting. Instead of adding three disclaimers asking the reader not to focus on the implementation details, why not just write it the way most people would:
todos = Todo.objects.filter(
owner=request.user,
is_done=False,
priority=1,
)
Re: Building a higher-level query API: the right way to use Django's ORM
#6I can't agree more. Always wrap your ORM with your own logic that makes sense to you and your application. You'll find code reuse will go way up, readability will go way up, testing is easier. You can actually test the model without bootstrapping the whole app. I've been preaching this in my SqlAlchemy talks and tutorials for years.
Re: Building a higher-level query API: the right way to use Django's ORM
#7The main point raised by the article is spot-on, and I'm ashamed to say that I had never recognised it as an issue before reading it. It applies even more strongly for more complex lookups (possibly involving Q objects), which I've always felt would find a better home in models.py than in views.py. And I too cringe every time I come across the django.db.models.manager source code. Some thoughts: The approach goes sli…
Agreed. I found the contrived chained filters at the beginning of the article off putting but the rest of the article was quite interesting.
Re: Building a higher-level query API: the right way to use Django's ORM
#8> Personally, I'm not completely convinced by the decorator-based idea. It obscures the details slightly, and feels a little "hacky".
The purpose of the decorator is, indeed, to obscure the implementation details in favour of more semantic code. But then again we're using an ORM which makes heavy use of metaprogramming to obscure the details of the database layer from us; I don't see how this is a bad thing.
Re: Building a higher-level query API: the right way to use Django's ORM
#9> Personally, I'm not completely convinced by the decorator-based idea. It obscures the details slightly, and feels a little "hacky". The purpose of the decorator is, indeed, to obscure the implementation details in favour of more semantic code. But then again we're using an ORM which makes heavy use of metaprogramming to obscure the details of the database layer from us; I don't see how this is a bad thing.
Yep, and my criticism of your suggested approach wasn't intended to be particularly strong by any means. I could definitely be sold on the idea. It just felt like a workaround to a problem that could probably be solved in a nicer way.
I think my main objection is that these query methods should conceptually be on the QuerySet, and so defining them on the Manager (the "wrong place") and magically copying them to the QuerySet (the "right place") feels somehow worse than the opposite.
I appreciate that you raised the discussion on the mailing list, as it highlights the fact that this is a common problem in big Django codebases. Even pulling something like PassThroughManager into core might work (perhaps with a nicer "manager_with_queryset" API, as suggested by Aramgutang).
Re: Building a higher-level query API: the right way to use Django's ORM
#10> Personally, I'm not completely convinced by the decorator-based idea. It obscures the details slightly, and feels a little "hacky". The purpose of the decorator is, indeed, to obscure the implementation details in favour of more semantic code. But then again we're using an ORM which makes heavy use of metaprogramming to obscure the details of the database layer from us; I don't see how this is a bad thing.
Yep, and my criticism of your suggested approach wasn't intended to be particularly strong by any means. I could definitely be sold on the idea. It just felt like a workaround to a problem that could probably be solved in a nicer way. I think my main objection is that these query methods should conceptually be on the QuerySet, and so defining them on the Manager (the "wrong place") and magically copying them to the Q…
But my problem is that most people wouldn't even think of subclassing QuerySet.
When we write methods that operate on collections of things, we typically use @classmethod. Without @classmethod, we'd have to write a custom metaclass (and instruct our class to use that) if we wanted even a single class method on a class. Multiple inheritance would break (or at least be difficult to reason about) when classes defined class methods, because there would have to be both an instance method resolution order and a class MRO. Fortunately Python's built-in `type` provides the descriptor protocol, which allows us to have class methods and instance methods and properties and all these other nice things without having to metaprogram or hack the interpreter.
All I'm asking for is a similar (if less ornate) interface for Django models, wherein the methods that operate on collections of things can be defined alongside the methods that operate on individual things, without requiring a knowledge of Manager/QuerySet internals.