Tips for Building High-Quality Django Apps at Scale

Yes, there's a whole universe of them.

Here's a good place to start looking for things to help make development easier:

https://djangopackages.org/

A good rule of thumb is that a Django "app" should be (1) standalone and (2) reusable.

If you were to write a Django project to automate some complicated internal process at your particular company, it's likely that you'd break up the code into separate Python modules to help give it an easy to understand structure. But it's unlikely that any other company would necessarily benefit from any of those separate modules by themselves.

However, let's say you come up with a clever idea about how to log errors in your project. Now, that's the sort of thing that other people might want to use in their own projects. If you were to restructure your code a bit and make it a bit more abstract then you could create a separate "app" that could be dropped into other Django projects.

Let's say you're following the approach of breaking down your project into separate apps, so you have an app called user_accounts. This app would be a folder containing files like:

  views.py, services.py, models.py, test_views.py

So in views.py you'd have a User class, with:

  A POST method that calls services.create_user(username, email_address, password)

  A GET method that calls services.get_user_profile(request.user)

  A PUT method that calls services.update_user_profile(x, y, z)

  A DELETE method that calls services.inactivate_user(request.user)

The return value of each of these views can just be whatever services.get_user_profile(request.user) returns, rendered into JSON.

Then each of the services performs whatever business logic it needs to, preferably directly in the method. But if it would be more readable split into multiple methods, then you can create some private helper methods in the services file prefixed with an underscore. You can also have a separate folder somewhere for utility functions meant to be reused across the app, e.g. get_user_emails(request.user, is_active=True, is_verified=True)

Basically though each view sanitizes the data, e.g. strips XSS out of strings, makes sure booleans are actually booleans, etc.

Then each service first does field-level validation with serializers, e.g. ensuring that usernames meet the appropriate requirements for usernames. Next if there is other business logic validation that needs to happen, it happens, e.g. making sure that only users with verified email addresses can perform certain actions.

After that you perform the actual business logic, e.g. transforming any data. Then you perform your CRUD operation, e.g. creating a user model. And lastly you return something, e.g. returning the user model.

Each endpoint and service method can be written pretty much following this pattern, which makes the codebase super readable because once you understand one endpoint you understand all of them. And the service methods are the reusable component of the architecture, so e.g. if you want the ability for admins to create users, then they are created with the exact same service method. (But called from your admin endpoints/services.)

> So where is your data model enforced if not in your tables?

It's enforced...

...in the models.

:P

> Just out of curiosity, have you had to maintain/modify any of these apps over these years and years?

Absolutely.

> If so, without any schema changes?

Usually, to be honest, yeah... Core, basal units of any given business process usually don't change. Only how they are abstracted, and that abstraction lives in code, not the database.

With a proper storage architecture it is very very very rare to need to change old data structures. It most definitely doesn't happen often enough to necessitate the use, overhead, and complexity of a baked-in migrations library.

If you think "adding a table" or "adding a field" is a "schema change" then that is where the problem lies... I think. Extending a schema should not require a migration library. CHANGING a schema, as in, destroying old data, and making new data, could surely use the help of a migration library, but, generally, if you're doing that often, you're doing something very wrong, and if you're not doing that often, then you definitely don't need a baked in migration library.

Ultimately, if things are so bad that you need a migrations library, you're better off fixing that shit at a low level, and starting over from scratch if necessary. Sometimes, instead of creating years and years of tech debt and burning resources on working with a clusterfuck, you just need to take what you can from that clusterfuck, and do it right. Then you only have a single migration, to go from old clusterfuck data to new extensible data structure.

Chances are, if you have old fucked up horrific data, it stays that way, and then some person or team comes along and writes a nice clean API on top of it. And then people have to maintain that horrific thing, and it's only horrific because it's interfacing with shit data layers in the first place.

Don't just abstract bad data architecture away into a service layer, is my point... Do it right.

Extensible is the key word here, but with an emphasis on decoupling.

If you have some dynamical data structure that is truly changing, structurally, often, and isn't simply being appended to or extended, then it probably should not be stored in a traditional SQL database in any way that necessitates schema changes. If you can't figure out how to represent data in an effective, decoupled, and extensible manner, then you're screwed from the get-go. Migrations library won't help you.