Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some notes on starting to use Django

jvns.ca

Post reply on HN
Print view View original

71–80 of 145 posts

Re: Some notes on starting to use Django

#71

Post by ndr »

senko wrote: post #58
ndr wrote: post #57

Earlier quoted context omitted.

I found it very lacking in how to do CD with no downtime. It requires a particular dance if you ever want to add/delete a field and make sure both new-code and old-code work with both new-schema and old-schema. The workaround I found was to run tests with new-schema+old-code in CI when I have schema changes, and then `makemigrations` before deploying new-code. Are there better patterns beyond "oh you can just be care…

You can do three stage: 1. Make a schema migration that will work both with old and new code 2. Make a code change 3. Clean up schema migration Example: deleting a field: 1. Schema migration to make the column optional 2. Remove the field in the code 3. Schema migration to remove the column Yes, it's more complex than creating one schema migration, but that's the price you pay for zero-downtime. If you can relax that…

I'm doing all of these and None of it works out of the box.

Adding a field needs a default_db, otherwise old-code fails to `INSERT`. You need to audit all the `create`-like calls otherwise.

Deleting similarly will make old-code fail all `SELECT`s.

For deletion I need a special 3-step dance with managed=False for one deploy. And for all of these I need to run old-tests on new-schema to see if there's some usage any member of our team missed.

Re: Some notes on starting to use Django

#72

Post by ndr »

m000 wrote: post #67
ndr wrote: post #57

Earlier quoted context omitted.

I found it very lacking in how to do CD with no downtime. It requires a particular dance if you ever want to add/delete a field and make sure both new-code and old-code work with both new-schema and old-schema. The workaround I found was to run tests with new-schema+old-code in CI when I have schema changes, and then `makemigrations` before deploying new-code. Are there better patterns beyond "oh you can just be care…

Deploying on Kubernetes using Helm solves a lot of these cases: Migrations are run at the init stage of the pods. If successful, pods of the new version are started one by one, while the pods of the new version are shutdown. For a short period, you have pods of both versions running. When you add new stuff or make benign modifications to the schema (e.g. add an index somewhere), you won't notice a thing. If the intro…

Without care new-schema will make old-code fail user requests, that is not zero downtime.

Re: Some notes on starting to use Django

#73

Post by senko »

Some more Django recommendations from Frank Wiles of Revsys (Django consultancy): https://frankwiles.com/questions/starting-django-projects/

I'll add a few of my own:

* Set up the project using uv

* I name the django project "project"; so settings are project/settings.py, main urls are project/urls.py, etc

* I always define a custom Django user model even if I don't need anything extra yet; easier to expand later

* settings.py actually conflates project config (Django apps, middleware, etc) and instance/environment config (Database access, storages, email, auth...); I hardcode the project config (since that doesn't change between environemnts) and use python-dotenv to pull settings from environment / .env; I document all such configurable vars in .env.example, and the defaults are sane for local/dev setup (such as DEBUG=true, SQLIte database, ALLOWED_HOSTS=*, and a randomly-generated SECRET_KEY); oh and I use dj-database-url to use DATABASE_URL (defaults to sqlite:///sqlite.db)

* I immediately set up up ruff, ty, pytest, pre-commit hook and GH workflow to run ruff/ty/pytest

Previously I had elaborate scaffolding/skeleton templates, or nowadays a small shell script and I tell Claude to adapt settings.py as per above instructions :)

Re: Some notes on starting to use Django

#74

Post by bodge5000 »

senko wrote: post #73

Some more Django recommendations from Frank Wiles of Revsys (Django consultancy): https://frankwiles.com/questions/starting-django-projects/ I'll add a few of my own: * Set up the project using uv * I name the django project "project"; so settings are project/settings.py, main urls are project/urls.py, etc * I always define a custom Django user model even if I don't need anything extra yet; easier to expand later * s…

I'll add one; Add shell_plus. It makes the django shell so much nicer to use, especially on larger projects (mostly because it auto-imports all your models). IIRC, it involves adding ipython and django_extensions as a dependency, and then adding django-extensions (annoyingly, note that the underscore changes to a dash, this trips me up everytime I add it) to your installed apps.

Saying that, I'm sure django-extensions does a lot more than shell_plus but I've never actually explored what those extra features are, so think I'll do that now

Edit: Turns out you can use bpython, ptpython or none at all with shell_plus, so good to know if you prefer any of them to ipython

Re: Some notes on starting to use Django

#75

Post by bodge5000 »

tmarice wrote: post #48

After working with Django for 8 years, I find it hard to move on to anything else. It's just the right amount of magic, and just the right amount of flexibility, and it's just such a joy to work with. Re: Django is OK for simple CRUD, but falls apart on anything complex - this is just untrue. I have worked in a company with a $500M valuation that is backed by a Django monolith. Reporting, recommender systems, file in…

> Re: Django is OK for simple CRUD, but falls apart on anything complex

Maybe my experience of working with Django on complex applications has coloured my view on it a bit, but I always think the opposite; it seems overkill for simple CRUD, even if I love using it

Re: Some notes on starting to use Django

#76

Post by wink »

sriram_malhar wrote: post #25
simonw wrote: post #17

Earlier quoted context omitted.

This is the main reason I'm extremely disciplined about making sure all of my personal projects have automated tests (configure to run in CI) and decent documentation. It makes it so much easier to pick them up again in the future when enough time has passed that I've forgotten almost everything about them.

I'm finding that in this build fast and break things culture, it is hard to revisit a project that is more than 3 years old. I have a couple of android projects that are four years old. I have the architecture documented, my notes (to self) about some important details that I thought I was liable to forget, a raft of tests. Now I can't even get it to load inside the new version of Android Studio or to build it. There…

I have relatively good experience with both Rust and Go here. It still works and maybe you need update 2-3 dependencies that released an incompatible version, but it's not all completely falling apart just because you went on a vacation (looking at you npm)

Re: Some notes on starting to use Django

#77

Post by tmarice »

ndr wrote: post #57
jgavris wrote: post #10

The Django ORM / migrations are still basically unmatched in happiness factor.

I found it very lacking in how to do CD with no downtime. It requires a particular dance if you ever want to add/delete a field and make sure both new-code and old-code work with both new-schema and old-schema. The workaround I found was to run tests with new-schema+old-code in CI when I have schema changes, and then `makemigrations` before deploying new-code. Are there better patterns beyond "oh you can just be care…

This is not specific to Django, but to any project using a database. Here's a list of a couple quite useful resources I used when we had to address this:

* https://github.com/tbicr/django-pg-zero-downtime-migrations

* https://docs.gitlab.com/development/migration_style_guide/

* https://pankrat.github.io/2015/django-migrations-without-dow...

* https://www.caktusgroup.com/blog/2021/05/25/django-migration...

* https://openedx.atlassian.net/wiki/spaces/AC/pages/23003228/...

Generally it's also advisable to set a statement timeout for migrations otherwise you can end up with unintended downtime -- ALTER TABLE operations very often require ACCESS EXCLUSIVE lock, and if you're migrating a table that already has an e.g. very long SELECT operation from a background task on it, all other SELECTs will queue up behind the migration and cause request timeouts.

There are some cases you can work around this limitation by manually composing operations that require less strict locks, but in our case, it was much simpler to just make sure all Celery workers were stopped during migrations.

Re: Some notes on starting to use Django

#78

Post by stuaxo »

Nice.

As a mostly-django-dev for the last 15 years, who's been exposed to FastAPI and various ORMs again recently, I should get round to write a doc about some Django bits.

Django is pretty nice, the changes between versions are small and can be managed by a human.

Part of the reason that you can have the big ecosystem is that there is a central place to register settings and INSTALLED_APPS, middleware etc.

That enables addons to bring their own templates and migrations.

There is a central place a bit further up in manage.py and that enables you to bring commandline extras to Django (and many of the things you install will have them).

Coming to a FastAPI app with alembic and finding a lot of that is build-it-yourself (and easily break it) is a bit of a shock.

The Django ORM at first can seem a little alien "why isn't this sqlalchemy" was my reaction a long time ago, but the API is actually pretty pragmatic and allows easy extension.

You can build up some pretty complex queries, and keep them optimised using the Django-Debug-Toolbar and its query viewer.

The ORM, Templates and other parts of Django pre-date many newer standards which is why they have their own versions. As a Django dev I only just discovered the rest of the world has invented testcontainers, and databases as a solution for a problem Django solved years ago with it's test database support.

I quite like the traditional setup where you have settings/common.py and then settings that extend that - e.g local.puy production.py

If you ever need a CMS in your Django project I strongly recommend Wagtail, it came after the initially most popular django-cms and learned a lot of lessons - feeling much more like a part of Django.

It has the same feeling of being productive as Django does when you first use it.

Re: Some notes on starting to use Django

#79

Post by hansonkd »

IceDane wrote: post #20
hansonkd wrote: post #11

Earlier quoted context omitted.

Its crazy to me after all these years that django-like migrations aren't in every language. On the one hand they seem so straightforward and powerful, but there must be some underlying complexities of having it autogenerate migrations. Its always a surprise when i went to Elixir or Rust and the migration story was more complicated and manual compared to just changing a model, generating a migration and committing. In…

There is no way to autogenerate migrations that work in all cases. There are lots of things out there that can generate migrations that work for most simple cases.

They don't need to work in every case. For the past `~15 years 100% of the autogenerated migrations to generating tables, columns or column names I have made just work. and i have made thousands of migrations at this point.

The only thing to manually migrate are data migrations from one schema to the other.

Re: Some notes on starting to use Django

#80

Post by hansonkd »

wiredfool wrote: post #56
cuu508 wrote: post #53

Earlier quoted context omitted.

Can you give an example how this would happen?

Ok, from memory -- There's a pre, do and post phase for the migrations. When you run a single migration, it's: pre, do, post. When you run 2 migrations, it's: pre [1,2], do: [1,2], post: [1,2]. So, if you have a migration that depends on a previous migration's post phase, then it will fail if it is run in a batch with the previous migration. When I've run into this is with data migrations, or if you're adding/assigin…

Does that affect the autogenerated migrations at all? Teh only time I ran into that issue as if I generated a table, created a data migration and then it failed because the table was created same transaction. Never had a problem with autogenerated migrations.
Post reply on HN