Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Django Styleguide

github.com

Post reply on HN
Print view View original

71–80 of 141 posts

Re: Django Styleguide

#71

Post by dnadler »

jbellew wrote: post #64
rowanseymour wrote: post #43

Earlier quoted context omitted.

Same here. If I was starting a non-Python project tomorrow I'd consider using Django to manage the database schema - especially now that we can describe custom indexes and constraints in migrations. Our project has gone thru over 1000 migrations so far tho we squash them down to 50 or so about once a year.

Thats interesting, if I was using TypeScript to access the data, how would I keep the schemas in sync between TS and python?

We do this manually along with a pydantic as a middleman between Django and TS. Works pretty well and is not a major inconvenience to keep things aligned.

Re: Django Styleguide

#72

Post by hbrn »

Spivak wrote: post #16
andybak wrote: post #5

> We use Celery for the following general cases: > > Communicating with 3rd party services (sending emails, notifications, etc.) > Offloading heavier computational tasks outside the HTTP cycle. > Periodic tasks (using Celery beat) Sigh. No mention of the trade-offs. There's simpler ways to do all these things. Celery is a big complex beast and it always pains me to see it as the default suggestion for simple tasks.

Because it’s mature, well integrated with Django and is a path so well-trodden there’s a McDonald’s on the way. Any possible use-case you can imagine for a job queue has been done in Celery and documented. Celery being complicated is also entirely on the operational side, once you actually have Celery using it from within your app is simple enough. Cron is awful for this use-case. You end up just inventing Celery but…

> Because it’s mature, well integrated with Django and is a path so well-trodden

> Cron is awful for this use-case. You end up just inventing Celery

Isn't it the other way around?

Crons are way more mature, well integrated (mgmt commands don't require 3rd party modules), and extremely well trodden. Crons are super predictable, have sensible defaults and plenty of tooling. Which you will have to reinvent with Celery.

There are some benefits to programmatic crons, but the downsides are huge.

Re: Django Styleguide

#74

Post by hbrn »

cjohnson318 wrote: post #66
hbrn wrote: post #63

Earlier quoted context omitted.

The short answer is yes. The long answer is if two entities are updating each other you might benefit from shifting all update responsibilities to one of them. Or even to a third entity that knows about both and keeps those two isolated from each other.

Woof. Thank you so much. I like the idea of a third party, like a mediator. Would that mediator be another app? Or should it be some module sitting in the project directory? (I'm not even sure Django would import something like that.)

> Would that mediator be another app?

Yes!

It is an app that might not even have any model classes. But it will contain business logic. And it will probably speak domain language, which is great.

If you're lucky, those two other apps will become pluggable. You will probably never replace them, but separation of concerns is always nice.

The downside of course is that you will have 3 apps instead of 1. That's the balance you have to maintain.

Re: Django Styleguide

#75

Post by rglullis »

well1912 wrote: post #65
rglullis wrote: post #59

Earlier quoted context omitted.

> Always keep your models slim. As simple as possible, but no simpler. Django models are meant to deal not just with the data, but also with business logic. If `course.has_finished` is a property of the course, why would you want to have a separate function outside of the class? > Do precomputation if you need the information in a template If the precomputation is only needed in a template, you can (should, IMO) use…

At my current company, we've had many teams over the years fail to make business logic in model methods work, and I think many other people have had similar results. The issues usually boil down to some combination of "business logic is too coupled to the data model" and "this method lives at an intersection of these two models and creates weird dependency problems". I now feel that Django puts you down a path for fa…

> The issues usually boil down to some combination of "business logic is too coupled to the data model" and "this method lives at an intersection of these two models and creates weird dependency problems".

Refactor is not a dirty word. The problems you are describing seem to be more of the nature of having too many things concentrated at specific model classes, and that this model should be decomposed, broken down. This is not a Django-specific issue.

Re: Django Styleguide

#76

Post by nickjj »

roflyear wrote: post #42
emptysea wrote: post #34

Earlier quoted context omitted.

I think as a concept a task queue with some workers makes a lot of sense but having used Celery in production, it leaves a lot to be desired We’ve run into various bugs and weird performance gotchas (like the workers prefetch jobs which is terrible if they aren’t all the same size)

Agreed. I have had more luck writing my own "jobs" engine that does stuff that I need that celery doesn't have anyway (retries, some record of execution, rate limiting). I'm sure if I really tried, I can get celery to be very reliable... but I never really got there. Also for whatever reason I have NEVER been able to get celery to be reliable for its scheduling/cron stuff. It just starts to fail. I use this library f…

What type of situations did you run into while using it?

I've been using Celery for a long time in production now. Nothing crazy and it's a fairly basic set up of "execute job, thanks!" along with a beat server. Over the last 6-7 years an off the top of my head guess would be that there's been at least 5 million jobs processed. It's not huge volume when measured over years but it's been stable.

One server was running for 6 months without being updated. That's a Celery worker process running for ~180 days uninterrupted. It served hundreds of thousands of jobs without maintenance. A lot of these were pretty beefy tasks too like performing HTTP requests that got 400mb XML responses and then parsed them. I didn't even have things like `worker_max_tasks_per_child` set either.

Re: Django Styleguide

#77

Post by anileated »

jbellew wrote: post #64
rowanseymour wrote: post #43

Earlier quoted context omitted.

Same here. If I was starting a non-Python project tomorrow I'd consider using Django to manage the database schema - especially now that we can describe custom indexes and constraints in migrations. Our project has gone thru over 1000 migrations so far tho we squash them down to 50 or so about once a year.

Thats interesting, if I was using TypeScript to access the data, how would I keep the schemas in sync between TS and python?

Can introspect and export Django models to JSON schema or similar, then in TS read it and use compiler low-level API to generate types. There may be libraries for either stage…

Re: Django Styleguide

#78

Post by cjohnson318 »

hbrn wrote: post #74
cjohnson318 wrote: post #66

Earlier quoted context omitted.

Woof. Thank you so much. I like the idea of a third party, like a mediator. Would that mediator be another app? Or should it be some module sitting in the project directory? (I'm not even sure Django would import something like that.)

> Would that mediator be another app? Yes! It is an app that might not even have any model classes. But it will contain business logic. And it will probably speak domain language, which is great. If you're lucky, those two other apps will become pluggable . You will probably never replace them, but separation of concerns is always nice. The downside of course is that you will have 3 apps instead of 1. That's the bala…

So, the way I understand it, job/services.py and load/services.py depend on mediator/mediator.py which depends on job/models.py and load/models.py, instead of job/services.py ultimately using load/services.py, and vice versa.

Thanks so much!

Re: Django Styleguide

#79

Post by zenith035 »

Don't create apps just for the sake of project structure even when models from multiple apps are closely related.

Moving models from one app to another is doable but it is a pain. It's even worse if you are relying on GFKs.

Re: Django Styleguide

#80

Post by theptip »

randlet wrote: post #37
olau wrote: post #17

I think there are too many concepts in this, but rather than being negative, here are some tips: Always keep your models slim. Don't stuff template related stuff in there. You need to look at those models often, so compact is a win. course_has_finished(course) is not much longer than course.has_finished(), and will allow you to expand the functionality as time goes on. Do precomputation if you need the information in…

"Also be wary of the formerly South, now built-in migrations stuff. It's built around a fragile model (perfect history representation), so has lots of foot guns." My experience has been opposite so I'd be interested in hearing your experiences if you are willing. I have been using the built in migrations since day 1 on a medium sized Django project with 350+ migrations and migration issues for our project have been e…

Diamond-dependencies are hard to work with, and so we forbade them. Specifically, you can't revert an individual migration, you just specify your desired "target" to roll back to, and therefore you can't unapply a single branch of a diamond dependency. This means if your second branch of a diamond dependency breaks the DB, but your app depends on the first branch, you're SOL and are now manually running SQL to fix your production DB. (Can you tell I'm speaking from experience here? :) )

Your migration code uses the model classes, but the migrations are using a rehydrated version of the model that doesn't include any methods; another footgun. Basically you need to copy in any logic that you're using into your migration file, or else that migration's logic will change as your code is refactored. You might naively think that because `model.do_the_thing()` works now, the migration is somehow pickling a snapshot of your model class. It's not.

Because of the above, you should really squash migrations frequently, but it's a big pain to do so -- particularly if you have dependency cycles between apps. ("Just Say No to Apps" is my default advice for new Django developers. If you have a real need for making a bit of library code sharable between multiple different Django projects then you know enough to break this rule. Within a single monolithic project, apps just cause pain.) Squashing migrations quickly gets to some extremely gnarly and hard-to-work-with logic in the docs.

Moving models between apps isn't supported by the migration machinery; it's an involved and dangerous process. One idea here that might save Apps is if you manually remove the App Prefix from your "owned" / internal apps; if I have Customer and Ledger apps, I don't really need to namespace the tables; `user`, `information`, `ledger_entries` are fine table names instead of `customer_user`, `customer_information`, `ledger_ledger_entries`, a normal DB admin would not namespace the table names. You neeed the app namespacing to make it safe to use someone else's apps, but I think namespacing for your own apps inside a single repo is harmful.

I find the migration framework to be worth using, but it's definitely got some sharp edges.

Post reply on HN