Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tips for Building High-Quality Django Apps at Scale

blog.doordash.com

Post reply on HN
Print view View original

31–40 of 125 posts

Re: Tips for Building High-Quality Django Apps at Scale

#31

Post by Osmose »

tclancy wrote: post #10
cwisecarver wrote: post #4

I've got more than 5 years of experience with Django on a number of teams and at a couple of companies and in my experience almost everything in this article is completely incorrect. The only things I would agree with is the point about project layout and avoiding django's squashmigrations for the truncate the migrations table, delete the migrations, and create a new initial migration. Practically everything else in…

I've got 10 years' experience on projects small and large and I have to agree. The title talks about building at scale but the article doesn't stress that which makes some of the advice downright weird. >If you don't really understand the point of apps, ignore them and stick with a single app for your backend. You can still organize a growing codebase without using separate apps. This is where the article lost me. If…

> I never hit a point where designing the project in apps became a problem.

The article literally describes why this is a problem in the first place. Cross-app model relations are a PITA, and splitting "sections" of your site into separate apps often has you end up with cross-app relations.

The more general point here is that: The functional separation between Django apps and the logical separation between "parts" of your site often do not match up, and thus you should be careful about splitting up your site into multiple apps.

In my experience, this is absolutely true and happens often. The article recommends separating the parts of your site into modules and packages within a single app, which is a great idea and something the Django docs don't make obvious as a choice.

Re: Tips for Building High-Quality Django Apps at Scale

#32

Post by tobltobs »

tinix wrote: post #30
orf wrote: post #19

Earlier quoted context omitted.

I agree with all of your points about the article, but... > However, they should take it a step further, and just avoid Django in the first place. Django is a tool, and like most tools it has a use. I find Django indispensable for writing specific kinds of applications, and it's admin interface is by far the best thing since sliced bread for internal/backoffice applications. It's not perfect by any means but it's ama…

> I'm not sure why you put quotes around migrations as if it's some alien, obscure or weird feature. If you've never written a web application that needs migrations then you're not writing the kinds of applications (or indeed any 'serious' application) that would benefit from Django IMO. I have been programming for almost 15 years, and have never once needed to use migrations. I have worked on projects for multiple y…

You get your DB model 100% correct the first time, always? Even years later your DB model is able to accommodate all those always changing business requirements?

Re: Tips for Building High-Quality Django Apps at Scale

#33

Post by Alex3917 »

tinix wrote: post #27
Alex3917 wrote: post #24

Earlier quoted context omitted.

> it's admin interface is by far the best thing since sliced bread for internal/backoffice applications. By the way, some best practices here that I've discovered: - Use the Django admin for performing actions closely tied to specific models that don't involve any business logic. E.g. changing which one of a user's email addresses is their primary address. - For admin business logic that spans multiple tables, (e.g.…

> The benefit of having your admin business logic wrapped in REST endpoints is that you are writing and testing all your admin logic the same way as you write and test all your other endpoints. You write all your business logic in REST endpoints? That's insanity... Are you even doing RESTful things with all those endpoints? Have you even considered the impact of HTTP overhead? This is a thread about scalability, afte…

> replacing your data access layer in INTERNAL code with a RESTful endpoint is kind of insane

Nothing is being replaced. Each view has a service method that performs the business logic, so you can either call the endpoint or else call the service method. There is zero performance implication, and basically zero extra complexity.

Re: Tips for Building High-Quality Django Apps at Scale

#34

Post by Osmose »

randomsearch wrote: post #2

I'm so glad to see the mention of "app" directories here. I've only dabbled in Django development, but I've always thought the desire to divide things into apps didn't really make any sense. It felt like the developers of Django had said "well, intuitively, there must be some unit of reuse at this level" and then stuck the notion of apps in there in an attempt to provide that reuse. This seems to me to be somewhat un…

Where app directories are incredibly useful is in Django library development. They're a really convenient way to hook in models, static files, template helpers, etc. to an existing Django site, without forcing users to do a bunch of setup work when integrating your library. It's not a coincidence that most Django libraries have a very similar setup:

1. Install the package 2. Add the package to settings.INSTALLED_APPS (and middleware, etc). 3. Profit!

For an instance of a Django site, apps are less useful, and the article's recommendation to be more careful when splitting your site into multiple apps is great advice.

Re: Tips for Building High-Quality Django Apps at Scale

#35

Post by linkmotif »

Django is not meant to be run at scale. At least not horizontal scale. If you're not going to use the ORM, which prevents you from scaling horizontally, why are you using Django? Django can be scaled by getting large instances with lots of RAM, which is quite affordable these days. It can also be scaled with caching. But if you want horizontal scaling, don't use Django. That's not what it's for.

Re: Tips for Building High-Quality Django Apps at Scale

#37

Post by tinix »

tobltobs wrote: post #32
tinix wrote: post #30

Earlier quoted context omitted.

> I'm not sure why you put quotes around migrations as if it's some alien, obscure or weird feature. If you've never written a web application that needs migrations then you're not writing the kinds of applications (or indeed any 'serious' application) that would benefit from Django IMO. I have been programming for almost 15 years, and have never once needed to use migrations. I have worked on projects for multiple y…

You get your DB model 100% correct the first time, always? Even years later your DB model is able to accommodate all those always changing business requirements?

Yeah, pretty much.

Haven't had a problem yet where I needed automated migrations, and I have been doing this for years upon years upon years.

Like I said, if you're leaving heavily on migrations, you're doing something wrong.

Re: Tips for Building High-Quality Django Apps at Scale

#38

Post by t0mbstone »

cwisecarver wrote: post #4

I've got more than 5 years of experience with Django on a number of teams and at a couple of companies and in my experience almost everything in this article is completely incorrect. The only things I would agree with is the point about project layout and avoiding django's squashmigrations for the truncate the migrations table, delete the migrations, and create a new initial migration. Practically everything else in…

It's not really helpful when people come along and just say, "That's wrong", without explaining why...

Re: Tips for Building High-Quality Django Apps at Scale

#39

Post by asdfgadsfgasfdg »

tinix wrote: post #30
orf wrote: post #19

Earlier quoted context omitted.

I agree with all of your points about the article, but... > However, they should take it a step further, and just avoid Django in the first place. Django is a tool, and like most tools it has a use. I find Django indispensable for writing specific kinds of applications, and it's admin interface is by far the best thing since sliced bread for internal/backoffice applications. It's not perfect by any means but it's ama…

> I'm not sure why you put quotes around migrations as if it's some alien, obscure or weird feature. If you've never written a web application that needs migrations then you're not writing the kinds of applications (or indeed any 'serious' application) that would benefit from Django IMO. I have been programming for almost 15 years, and have never once needed to use migrations. I have worked on projects for multiple y…

How do you add columns to your db?

Do your requirements never change?

I am fully capable of doing everything done with migrations with SQL - but I always use migrations (even for custom sql outside of the ORM's capability). Why? because then we have a record of the schema state at any given moment (that is in sync with the application code at that point) and how it got to where it is now.

Re: Tips for Building High-Quality Django Apps at Scale

#40

Post by tclancy »

Osmose wrote: post #31
tclancy wrote: post #10

Earlier quoted context omitted.

I've got 10 years' experience on projects small and large and I have to agree. The title talks about building at scale but the article doesn't stress that which makes some of the advice downright weird. >If you don't really understand the point of apps, ignore them and stick with a single app for your backend. You can still organize a growing codebase without using separate apps. This is where the article lost me. If…

> I never hit a point where designing the project in apps became a problem. The article literally describes why this is a problem in the first place. Cross-app model relations are a PITA, and splitting "sections" of your site into separate apps often has you end up with cross-app relations. The more general point here is that: The functional separation between Django apps and the logical separation between "parts" of…

>Cross-app model relations are a PITA

A PITA how? They say they ran into migration issues due to the apps approach but it sounds like they ran into issues due to the sheer number of migrations happening across a bunch of developers. That sounds like a likely problem on big teams, but I don't think it's one best solved by not using the app approach and I don't think it's one whose underlying problem is ForeignKeys to models in other apps. Again, it would be nice to know if this was on MySQL or a different database as what finally caused me to move to Postgres 8 years or so ago was the heartburn of migration on MySQL. I think I've run into one or two migration knots since then and they were both due to me moving a little too fast.

And at scale, I would assume you aren't actually running the migrations but generating SQL from them and running that. Still could run into the same problems, but you could sort that out by hand when you do. Not the best answer, but from the article it feels like a more formalized/ strict approach to who gets to modify the database and when would be good.

>splitting "sections" of your site into separate apps often has you end up with cross-app relations.

It also encourages you to do some up-front data modeling which is a skill that gets rarer as ORMs get more common.

/old man yells at cloud

Post reply on HN