Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to serve Django Statics (and not go insane)

blog.sendhub.com

Post reply on HN
Print view View original

11–20 of 34 posts

Re: How to serve Django Statics (and not go insane)

#11

Post by rdpfeffer »

ashray wrote: post #7
rdpfeffer wrote: post #4

Earlier quoted context omitted.

Also, I can't tell you how much I hate that "cloudflare" sounds so much like "cloudfront". I've been in at least 6 convos where people were mixing one for the other. Esp, since they solve some of the same problems.

Actually that's exactly what happened to me! :P I thought oh so cloudflare can fix this, eh. I should just reverse proxy my stuff through cloudflare. Then I realized, oh they said cloudfront. So you could still technically keep your stuff in your S3 bucket, though you'd end up paying more for S3 originating transfer as well. In this case, cloudFRONT (the non amazon one!!) may be the better choice. Hate that they soun…

Hilarious, I wonder how much they gain from devs getting mixed up on recommendations. Probably better for Cloudflare if you ask me. Personally, I don't think it makes sense to use CloudFront if you are already using cloudflare. Its easier to use if your just talking web statics.

Re: How to serve Django Statics (and not go insane)

#12

Post by cdoxsey »

ashray wrote: post #10
rdpfeffer wrote: post #6

Earlier quoted context omitted.

We got it working but there we're a TON of drawbacks that made it not worth the effort. Here are the top three. 1. The config was overly complex and not flexible enough for our needs. 2. It added extra, and unnecessary deploy steps which slowed down our deployments. 3. Configuring GZIP on S3 and Cloudfront is prohibitively complex. CloudFlare on the other hand, solves many of these problems for us.

Hang on! So your article says "CDN to cache them (which CloudFront supports)" but you're actually using CloudFLARE ? :O We're all so confused by this! Haha :D

Both CloudFlare and CloudFront can do this.

http://aws.typepad.com/aws/2010/11/amazon-cloudfront-support...

Re: How to serve Django Statics (and not go insane)

#13

Post by ashray »

cdoxsey wrote: post #12
ashray wrote: post #10

Earlier quoted context omitted.

Hang on! So your article says "CDN to cache them (which CloudFront supports)" but you're actually using CloudFLARE ? :O We're all so confused by this! Haha :D

Both CloudFlare and CloudFront can do this. http://aws.typepad.com/aws/2010/11/amazon-cloudfront-support...

Actually, on looking deeper it appears that cloudfront does NOT support dynamic gzipping.

So unfortunately, cloudflare would be your only option.

Also, google's pagespeed service (in beta) is also an option. Also free, so that's good :D

EDIT: Okay, wrong again. Appears that cloudfront will serve dynamically gzipped content if your origin server responds with gzipped content to the Accept-Encoding header. So origin server = S3 is not a good idea, but origin through your own web server should work fine.

Re: How to serve Django Statics (and not go insane)

#14

Post by rdpfeffer »

jaytaylor wrote: post #9
rdpfeffer wrote: post #6

Earlier quoted context omitted.

We got it working but there we're a TON of drawbacks that made it not worth the effort. Here are the top three. 1. The config was overly complex and not flexible enough for our needs. 2. It added extra, and unnecessary deploy steps which slowed down our deployments. 3. Configuring GZIP on S3 and Cloudfront is prohibitively complex. CloudFlare on the other hand, solves many of these problems for us.

In the article you referenced CloudFront, but now you're saying CloudFlare.. so which is it?

The article has been modified to clarify which service we are using as a reverse proxy and why.

Re: How to serve Django Statics (and not go insane)

#16

Post by wahnfrieden »

jaytaylor wrote: post #5

You know, I've always found django-compressor to be adequate for my django statics needs. Was there a real need to reinvent the wheel in this case? Were you simply unable to configure django-compressor (or any other pre-existing django statics library) properly?

A little off topic, but a fault with django-compressor is that if you ever want to use jinja2, it only supports generating the compressed versions on the fly rather than via manage.py - I made a nasty hack that lets us do that but it wasn't easy.

An even worse problem is that you won't be able to transition to using async dependency loading for your JavaScript since there's no way to get the filenames of each compressed bundle that you'll need to pass to whatever lib you're using.

These problems don't apply much when you're just prototyping something, but it turned out to be an awful hole we dug ourselves into when we ran into severe performance bottlenecks with Django's template rendering and serving all our JS on page load.

Re: How to serve Django Statics (and not go insane)

#18

Post by rdpfeffer »

wahnfrieden wrote: post #16
jaytaylor wrote: post #5

You know, I've always found django-compressor to be adequate for my django statics needs. Was there a real need to reinvent the wheel in this case? Were you simply unable to configure django-compressor (or any other pre-existing django statics library) properly?

A little off topic, but a fault with django-compressor is that if you ever want to use jinja2, it only supports generating the compressed versions on the fly rather than via manage.py - I made a nasty hack that lets us do that but it wasn't easy. An even worse problem is that you won't be able to transition to using async dependency loading for your JavaScript since there's no way to get the filenames of each compres…

preach! Haha. Turns out there's more than one way to hang yourself with this stuff. The problems we ran into were similar.

Re: How to serve Django Statics (and not go insane)

#19

Post by jaytaylor »

mikeknoop wrote: post #15

I've gotten https://github.com/cyberdelia/django-pipeline working recently. It's a young project but it's supporting a heavy frontend asset pipeline for us now and working great.

I've tried django-pipeline and it seemed great when I started, but ultimately I had to abandon it because I discovered it would've taken an inordinate amount of work to get it compressing the CSS/JS assets efficiently. Out of the box (just following the documentations suggestions) the django-pipeline compressed JS was 5-10X the size that I get after django-compressor is done with it (and with compressor I didn't have to do any tuning or optimization beyond basic configuration).

At any rate, I'm happy that you were able to get it working, Mike! That is really neat. Do you think you'll write a blog post about how you did it?

Re: How to serve Django Statics (and not go insane)

#20

Post by IgorPartola »

Anyone know what proxies still don't like foo.js?v=1 style URLs? I thought we were past this.

Also, a static build process (a la require.js) solves a lot of these for you in a way that's not specific to Django or any other framework. You also don't end up bloating your web app code with the concern for how static files are to be processed, which I personally like a lot.

Another way to go is to use Google's mod_pagespeed [1], which once again is not framework-specific.

Lastly, you can try another trick where you pre-generate .gz versions of all the files too, to really speed things up. It's nice not to have to do things on the fly and web servers like nginx can take quite a lot of traffic serving static files, so you can hold off on going the CDN route, unless of course geography matters more to you than offloading server resources.

[1] https://developers.google.com/speed/pagespeed/mod

Post reply on HN