Live data from Hacker News

Nginx: a caching, thumbnailing, reverse proxying image server

charlesleifer.com

21–30 of 59 posts

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#21
post #14
post #7

I was wondering if an additional cookie based security token is possible with secure_link module. (show image only if session cookie is present) Apparently, it is possible, great! One of the google results: https://gist.github.com/hilbix/5921589

Secure_link module shouldn't be used for anything security related. I think it uses plain md5 and without even an hmac.

[deleted]

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#22
post #18

This is a cool little project showcasing some of NginX's lesser known modules in action. However, I have one nitpick. The caching server's main location is a bit of a cargo-cult unoptimization. They have specified: location ^/(.+)$ { When the equivalent: location / { Is shorter, clearer, and does not incur a regular expression capture on every request that is never used.

I (I'm not the author) shamefully do some cargo cult Nginx because I don't understand how the location matching really works. My ideal web server would have a configuration that resembles a decision procedure in a programming language, so I can have a mental model for what makes it choose one route over the others, and how settings are combined, etc. Somehow Nginx repeatedly makes me and my coworkers very confused—ar…

NginX has come a long way and so has the documentation: http://nginx.org/en/docs/

I highly recommend this document to start, which describes how NginX processes a request: http://nginx.org/en/docs/http/request_processing.html

For getting the mental model right, you have to understand that NginX configuration is declarative, like SQL or PROLOG. That means as opposed to an imperative or procedural language you might be more familiar with, in NginX you describe "what" you want to have handled, not "how" and NginX worries about the "how" under the hood, for the most part.

Of course like any abstraction it sometimes breaks and leaks, but this is practical software we're talking about. One specific big headache for people is when using "if" in NginX, which is a decidedly imperative construct! As such you do have to tread lightly with it; observe If Is Evil: https://www.nginx.com/resources/wiki/start/topics/depth/ifis...

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#23
post #10

I found the nginx native image library completely inadequate a few years ago, so I had to use the perl module to connect it to imagemagick libraries instead. Much higher quality results. Not sure if they have improved the native library but I doubt it has the flexibility. The important part is caching the results because of the expensive cpu time. ps. googling about the current state of this came up with an interesti…

I second this, for a recent e-commerce project we started using nginx's image_filter but the quality of the resized images was unacceptable.

We ended up using Thumbor, which was able to give us much better looking images (but at the cost of being extremely difficult to deploy on the CentOS 6.x servers we had at that time).

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#24
post #17

This is a cool little project showcasing some of NginX's lesser known modules in action. However, I have one nitpick. The caching server's main location is a bit of a cargo-cult unoptimization. They have specified: location ^/(.+)$ { When the equivalent: location / { Is shorter, clearer, and does not incur a regular expression capture on every request that is never used.

The difference is that the last one also matches requests to "/", while the regex ensures that there is at least a single character after the "/". That allows "/" to be used for other purposes, e.g. redirecting to the main page of the site, or displaying a nice error page. (However, in this case it simply shows the default Debian Nginx installation page: http://m.charlesleifer.com/ )

For that use case, I would recommend two locations like so:

    location = / {
      # Just whatever happens for "/" requests goes here
    }
    
    location / {
      # Every other requests ends up here
    }

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#26
We use a similar set-up at work, but we don't do the image resizing in nginx, rather we have a python backend that it proxies up to, that runs a tiny wsgi app that uses requests & pil to do the resizing and transforming. It uses proxy_cache_lock to dedupe requests to the backend.

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#27
We do something similar : when nginx doesn't find the file for a given format (width and height are in the url), it's a 404 and we use the 404 handler to proxy to a php app that does the resize/crop (and we have a list of the formats we use more often in the app so we store the generated thumb on the disk), the resize/crop operation is centered around coordinates that we determine with opencv (it detects faces and/or focus points) when the user uploads the picture.

We use varnish in front of nginx

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#28

Earlier quoted context omitted.

I stopped using varnish because, after all, my site is just a blog and Nginx is just fine for my needs. It has a single backend server, so there's no load-balancing. Basically, I was already using Nginx to serve static files, and my caching logic was so simple it was actually easier to move it into Nginx.

Perhaps better a better question is why you were using Varnish in the first place? I'm genuinely curious because I've never stood up Varnish before so maybe there's an anti-pattern / gotcha to learn from.

Well without some sort of caching, his site very well could fall over when being on the front page of something like HN. Putting Varnish in front a site, even with something simple like a 1 minute cache on everything, makes you pretty much immune from having "large amounts of traffic" being a real problem.

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#29
post #19

His setup is vulnerable to hash length extension attacks: secure_link_md5 "my awesome secret $uri"; This means anyone can extend the URI (eg. adding "/../../../some/other/file" at the end of the filename) and compute a valid key, and force the server to access an arbitrary image file on the backend. If the backend is a private server not publicly accessible, then this may be a security issue. This is why in the ngx_h…

Length extension attacks work for both append and prepend. I don't know if you're right, or whether or not either of them are vulnerable, but the correct solution to this problem is to use the HMAC[0] construction.

[0] https://en.wikipedia.org/wiki/Hash-based_message_authenticat...

Re: Nginx: a caching, thumbnailing, reverse proxying image server

#30
One of the great benefits of Google Appengine and Google Cloud Storage is that you get this for free. Merely changing the size param at the end of the url resizes the pic. And since this all happens on the edge cache, it never touches your app, and you don't get charged for bandwidth.

http://lh4.ggpht.com/sTP-3BqnirkHm40qfb496w85A1bf7BpeXthFJ92...

http://lh4.ggpht.com/sTP-3BqnirkHm40qfb496w85A1bf7BpeXthFJ92...

Post reply on HN