Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenRoss – fast, scalable, on-demand image resizer

developers.lyst.com

Post reply on HN
Print view View original

11–20 of 34 posts

Re: OpenRoss – fast, scalable, on-demand image resizer

#11

Post by simonw »

mantraxC wrote: post #6
dalke wrote: post #3

Earlier quoted context omitted.

Isn't that what they started with, and decided against? "In our infancy, we saved all product images with 10 preset sizes, and then rendered the image which was nearest in size to what we required. As we grew, this solution became unwieldy for the levels of traffic we were experiencing and nor was it appropriate for our mobile app."

They say that, but there's one giant missing ", because..." in that paragraph. They never explain why they render in 10 sizes (why don't they know which sizes they need), and why is loading one of those sizes on a mobile app not feasible. It's stated like that and left hanging in the air. Either their use case is very weird, or they're deliberately vague for some reason only they know about. Also to clarify why 10 si…

"This screams "we didn't think this through much"."

That's actually quite rude.

Re: OpenRoss – fast, scalable, on-demand image resizer

#12

Post by simonw »

I've worked with systems that pre-size images on creation, and I'd much rather work with a system like this that resizes and caches on demand.

You never know when you're going to need a new size (the introduction of retina images a few years ago doubled the dimensions of images you need to serve for example, and your design team is likely to come up with new size requirements occasionally as well) and backfilling to resize millions of images is a big pain and costs a lot in terms of storage.

Re: OpenRoss – fast, scalable, on-demand image resizer

#14

Post by m0th87 »

I'm a fan of pilbox [1]. It's "fast" (builds off of OpenCV + Tornado), easy to understand and does much more than just image resizing. Also, it should be simpler to deploy than OpenRoss - since it doesn't bake in nginx, you can deploy it to Heroku (which is what we do), or behind Amazon's ELB.

We use it on a fairly large production site, and it works fine. If you throw it behind CloudFront or some other CDN that supports cache busting, it's plenty fast.

1: https://github.com/agschwender/pilbox

Re: OpenRoss – fast, scalable, on-demand image resizer

#15

Post by johne20 »

mantraxC wrote: post #6
dalke wrote: post #3

Earlier quoted context omitted.

Isn't that what they started with, and decided against? "In our infancy, we saved all product images with 10 preset sizes, and then rendered the image which was nearest in size to what we required. As we grew, this solution became unwieldy for the levels of traffic we were experiencing and nor was it appropriate for our mobile app."

They say that, but there's one giant missing ", because..." in that paragraph. They never explain why they render in 10 sizes (why don't they know which sizes they need), and why is loading one of those sizes on a mobile app not feasible. It's stated like that and left hanging in the air. Either their use case is very weird, or they're deliberately vague for some reason only they know about. Also to clarify why 10 si…

I have had this exact problem/need overtime on several projects, and it doesn't mean "we didn't think this through much". Products change/grow and there becomes a need for tens of different images. First it starts with a thumbnail or two, then composite images, then, etc. Not to mention you need 2x images for retina devices.

Bottom line, overtime you tend to understand that there is great flexibility in this type of architecture.

Re: OpenRoss – fast, scalable, on-demand image resizer

#16

Post by simonw »

mantraxC wrote: post #6
dalke wrote: post #3

Earlier quoted context omitted.

Isn't that what they started with, and decided against? "In our infancy, we saved all product images with 10 preset sizes, and then rendered the image which was nearest in size to what we required. As we grew, this solution became unwieldy for the levels of traffic we were experiencing and nor was it appropriate for our mobile app."

They say that, but there's one giant missing ", because..." in that paragraph. They never explain why they render in 10 sizes (why don't they know which sizes they need), and why is loading one of those sizes on a mobile app not feasible. It's stated like that and left hanging in the air. Either their use case is very weird, or they're deliberately vague for some reason only they know about. Also to clarify why 10 si…

"Even the most crippled client-side technique can do the cropping and whitespace for you (yeah, even html)."

Facebook, Twitter and Google Plus all have a feature where a URL gets expanded out to a "rich preview" (Twitter Product Cards for example are particularly relevant to a site like Lyst: https://dev.twitter.com/docs/cards/types/product-card )

These all work best with an image that has been pre-cropped and resized.

As an added bonus, a new one of these emerges every now and then - with a different size requirement.

Re: OpenRoss – fast, scalable, on-demand image resizer

#17

Post by heynemann »

There's also thumbor (https://github.com/thumbor/thumbor). It's a very mature implementation of this type of server and has been very battle-tested (https://github.com/thumbor/thumbor/wiki/Who%27s-using-it).

At globo.com we have near a billion images (we are a big portal). Can you imagine pre-generating that many images every time a new format gets added?

We serve everything with thumbor with a Varnish cache in front of it and we're very happy with it. It has enabled our designers to work with any image size they can think of.

If you guys need more info, please check thumbor's docs: https://github.com/thumbor/thumbor/wiki/

Re: OpenRoss – fast, scalable, on-demand image resizer

#18

Post by hltbra »

heynemann wrote: post #17

There's also thumbor ( https://github.com/thumbor/thumbor ). It's a very mature implementation of this type of server and has been very battle-tested ( https://github.com/thumbor/thumbor/wiki/Who%27s-using-it ). At globo.com we have near a billion images (we are a big portal). Can you imagine pre-generating that many images every time a new format gets added? We serve everything with thumbor with a Varnish cache in f…

We use Thumbor at Yipit, and we are very happy with it. One thing that was not mentioned was extensibility... With thumbor it's easy to create new plugins and filters to extend your installation set of features.

We had detailed how we scaled thumbor at Yipit last year: http://tech.yipit.com/2013/01/03/how-yipit-scales-thumbnaili... The blog post doesn't mention S3, but we have a storage plugin that reads from and writes to S3.

Re: OpenRoss – fast, scalable, on-demand image resizer

#19

Post by bluejade »

At populr.me, we use a variation on the architecture described here. One difference is that we cache resized images to S3, rather than to on-disk cache. This enables all servers to share the cache. Otherwise, when a new server is brought online, it doesn't benefit from the cache, so for a time, every request it receives incurs the most costly path of source image retrieval and resizing.

An added benefit to caching to S3 is that since S3 won't run out of space, we can cache rendered images for longer (we use S3 lifecycle to keep cache expiration simple). The scaled images tend to be smaller than the source images, so the retrieval from S3 is pretty fast. Over the past week, retrieving scaled images from S3 has cost ~46ms versus ~84ms for the larger source images.

Post reply on HN