Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Discord Resizes 150M Images Every Day with Go and C++

blog.discordapp.com

Post reply on HN
Print view View original

71–80 of 157 posts

Re: How Discord Resizes 150M Images Every Day with Go and C++

#71

Post by Xeoncross »

Lutin wrote: post #49
Xeoncross wrote: post #8

Why don't more companies resize images client-side first using and then save the server some work by only asking it to verify the result by - resizing to the same size - removing metadata This results in much faster transfer (10x less bandwidth used often for mobile uploads) and reduces server load by "farming" out the work to the clients. https://developer.mozilla.org/en-US/docs/Web/API/CanvasRende... # Edit: On Kee…

This is for proxying images that users link in chat, not for when users upload images to the service. It doesn't make sense to talk about doing this resize on the client, as the client doesn't have the image.

Yes, the use-case of proxying images is a different mater. I was talking about client uploads since so many companies seem determined to waste my bandwidth and time uploading without resizing first.

Re: How Discord Resizes 150M Images Every Day with Go and C++

#72

Post by [deleted] »

fleitz wrote: post #59
kylehotchkiss wrote: post #43

I wish Cloudfront supported resize parameters so we wouldn't have to keep buildings these or paying a lot for Imgix.

How much would you pay for an image resizing service? I'd been thinking for a while of putting a fleet of autoscaled thumbor boxes behind cloudfront and making a billing API for it.

[deleted]

Re: How Discord Resizes 150M Images Every Day with Go and C++

#73

Post by indiandennis »

SaltyBackendGuy wrote: post #64
AndrewStephens wrote: post #16

Earlier quoted context omitted.

A few reasons: 1) Although the site serves up images at 1024 pixels (or whatever) today, in the future they may want larger images. When everyone is rocking 10K monitors and 6K phone displays, those small images are going to look pretty bad. 2) The original image has some metadata that they want to keep (geolocation, etc). 3) They think they can do a better and more consistent job resizing than the various browsers,…

> 2) The original image has some metadata that they want to keep (geolocation, etc). Isn't exif data something you should strip out?

Discord makes most of their revenue from selling user data so they probably want to keep as much as possible.

Re: How Discord Resizes 150M Images Every Day with Go and C++

#74

Post by manigandham »

Nice, but why? https://cloudinary.com, https://www.imgix.com, or https://www.filestack.com already exist and are well worth it for 99% of apps. Even at scale, it really doesn't cost that much to have someone else do it. You can use a thin proxy through your existing CDN if you want to save on their bandwidth fees.

Also http://thumbor.org and https://imageresizing.net if you want a library to host yourself which are already very fast and well tested. Put them in a docker container on a kubernetes cluster and it's all done in an hour.

Re: How Discord Resizes 150M Images Every Day with Go and C++

#75

Post by gsich »

Xeoncross wrote: post #8

Why don't more companies resize images client-side first using and then save the server some work by only asking it to verify the result by - resizing to the same size - removing metadata This results in much faster transfer (10x less bandwidth used often for mobile uploads) and reduces server load by "farming" out the work to the clients. https://developer.mozilla.org/en-US/docs/Web/API/CanvasRende... # Edit: On Kee…

Mandatory XKCD: https://xkcd.com/1683/

Re: How Discord Resizes 150M Images Every Day with Go and C++

#76

Post by zitterbewegung »

manigandham wrote: post #74

Nice, but why? https://cloudinary.com , https://www.imgix.com , or https://www.filestack.com already exist and are well worth it for 99% of apps. Even at scale, it really doesn't cost that much to have someone else do it. You can use a thin proxy through your existing CDN if you want to save on their bandwidth fees. Also http://thumbor.org and https://imageresizing.net if you want a library to host yourself which are…

Maybe it’s because that they don’t want a dependency on a external service that could go down ?

Re: How Discord Resizes 150M Images Every Day with Go and C++

#77

Post by kylehotchkiss »

fleitz wrote: post #59
kylehotchkiss wrote: post #43

I wish Cloudfront supported resize parameters so we wouldn't have to keep buildings these or paying a lot for Imgix.

How much would you pay for an image resizing service? I'd been thinking for a while of putting a fleet of autoscaled thumbor boxes behind cloudfront and making a billing API for it.

Imgix's $10 minimum is so much for a personal site with maybe 500 uniques a month. If you're going for a service like that, think of people like me who host on s3/cloudfront for $.20/month. But let people scale up to millions of pageviews a month.

Don't need anything fancy. Just w=? h=? would be great, developers can handle the DPI stuff with sourceset tags.

Re: How Discord Resizes 150M Images Every Day with Go and C++

#78

Post by RussianCow »

indiandennis wrote: post #73
SaltyBackendGuy wrote: post #64

Earlier quoted context omitted.

> 2) The original image has some metadata that they want to keep (geolocation, etc). Isn't exif data something you should strip out?

Discord makes most of their revenue from selling user data so they probably want to keep as much as possible.

Do you have a source? That's a pretty bold claim, and I haven't seen anything else to back that up.

Re: How Discord Resizes 150M Images Every Day with Go and C++

#79

Post by antisthenes »

Xeoncross wrote: post #8

Why don't more companies resize images client-side first using and then save the server some work by only asking it to verify the result by - resizing to the same size - removing metadata This results in much faster transfer (10x less bandwidth used often for mobile uploads) and reduces server load by "farming" out the work to the clients. https://developer.mozilla.org/en-US/docs/Web/API/CanvasRende... # Edit: On Kee…

The last thing I want is client-side image resizing when my browser is already choking on heavy javascript.

Re: How Discord Resizes 150M Images Every Day with Go and C++

#80

Post by StreamBright »

zitterbewegung wrote: post #76
manigandham wrote: post #74

Nice, but why? https://cloudinary.com , https://www.imgix.com , or https://www.filestack.com already exist and are well worth it for 99% of apps. Even at scale, it really doesn't cost that much to have someone else do it. You can use a thin proxy through your existing CDN if you want to save on their bandwidth fees. Also http://thumbor.org and https://imageresizing.net if you want a library to host yourself which are…

Maybe it’s because that they don’t want a dependency on a external service that could go down ?

So you have an internal dependency that could go down?
Post reply on HN