Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cost of serving billions of images per month

medium.com

Post reply on HN
Print view View original

41–50 of 138 posts

Re: Cost of serving billions of images per month

#41

Post by pram »

subpixel wrote: post #37
manigandham wrote: post #9

What's the business model here? We would never run with such expensive options for this functionality, especially if it was a free/donations powered service. This company raised $7.5M though: https://techcrunch.com/2018/02/15/unsplash-simple-token-seri...

They raised money from the crypto-coin outfit run by the former fab.com CEO. At Fab and at his previous company it seems the modus oprandi was to raise millions and then pocket as much as possible before the shit hit the fan.

I ordered so much stuff from Fab back in the day! I thank the investors for their largess

Re: Cost of serving billions of images per month

#42

Post by scarface74 »

abhiyerra wrote: post #40
tpetry wrote: post #18

Earlier quoted context omitted.

Don‘t underestimate the massive cpu power needed for scaling images at their request rate

Wonder if they could use S3 bucket with a lambda to resize. I’m sure their images are a long tail so certain images likely have way more traffic than others. At a previous company where we were also doing a lot of imgix traffic we did something like this but with a go service but now I’d just offload to s3 and use lambda to avoid Devops..

I’ve done exactly this. But not at any scale. But if they are specializing in this and with low margins, AWS is not the answer. The compute cost would be relatively cheap and possibly even the storage cost. I would think that the bandwidth costs would be expensive.

Re: Cost of serving billions of images per month

#43

Post by mypalmike »

quickthrower2 wrote: post #22
quickthrower2 wrote: post #5

Earlier quoted context omitted.

Moreover what if Heroku goes out of business? Then they’re f’d. Hosting on k8s, using Terraform or whatever would give them a lot of automation and provider independence.

I’m guessing I’m being grayed because I’m suggesting Heroku might go out of business? Ok let me explain. That is one scenario, it’s unlikely but there might be other similar scenarios that pose risk being tied in to one providers stack. DDOS, TOS issues, simply out growing them, Heroku themselves having TOS issues with their cloud. There are almost weekly posts on HN of “this major service is down” and “that major se…

I didn't downvote, but...

"I understand a scrappy startup needing to focus on growth and using the most convenient tools"

Well, there's your answer. As a scrappy startup, you can't afford to plan migrations you aren't planning to execute. It's that simple. Heroku is not going away (they're owned by salesforce) and they have a good reputation for uptime.

The "what if" worst case scenarios have to be weighed against other business concerns. And this company appears to have done a good job of navigating such things.

Re: Cost of serving billions of images per month

#44

Post by scarface74 »

quickthrower2 wrote: post #22
quickthrower2 wrote: post #5

Earlier quoted context omitted.

Moreover what if Heroku goes out of business? Then they’re f’d. Hosting on k8s, using Terraform or whatever would give them a lot of automation and provider independence.

I’m guessing I’m being grayed because I’m suggesting Heroku might go out of business? Ok let me explain. That is one scenario, it’s unlikely but there might be other similar scenarios that pose risk being tied in to one providers stack. DDOS, TOS issues, simply out growing them, Heroku themselves having TOS issues with their cloud. There are almost weekly posts on HN of “this major service is down” and “that major se…

Well let’s see where to start. Terraform is not magic. All of the provisioners are specific to a platform.

Re: Cost of serving billions of images per month

#45

Post by scarface74 »

kitotik wrote: post #36
Mizza wrote: post #23

Earlier quoted context omitted.

Can we please, please, please kill the myth that Terraform in any way helps in any way with provider independence?

I find it much easier to maintain terraform code that allows me to use multiple cloud providers than if I had to do the same without terraform. You seem to have some insight to share here though, could you expound? *edit typo

How would Terraform help you move between cloud providers when all of the provisioners are specific to the cloud platform.

If you’re a small startup, the least of your business risks are one of the major cloud providers shutting down.

Re: Cost of serving billions of images per month

#46

Post by scarface74 »

ron22 wrote: post #17
busymom0 wrote: post #12

Is there any particular reason they are still using Heroku instead of something like DO or AWS+S3? They can probably save a lot of money that way. Unless Heroku is offering them some good deal?

They tell you in the blog post. "We continue to use Heroku as our main web platform. Despite its premium cost over AWS, Azure, and Google Cloud, Heroku’s built-in deployment and configuration tools allow our team to move faster, more confidently, and more reliably. As we’ve detailed previously, the alternatives would undoubtably be cheaper on paper. But in reality, the increased simplicity and freedom offered by Hero…

I’m far from a Devops guy. But deploying to AWS using their native tools are Azure using Microsoft’s tools is dead simple.

Re: Cost of serving billions of images per month

#48

Post by m0zg »

tpetry wrote: post #18
manigandham wrote: post #14

Earlier quoted context omitted.

And some projects really are just simple. Scale makes no difference to their services, it's a basic OLTP + search with resizing image proxy. This doesn't have any SLAs nor is it critical software either. 24/7 oncall is completely unnecessary.

Don‘t underestimate the massive cpu power needed for scaling images at their request rate

Why would it depend on request rate rather than upload rate? Just pre-scale several variants on upload. Then run an MR job every now and then and delete the variants that haven't been touched for more than e.g. a month. Scale those on demand, store recents and frequently accessed pre-scaled. This is literally a few days of work.

Re: Cost of serving billions of images per month

#49

Post by HelloFellowDevs »

busymom0 wrote: post #13
amanzi wrote: post #11

So they must have some form of business model, right? All I can see on their website is that everything is for free, even their API which can be used for free without any limits or restrictions. And given the already huge Imgix costs, abusing or over-using their API to request and resize images would directly increase their overall costs. According to Crunchbase ( https://www.crunchbase.com/organization/unsplash#sect…

This reminds me of Soundcloud a few years ago when they were giving away API access for free for unlimited plays but then were almost close to bankrupt and pretty much shut down all API plays. I ended up shutting down my 3rd party app then but not sure what's their API limits now. Twitter recently made huge changes to their API limits too which impacted a lot of 3rd party apps. I hope Unsplash doesn't end up similar…

On the note of Twitter's restriction of API usage, Jack talked about it recently on the podcast Tales from the Crypt [0]. He doesn't give any specifics but attributes Twitters early struggle with uptime to unrestricted API usage [1].

[0] https://podcasts.apple.com/us/podcast/tales-from-the-crypt/i...

[1] https://overcast.fm/+KiHpBJHGo/15:30

Post reply on HN