Earlier quoted context omitted.
They also have extensive partnerships with certain brands — their homepage currently serves ads disguised as content from a handful of companies. (I say this as a user of and contributor to Unsplash)
So ads? I thought it would be something a little more inspired.
Cost of serving billions of images per month
71–80 of 138 posts
Re: Cost of serving billions of images per month
#72Earlier quoted context omitted.
Don‘t underestimate the massive cpu power needed for scaling images at their request rate
Wonder if they could use S3 bucket with a lambda to resize. I’m sure their images are a long tail so certain images likely have way more traffic than others. At a previous company where we were also doing a lot of imgix traffic we did something like this but with a go service but now I’d just offload to s3 and use lambda to avoid Devops..
Re: Cost of serving billions of images per month
#73Earlier quoted context omitted.
"He said it’s too soon to know exactly what that [business] model will be, but it will involve blockchain technology and cryptocurrency" [1] [1] https://techcrunch.com/2018/02/15/unsplash-simple-token-seri...
Is there a way I can short this company?
Re: Cost of serving billions of images per month
#74Earlier quoted context omitted.
I’m guessing I’m being grayed because I’m suggesting Heroku might go out of business? Ok let me explain. That is one scenario, it’s unlikely but there might be other similar scenarios that pose risk being tied in to one providers stack. DDOS, TOS issues, simply out growing them, Heroku themselves having TOS issues with their cloud. There are almost weekly posts on HN of “this major service is down” and “that major se…
Heroku mandates the use or 12Factor apps, which makes it much easier to migrate. They support docker as well. Pretty much every app I write I usually starts on Heroku and migrate later if the client / traffic needs it, and it’s always been seamless. Most apps have Dev/test on Heroku with a staging and production on AWS, and because of the 12 factor principles, which also work on Beanstalk / ECS / normal servers it’s…
Re: Cost of serving billions of images per month
#75Earlier quoted context omitted.
Don‘t underestimate the massive cpu power needed for scaling images at their request rate
Why would it depend on request rate rather than upload rate? Just pre-scale several variants on upload. Then run an MR job every now and then and delete the variants that haven't been touched for more than e.g. a month. Scale those on demand, store recents and frequently accessed pre-scaled. This is literally a few days of work.
But since images are immutable, this is a perfect use-case for lambda functions fronted by a CDN.
Re: Cost of serving billions of images per month
#76Earlier quoted context omitted.
So ads? I thought it would be something a little more inspired.
Yes. Of the first 7 images 3 are adverts. Then you can scroll down a long way on mobile without any more ads.
Re: Cost of serving billions of images per month
#77Earlier quoted context omitted.
I’m far from a Devops guy. But deploying to AWS using their native tools are Azure using Microsoft’s tools is dead simple.
Yeah something doesn't add up. I'll believe Heroku is easier, but can it be that much easier? For people spending piles of cash on their webscale?
Every time we need to re-negotiate our contract with Heroku we ask ourselves whether we should move away from Heroku or not. So far, the answer has always been no. Complexity would go up and the time spent on DevOps would go up. Right now, any idiot can do a deployment, or a rollback. Most of the things we need are taken care off for us by Heroku and it doesn't cost us any effort. We use Heroku Postgres for our database and we get point-in-time restore for free, without doing anything. We can upgrade our database or create a new follower in minutes. It just requires a couple of clicks. On top of that, all the monitoring is taking care of by Heroku. It just gives us a nice dashboard.
As for costs, on an negotiated Enterprise contract, it's not that bad. If we'd move to AWS or DO for example, we might save a little bit, but the extra engineering resources would quickly make those savings useless. Even if it's a bit more expensive to run on Heroku, it's still cheaper than cobbling together all of those features on another platform.
We've made a couple of small engineering investments to get features that Heroku doesn't offer. For example, we built a simple auto-scaler for dyno's that consume from RabbitMQ. This helps a bit to keep our costs down and respond to massive spikes quickly.
We run our web servers, database, caching, data pipeline etc on Heroku. However, we are not _that_ dependent on Heroku. We use Docker for deployments, so we can easily reproduce what's happening and it reduces our vendor lock-in.
I hope this explains a little why some people might choose to stick to Heroku. It works for us, it might not work for you or anyone else.
Re: Cost of serving billions of images per month
#78$99K/month in hosting costs and no visible source of revenue... I think this baby is ready for an IPO!
Re: Cost of serving billions of images per month
#79Earlier quoted context omitted.
$50k/month comes out to $600k/year which is the cost around 2-3 engineers counting overhead. To manage infrastructure at that scale themselves they'd need someone who knows DevOps. To write services at that scale they'd need someone who knows backend engineering and micro services. Then they'd need to be able to have 24/7 on call rotations for when things break. And of course, like any rewrite it seems like a "couple…
600k year for 2-3 is only real in the silicon valley bubble. You could get just as talented of a team from a Nordic Country for example and only pay 120kish