We saved $500k per year by rolling our own "S3"
engineering.nanit.com
We saved $500k per year by rolling our own "S3"
1–10 of 261 posts
Re: We saved $500k per year by rolling our own "S3"
#2[flagged]
Re: We saved $500k per year by rolling our own "S3"
#3Tbh I feel this in one of those that would be significantly cleaner without serverless in first place.
Sticking something with 2 second lifespan on disk to shoehorn it into aws serverless paradigm created problems and cost out of thin air here
Good solution moving at least partially to a in memory solution though
Re: We saved $500k per year by rolling our own "S3"
#4I'm curious how many engineers per year this costs to maintain
Re: We saved $500k per year by rolling our own "S3"
#5Video processing is one of those things that need caution when doing serverlessly. This solution makes sense, especially because S3s durability guarantees aren't needed.
Re: We saved $500k per year by rolling our own "S3"
#6I'm curious how many engineers per year this costs to maintain
Yes, that was my thought as well. Breakeven might be like 1 (give or take 2x)?
Re: We saved $500k per year by rolling our own "S3"
#7I'm curious how many engineers per year this costs to maintain
And I am curious how many engineer years it requires to port code to cloud services and deal with multiple issues you cannot even debug due to not having root privileges in the cloud.
Without cloud, saving a file is as simple as "with open(...) as f: f.write(data)" + adding a record to DB. And no weird network issues to debug.
Re: We saved $500k per year by rolling our own "S3"
#8Tbh I feel this in one of those that would be significantly cleaner without serverless in first place. Sticking something with 2 second lifespan on disk to shoehorn it into aws serverless paradigm created problems and cost out of thin air here Good solution moving at least partially to a in memory solution though
Yeah, so now you're basically running a heavy instance in order to get the network throughput and the RAM, but not really using that much CPU when you could probably handle the encode with the available headroom. Although the article lists TLS handshakes as being a significant source of CPU usage, I must be missing something because I don't see how that is anywhere near the top of the constraints of a system like this.
Regardless, I enjoyed the article and I appreciate that people are still finding ways to build systems tailored to their workflows.
Re: We saved $500k per year by rolling our own "S3"
#9I'm curious how many engineers per year this costs to maintain
A small fraction of 1, probably? It sounds like a fairly simple service that shouldn't require much ongoing development
Re: We saved $500k per year by rolling our own "S3"
#10I'm curious how many engineers per year this costs to maintain
And I am curious how many engineer years it requires to port code to cloud services and deal with multiple issues you cannot even debug due to not having root privileges in the cloud. Without cloud, saving a file is as simple as "with open(...) as f: f.write(data)" + adding a record to DB. And no weird network issues to debug.
> as simple as "with open(...) as f: f.write(data)"
Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and...
Without on-prem, saving a file is as simple as s3.put_object() !