Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We saved $500k per year by rolling our own "S3"

engineering.nanit.com

Post reply on HN
Print view View original

41–50 of 261 posts

Re: We saved $500k per year by rolling our own "S3"

#41

Post by YZF »

Lucian6 wrote: post #38

[flagged]

Right. Having worked on a commercial S3 compatible storage I can tell y'all that there's a lot more to it then just sticking some files on JBOD. It does depend on your specific requirements though. 1.5 FTE over 18 months sounds on the low side for everything you've described.

That said the article seems to be more about an optimization of their pipeline to reduce the S3 usage by holding some objects in memory instead. That's very different than trying to build your own object store to replace S3.

Re: We saved $500k per year by rolling our own "S3"

#42

Post by grebc »

rajamaka wrote: post #23
bcrosby95 wrote: post #12

Earlier quoted context omitted.

You can't ever definitively answer most of those questions on someone else's cloud. You just take Amazons word for whatever number of nines they claim it has.

Not needing to ask the questions is the selling point.

Bro were you off grid last week. Your questions equally apply to AWS, you just magically handwave away all those questions as if AWS/GCP/Azure outages aren’t a thing.

Re: We saved $500k per year by rolling our own "S3"

#43

Post by Nextgrid »

rajamaka wrote: post #22
codedokode wrote: post #11

Earlier quoted context omitted.

With s3, you cannot use ls, grep and other tools. > Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and... Wow that's a lot to learn before using s3... I wonder how much it costs in salaries. > With what network topology? You don't need to care about this when using SSDs/HDDs. > With what ac…

> You don't need to care about this when using SSDs/HDDs. You do need to care when you move beyond a single server in a closet that runs your database, webserver and storage. > No risk of leaking files due to misconfigured cloud services. One misconfigured .htaccess file for example, could result in leaking files.

With bare-metal machines you can go very far before needing to scale beyond one machine.

Re: We saved $500k per year by rolling our own "S3"

#45

Post by bpodgursky »

codedokode wrote: post #28
OsrsNeedsf2P wrote: post #20

Earlier quoted context omitted.

It's the government who lost 850TB of citizen data with no backups[0] Because Cloud bad. [0] https://www.techradar.com/pro/security/the-south-korean-gove...

Storing the data in a foreign cloud would allow foreign nation to play funny tricks on the country. What they need is not the cloud but sane backup system.

ap-northeast-2 is literally in Seoul

Re: We saved $500k per year by rolling our own "S3"

#46

Post by patrick451 »

rajamaka wrote: post #23
bcrosby95 wrote: post #12

Earlier quoted context omitted.

You can't ever definitively answer most of those questions on someone else's cloud. You just take Amazons word for whatever number of nines they claim it has.

Not needing to ask the questions is the selling point.

Until it goes down because because aws STILL hasn't made themselves completely multi-region or can't figure our their DNS.

Re: We saved $500k per year by rolling our own "S3"

#47

Post by beoberha »

codedokode wrote: post #7
none2585 wrote: post #4

I'm curious how many engineers per year this costs to maintain

And I am curious how many engineer years it requires to port code to cloud services and deal with multiple issues you cannot even debug due to not having root privileges in the cloud. Without cloud, saving a file is as simple as "with open(...) as f: f.write(data)" + adding a record to DB. And no weird network issues to debug.

A lot of reductive anti-cloud stuff gets posted here, but this might be the granddaddy of them all.

Re: We saved $500k per year by rolling our own "S3"

#48

Post by mjr00 »

codedokode wrote: post #7
none2585 wrote: post #4

I'm curious how many engineers per year this costs to maintain

And I am curious how many engineer years it requires to port code to cloud services and deal with multiple issues you cannot even debug due to not having root privileges in the cloud. Without cloud, saving a file is as simple as "with open(...) as f: f.write(data)" + adding a record to DB. And no weird network issues to debug.

> Without cloud, saving a file is as simple as "with open(...) as f: f.write(data)" + adding a record to DB. And no weird network issues to debug.

There may be some additional features that S3 has over a direct filesystem write to a SSD in your closet. The people paying for cloud spend are paying for those features.

Re: We saved $500k per year by rolling our own "S3"

#49

Post by sekh60 »

bpodgursky wrote: post #45
codedokode wrote: post #28

Earlier quoted context omitted.

Storing the data in a foreign cloud would allow foreign nation to play funny tricks on the country. What they need is not the cloud but sane backup system.

ap-northeast-2 is literally in Seoul

CLOUD Act.

Re: We saved $500k per year by rolling our own "S3"

#50

Post by AdieuToLogic »

rajamaka wrote: post #22
codedokode wrote: post #11

Earlier quoted context omitted.

With s3, you cannot use ls, grep and other tools. > Save where? With what redundancy? With what access policies? With what backup strategy? With what network topology? With what storage equipment and file system and HVAC system and... Wow that's a lot to learn before using s3... I wonder how much it costs in salaries. > With what network topology? You don't need to care about this when using SSDs/HDDs. > With what ac…

> You don't need to care about this when using SSDs/HDDs. You do need to care when you move beyond a single server in a closet that runs your database, webserver and storage. > No risk of leaking files due to misconfigured cloud services. One misconfigured .htaccess file for example, could result in leaking files.

>> No risk of leaking files due to misconfigured cloud services.

> One misconfigured .htaccess file for example, could result in leaking files.

I don't think you are making a compelling case here, since both scenarios result in an undesirable exposure. Unless your point is both cloud services and local file systems can be equally exploited?

Post reply on HN