31–40 of 76 posts
(As an aside: I've been following your work for the past four years - as someone who is quite passionate about infosec, I'd like to thank you deeply for your contributions to the community.)
As always, picodollar pricing is so confusing.
Earlier quoted context omitted.
2¢, 3¢, 10¢ per day, does that really matter? We are talking about less than a Starbucks for a full month of backups. Sorry, but developers who literally optimize cents over their time and effort is a pet peeve of mine.
2¢, 3¢, 10¢ per day, does that really matter? It doesn't matter right now, but if you're building a company which you hope to scale up, it's good to have costs which won't get too big when said scaling happens -- because when your company explodes overnight, you're going to be too busy keeping everything else running to spend time reworking your backup strategy. (Also: If you don't have backups now because you're "no…
This is a popular sentiment among engineers at startups and in my opinion founders should work hard to (kindly) beat it out of the team as soon as possible.
Basically, you can use that line to justify nearly any engineer hobby. We need microservices! We need message queues! We need master-to-master replicated NoSQL databases spread geographically! We need Redis, Kafka and Cassandra with a CQRS event source pumping data in there, oh and also to a Postgres so we can do arbitrary queries for management reports! We need our backups to cost not 3 but 2 cents a month!
But the truth is that, statistically, the chance that a startup will not actually scale up, is much bigger than the chance that once it does, the sharply increasing Tarsnap bill is going to drive it to bankruptcy.
I agree with you that you need backups from the start, but whether they cost $.02 per month or $10 per month initially really doesn't matter. There's a lot of features to build (and kill), users acquire, content to market, and the team is tiny.
I used to laugh about Twitter during their early growth days, fail whales all over the place. "What, they made that in Rails? Idiots". Now that I'm a startup founder myself, I realize that they did it perfectly right and I strongly doubt Twitter would've been what it is today if they had wasted time getting the perfectly scalable tweet processing timeline before putting the site out there.
[1] https://summitroute.com/blog/2016/12/25/creating_disaster_re...
[2] https://summitroute.com/blog/2016/12/25/using_google_for_bac...
Earlier quoted context omitted.
2¢, 3¢, 10¢ per day, does that really matter? It doesn't matter right now, but if you're building a company which you hope to scale up, it's good to have costs which won't get too big when said scaling happens -- because when your company explodes overnight, you're going to be too busy keeping everything else running to spend time reworking your backup strategy. (Also: If you don't have backups now because you're "no…
> but if you're building a company which you hope to scale up This is a popular sentiment among engineers at startups and in my opinion founders should work hard to (kindly) beat it out of the team as soon as possible. Basically, you can use that line to justify nearly any engineer hobby. We need microservices! We need message queues! We need master-to-master replicated NoSQL databases spread geographically! We need…
Yes, simplify your system by using as few moving parts as possible. But that also means don't use bloated frameworks that silently slow down your iteration pace with technical debt.
So many startups I've consulted with were stuck with having to redo core architecture right when they found market fit. It's a tough position to be in. The main rule to follow is that good design allows better design to happen later, it's worth investing in that
This has appeared on HN a few times.. I'd really like to use tarsnap but it's simply too expensive to be viable (to which I guess the response is: don't use tarsnap for full-backups -- but this makes it practically useless as far as diminishing the complexity of my current backup scheme[0]). [0] http://duplicity.nongnu.org/
I'm a little unclear - does Duplicity run on macOS?
Earlier quoted context omitted.
> but if you're building a company which you hope to scale up This is a popular sentiment among engineers at startups and in my opinion founders should work hard to (kindly) beat it out of the team as soon as possible. Basically, you can use that line to justify nearly any engineer hobby. We need microservices! We need message queues! We need master-to-master replicated NoSQL databases spread geographically! We need…
I think it's a false dichotomy that great architecture and fast iteration are at odds. In fact, great architecture is what allows fast iteration to happen. Yes, simplify your system by using as few moving parts as possible. But that also means don't use bloated frameworks that silently slow down your iteration pace with technical debt. So many startups I've consulted with were stuck with having to redo core architect…
> So many startups I've consulted with were stuck with having to redo core architecture right when they found market fit. It's a tough position to be in.
Good point, but watch out for survivor bias here: plenty startups failed before they reached product market fit, and some did so because they wasted time doing the wrong things. All things be told, I'd rather run a startup that needs to hire you to fix the core at the worst possible moment, than a startup that fails. But I agree fully nevertheless: good design begets good design and it really doesn't take much time.
Earlier quoted context omitted.
Colin, Any interest or thoughts about adding Google Cloud Storage as an option vs AWS S3? S3 (us-east-1) runs $0.023 per GB. Google Cloud Storage in a single region runs slightly less at $0.02 per GB. Howerver, bandwidth may be the biggest cost factor for you not storage.
The price difference between S3 and Google Cloud Storage isn't enough to make a meaningful difference, particularly since I'd need to set up server code in Google Cloud to manage it. Bandwidth isn't a huge cost; but the server which keeps track of where all the bits are in S3 and shuffles them around is surprisingly expensive.
What about if you were to age things into Nearline and Coldline using lifecycle policies? Also, someone claims you do multiple regions, which our "default" multiregional does at less than half the cost of S3.
Admittedly we only recently (finally!) added per-object storage classes. But unless I misunderstand how your blocks work in tarsnap, wouldn't that be amenable to having some large portion as Nearline and possibly even Coldline? If not, I'd love to understand! (contact info in my profile).
If you're doing backups for your business, I've written on how to properly encrypt backups[1] and how to use Google Compute Engine for backups[2]. I'm working on write-ups for AWS and Azure that should post within the new few weeks. [1] https://summitroute.com/blog/2016/12/25/creating_disaster_re... [2] https://summitroute.com/blog/2016/12/25/using_google_for_bac...
Also not mentioned is that each service also supports versioning, which for backups that don't do block-based backup, can be an alternative DR plan (e.g., don't allow some users to delete the last version).
All in all, a good start (complete with helpful screenshots!). Looking forward to the guides on S3 and Azure Blobs.
Disclosure: I work on Google Cloud.
This has appeared on HN a few times.. I'd really like to use tarsnap but it's simply too expensive to be viable (to which I guess the response is: don't use tarsnap for full-backups -- but this makes it practically useless as far as diminishing the complexity of my current backup scheme[0]). [0] http://duplicity.nongnu.org/
Besides that, you might just consider replicating to a mini-computer of some sort in another location that you have permission to use like a friend's or relative's house. A raspberry pi connected to an external drive should probably do the trick.
Duplicity looks like a good tool by the way, thanks for bringing it up.