51–58 of 58 posts
Production DB without backups is unthinkable. It just takes one human mistake to erase tons of data. Consistent and regular backups are must have for any production system.
I was at AWS from 2008 to 2016. Werner Vogels, Amazon's CTO (yep, not just AWS', but Amazon's, as he had to point out numerous times) has been one of the most talented, humble and generous senior exec I've ever met in my life. Lots of good memories of time spent with him, and one of the sad aspects for me of leaving Amazon. His blog writings are really interesting. If you haven't already, I suggest you search the arc…
I have always tremendously admired that guy; what a job he has! He is literally responsible for about half of the shit on the internet not going down (and also running the biggest internet shopping mall and making sure a bunch of crappy speakers can talk, but both of those are straightforward by comparison IMO). Can you even imagine? I want to know what his direct report structure looks like.
Earlier quoted context omitted.
>Dynamo gets expensive and it gets expensive quick, DynamoDB's pricing scales sublinearly with volume; if it starts getting expensive it was an initial misuse of DynamoDB that got obvious with scale. There are a lot of factors that go into whether you should use DynamoDB and how you implement it. I recommend anyone who is considering using it very carefully understand this page first: http://docs.aws.amazon.com/amazo…
This is how enterprise developers use the database, sometimes: https://thedailywtf.com/articles/The-Query-of-Despair You do this on your own server, slowness and bad performance are the result (but it may never, or very rarely get called). You do it on dynamo, a $10k bill may be the result.
The only way you could be "surprised" with a $10k bill is if you set up autoscaling for it with your upper limits (which it requires you to choose) high enough to reach $10k. And then you'd have to forget that you did that.
I'm more interested in solutions like Spanner and Cockroach. Different tradeoffs for different applications, but they seem to be the most general purpose of the highly scalable databases. DynamoDB is cool and I've tried to adopt it for things, but it's surprisingly hard to imagine an application where the model isn't somewhat limiting. The capacity provisioning is also quite painful, which doesn't help matters any.
It feels like using a relational model (with SQL) is fundamentally different in that you're designing how you're storing your data based on how it relates to other data. It's certainly a whole lot easier to design for, but it makes me wonder if it is possible to close that gap in terms of "easy to design / SQL / vertically scalable" and "serverless / cheap / horizontally scalable".
Earlier quoted context omitted.
Anything that can go to dynamo, can go to s3, especially at that volume. And you get proper multi region replication, read/write capacity based on actual usage and instant scaling. I stay by my comment that dynamodb is a joke wrapped in thick layer of marketing crap.
You cannot replace DynamoDB with S3. S3 can not perform atomic and strongly consistent operations. Edit: as other commenters have noted, you can perform a read after write on a new key.
Earlier quoted context omitted.
I have always tremendously admired that guy; what a job he has! He is literally responsible for about half of the shit on the internet not going down (and also running the biggest internet shopping mall and making sure a bunch of crappy speakers can talk, but both of those are straightforward by comparison IMO). Can you even imagine? I want to know what his direct report structure looks like.
He is an individual contributor
Earlier quoted context omitted.
I have always tremendously admired that guy; what a job he has! He is literally responsible for about half of the shit on the internet not going down (and also running the biggest internet shopping mall and making sure a bunch of crappy speakers can talk, but both of those are straightforward by comparison IMO). Can you even imagine? I want to know what his direct report structure looks like.
He is an individual contributor