21–30 of 146 posts
I don't think we're getting the whole story from the author. I'm not the biggest fan of Dynamo either for reasons I won't get into here, but this type of workload is exactly what Dynamo was built for: serving a website with millions of customers.
Disclaimer: I work at Amazon, my views are my own.
When I started building my first app in 2011 MongoDB was the rage. So I build the back-end using the futuristic 'No-SQL' technology. It turned out to be slow (~1 min Query time), inconsistent, and missing an RDBMS layer. Move the thing to PHP/Mysql problems were gone. I still have not found a use case outside of web (comments/discussion) sites where the high integration with Javascript actually makes sense.
I hope you don't mind if I piggyback on this to echo this sentiment. Although I dislike MySQL for its many gotchas (data corruption level stuff too!) I was looking for a _long_ _long_ time for a high consistency NoSQL database.. we basically need document storage of large binary data. Ironically literally nothing in NoSQL land does write-through to disk, they just write to vfs and hope it works; additionally, those t…
That has always been the case. Why would it be so shocking to you? Atomic writes have already been built. Sounds like you were standing there with a hammer looking for a nail.
DynamoDB is definitely not the silver bullet people hope it is but does work exceptionally well for what it was designed for.
I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned. Fundamentally, the problem seems to be that choosin…
I don't know what the article author is storing, but it's noted that 10GB of data is stored per node. That's quite a bit of data for a single table, and the 10GB is per shard of a table, not per 'database' (DynamoDB only has a notion of tables). Amazon has deep dive talks on DynamoDB on YouTube[1] that go into lots of these details and how to avoid problems from them. It's not that different from understanding how to…
I can understand having to optimize your key space, but in this case it necessitates extreme premature optimization.
I don't know what the article author is storing, but it's noted that 10GB of data is stored per node. That's quite a bit of data for a single table, and the 10GB is per shard of a table, not per 'database' (DynamoDB only has a notion of tables). Amazon has deep dive talks on DynamoDB on YouTube[1] that go into lots of these details and how to avoid problems from them. It's not that different from understanding how to…
I don't know what the article author is storing, but it's noted that 10GB of data is stored per node. That's quite a bit of data for a single table, and the 10GB is per shard of a table, not per 'database' (DynamoDB only has a notion of tables). Amazon has deep dive talks on DynamoDB on YouTube[1] that go into lots of these details and how to avoid problems from them. It's not that different from understanding how to…
We store 100-1000 GB of data in each wide table on our untrendy on-prem SQL server boxes. 10GB is peanuts. In fact, I propose (given the limitations suggested by the author) that DynamoDB may have NO practical uses worth exploring. If NoSQL is about scale, and it can't scale, what's it good for? I can understand having to optimize your key space, but in this case it necessitates extreme premature optimization.
Earlier quoted context omitted.
We store 100-1000 GB of data in each wide table on our untrendy on-prem SQL server boxes. 10GB is peanuts. In fact, I propose (given the limitations suggested by the author) that DynamoDB may have NO practical uses worth exploring. If NoSQL is about scale, and it can't scale, what's it good for? I can understand having to optimize your key space, but in this case it necessitates extreme premature optimization.
Each shard is 10GB. The unit of scale is a shard. You can have as many shards as you want.
I don't know what the article author is storing, but it's noted that 10GB of data is stored per node. That's quite a bit of data for a single table, and the 10GB is per shard of a table, not per 'database' (DynamoDB only has a notion of tables). Amazon has deep dive talks on DynamoDB on YouTube[1] that go into lots of these details and how to avoid problems from them. It's not that different from understanding how to…
It's amazing how much you can put into a reply without directly addressing any of the concerns. Are you serious that dynamodb can't even hold 10GB? If you intended to defend the database I think you failed. Sqlite3 can store 10 gigs without sweating. What on earth can you use dynamo db for if it doesn't scale with the data? The way you describe it seems much WORSE than the OP.
Edit: Sorry, I did mention 10GB originally in relation to a table. That was incorrect of course.