Does anyone have any real world experience with how the Amazon DynamoDB Accelerator that AWS released as a preview a few months ago can help out?
There are very few suitable use cases for DynamoDB
41–50 of 146 posts
Re: There are very few suitable use cases for DynamoDB
#42When I started building my first app in 2011 MongoDB was the rage. So I build the back-end using the futuristic 'No-SQL' technology. It turned out to be slow (~1 min Query time), inconsistent, and missing an RDBMS layer. Move the thing to PHP/Mysql problems were gone. I still have not found a use case outside of web (comments/discussion) sites where the high integration with Javascript actually makes sense.
I hope you don't mind if I piggyback on this to echo this sentiment. Although I dislike MySQL for its many gotchas (data corruption level stuff too!) I was looking for a _long_ _long_ time for a high consistency NoSQL database.. we basically need document storage of large binary data. Ironically literally nothing in NoSQL land does write-through to disk, they just write to vfs and hope it works; additionally, those t…
You are talking complete nonsense.
HBase, Cassandra, MongoDB, Riak, Couchbase etc all write through to disk with proper fsync flushes. And I've never heard of any database that has a model where it writes to a virtual file system - whatever that even means.
Please provide some specific examples.
Re: There are very few suitable use cases for DynamoDB
#43Earlier quoted context omitted.
I hope you don't mind if I piggyback on this to echo this sentiment. Although I dislike MySQL for its many gotchas (data corruption level stuff too!) I was looking for a _long_ _long_ time for a high consistency NoSQL database.. we basically need document storage of large binary data. Ironically literally nothing in NoSQL land does write-through to disk, they just write to vfs and hope it works; additionally, those t…
Isn't that the point of no_sql. You trade consistency for speed. Best used to mark temporary state in a web app. If you need atomic consistency in a database you should use a relational database that does atomic writes. That has always been the case. Why would it be so shocking to you? Atomic writes have already been built. Sounds like you were standing there with a hammer looking for a nail.
Please be specific. NoSQL comprises hundreds of different databases with very different semantics.
Technically PostgreSQL is a NoSQL database and it along with Cassandra for example have modes in which they are fully and atomically consistent.
Re: There are very few suitable use cases for DynamoDB
#44Earlier quoted context omitted.
I often give this same advice, but assuming this is the case, why reach for DynamoDB at all? Are there small-data use cases where DynamoDB makes more sense than RDS or a hosted redis? At least with those, if you do run into scaling issues you haven't locked yourself into dynamo.
I would imagine the use of AWS Lambda might be a pretty decent incentive to use DynamoDB. I'm guessing that stateless DynamoDB queries are faster to fire off than dealing with Postgres or MySQL connection setup and teardown.
I'm not familiarized with DynamoDB, but why would connecting to it be any faster than connecting to Postgres or MySQL?
Re: There are very few suitable use cases for DynamoDB
#45"Your business has millions of customers and no single customer can do so many actions so quickly that the individual could create a hot key. Under this key you are storing around 2KB of data. ... Potentially getting 1–5 requests per second for a given second but certainly not a sustained load of that. ... This will not work at scale in DynamoDb." What? Why? Suppose that's 5 million customers, you will only have a 10…
Amazon is limiting it to 5 operations per second per shard, not per customer. That is if you have 200 shards and 1000 write capacity.
It's a problem with how Amazon calculates your capacity units as it scales up Dynamo for you, if you have a particularly large data set with a few very active users.
Re: There are very few suitable use cases for DynamoDB
#46I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned. Fundamentally, the problem seems to be that choosin…
Conpletely agreed; using DynamoDB for primary storage suffers from needing to design data models around the underlying technology. This is true for secondary keys as well, and is even constrained by the odd pricing model that DynamoDB follows. I've been happy using DynamoDB as a large-scale caching layer, but even that only fits very specific use case criteria.
> using DynamoDB for primary storage suffers from needing to design data models around the underlying technology.
Every database system, at large scale, suffers from this problem.
Re: There are very few suitable use cases for DynamoDB
#47We use DynamoDB quite a bit at Thumbtack. Our biggest issue is backups - just wrote a short note about our experiences with DynamoDB here: https://medium.com/@natekupp/dynamodb-and-backups-16dba0dbcd...
Re: There are very few suitable use cases for DynamoDB
#48When I started building my first app in 2011 MongoDB was the rage. So I build the back-end using the futuristic 'No-SQL' technology. It turned out to be slow (~1 min Query time), inconsistent, and missing an RDBMS layer. Move the thing to PHP/Mysql problems were gone. I still have not found a use case outside of web (comments/discussion) sites where the high integration with Javascript actually makes sense.
I hope you don't mind if I piggyback on this to echo this sentiment. Although I dislike MySQL for its many gotchas (data corruption level stuff too!) I was looking for a _long_ _long_ time for a high consistency NoSQL database.. we basically need document storage of large binary data. Ironically literally nothing in NoSQL land does write-through to disk, they just write to vfs and hope it works; additionally, those t…
Redis
Re: There are very few suitable use cases for DynamoDB
#49Quick aside:
It seems that a lot of the qualms with various databases stem from a misunderstanding of their use cases. A lot of the features of major SQL databases, namely ACID, are misappropriated to be features of databases in general. This key misunderstanding seems to cause a lot of SWEs insane as they later realize that NoSQL DBs are not always Available and Strongly Consistent.
Responding to Author:
I wasn't totally convinced by the author's argument against DynamoDB. This article [1] offers a good solution to pretty much all of OP's problems. Most significantly, hashing user data using date.
While DynamoDB is certainly different from most other databases, that doesn't mean that there aren't sensical usage methodologies.
Links:
[1] https://medium.com/building-timehop/one-year-of-dynamodb-at-...
Re: There are very few suitable use cases for DynamoDB
#50TL;DR - Don't use something as coarse as customer_id as a partition key. Alternatively move to GCP/BigTable. Any DynamoDB tuning advice will say how important it is to have well distributed hash keys. As for the second part, why not use Cloud Spanner? I wish AWS had something like it.
[deleted]