Live data from Hacker News

There are very few suitable use cases for DynamoDB

syslog.ravelin.com

111–120 of 146 posts

Re: There are very few suitable use cases for DynamoDB

#111
post #7

The gist of this seems to be that DynamoDB becomes a problem if you have millions of customers. Don't worry. You don't. And there will be many good reasons to refactor the architecture before you do.

We have close to 25M monthly active users and DynamoDB works pretty well for our use case. We store user generated content in DynamoDB. Our largest table is over 1 TB. The mapping of users to their generated content is in Postgres. So doing work on behalf of a particular user will generally be distributed across multiple nodes. We've enjoyed the very predictable performance of DynamoDB as well as the operational simp…

Sounds interesting! But if the mapping is stored in Postgres, your system's availability is capped by that.

Re: There are very few suitable use cases for DynamoDB

#112
post #79

There are a few misguided views in this article and in some of these comments. 1. Every shardable database (Cassandra, Dynamo, BigTable) has to worry about hot spots. Picking a UUID as a partition key is only step one. What happens if one user is a huge majority of your traffic? All of their reads/writes are going to a single partition and of course you are going to suffer from performance issues from that hot spot.…

> Every shardable database (Cassandra, Dynamo, BigTable) has to worry about hot spots.

Yeah, but the issue with DynamoDB seems to be bursts of access triggering "throughput exceptions" caused a very static bandwidth allocation which is going down with the number of shards and not so graceful handling of overload situations.

It is imho. an anti-pattern to split up the bandwidth like they do. It negates the multiplexing gain for no good reason except a rigid control model.

Re: There are very few suitable use cases for DynamoDB

#113
post #92
post #8

I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned. Fundamentally, the problem seems to be that choosin…

If you have significantly more buckets than users though shouldn't the variations in access pattern even out?

It depends on what the extremes look like. Say things are power law distributed -- you have three users that require 1000000x capacity -- then as you shard more you basically guarantee that some shards are much, much more heavily loaded than others.

Re: There are very few suitable use cases for DynamoDB

#114

NoSQL should stay as a quick and dirty solution for storing key-values. When you start to schematize it and use it as a "DB," you are going down the wrong path. That is because NoSQL is a glorified cache, not a DB. It is essentially using the memory on a large number of nodes to buffer bursty throughput, and using background processes to collate the data later onto disk. There is almost no case where an explicit dist…

> It is essentially using the memory on a large number of nodes to buffer bursty throughput, and using background processes to collate the data later onto disk.

You're not describing NoSQL. You're describing something like MongoDB specifically. You're right about the specific case, wrong in the general case.

> There is almost no case where an explicit distributed caching or queuing solution backed by a traditional DB isn't strictly better.

Bah.

If you're looking at big enough data sets (think "Walmart scale", not "web scale") then you have to design your data store to cope with this volume. First, it has to be replicated to multiple servers to serve more queries and for reliability, and is an eventually consistent system a consequence (basically any time you use your read replica, or do anything on a multimaster configuration). You'll need something shardable or multimaster to achieve horizontally scalablity and you'll probably also lose your foreign key enforcement and uniqueness constraints when you do this, so you might as well go the distance and fully denormalise everything for consistent access times. (Have you ever had the Postgres query planner suddenly make something take 10x longer than it usually does because your data patterns changed and decided to use a different query plan than it's ever used before? Yeah, that's a world of fun. Now make that thing a time-critical daily batch job and really squirm.)

When you do the things to make this work, you're basically restricting yourself, abandoning database features. As a consequence, you are going to be in for a world of pain, period, sorry. One key advantage of good NoSQL software is that it doesn't give you those features to begin with, so it's harder to get it wrong. It also has data structures designed to make this sort of thing work well and be performant in ways that general purpose databases won't be, on account of being designed for general purposes.

Re: There are very few suitable use cases for DynamoDB

#115
post #20
post #10

Earlier quoted context omitted.

I often give this same advice, but assuming this is the case, why reach for DynamoDB at all? Are there small-data use cases where DynamoDB makes more sense than RDS or a hosted redis? At least with those, if you do run into scaling issues you haven't locked yourself into dynamo.

I would imagine the use of AWS Lambda might be a pretty decent incentive to use DynamoDB. I'm guessing that stateless DynamoDB queries are faster to fire off than dealing with Postgres or MySQL connection setup and teardown.

Lambda keeps the containers running if there's traffic coming to them, so the setup cost isn't paid per request unless there are few requests.

Re: There are very few suitable use cases for DynamoDB

#116
post #111

Earlier quoted context omitted.

We have close to 25M monthly active users and DynamoDB works pretty well for our use case. We store user generated content in DynamoDB. Our largest table is over 1 TB. The mapping of users to their generated content is in Postgres. So doing work on behalf of a particular user will generally be distributed across multiple nodes. We've enjoyed the very predictable performance of DynamoDB as well as the operational simp…

Sounds interesting! But if the mapping is stored in Postgres, your system's availability is capped by that.

Yes. We're migrating towards a more available store (one option we're testing out is Aurora) but availability wasn't our primary motivation in choosing DynamoDB. The main motivation was consistent performance with a dataset too large to fit into memory.

Re: There are very few suitable use cases for DynamoDB

#117
So, Amazon actually kind of agrees! They talked about this very issue of hot keys and overprovisioning at their most recent conference. The thing is it was buried in a session on Aurora and they didn't mention DynamoDB by name -- they just called it nosql -- but they noted that a customer cut their costs 40% by moving to Aurora. Because it does automatic heat management and just bills you for what IO you use.

This is somewhat at odds with their top-level messaging which still pushes DynamoDB as the most scalable solution. And perhaps it is... there are some scalability limits to Aurora. Writes are bottlenecked by one instance. 64TB max. I think performance drops when you exceed the in memory cache. But those limits are still quite large.

Basically I sense some tension between the DynamoDB and Aurora teams and I wonder where this is all going to shake out in the long run.

Here's the full quote (I transcribed it so may contain errors):

"The one thing that surprised me is that there are some customers who are moving their nosql workload to aurora. There are two reasons for that. One, it’s a lot easier to use Aurora because it’s mysql compatible compared to nosql because the interfaces and transaction characteristics are so different. What is also interesting is people also saved money because the IO cost is much lower. In no SQL if you have a large table it gets partitioned then the IO gets partitioned across all the table partitions that you have. And if you have one partition that is hot then you have to provision based on the IO requirement of the hot partition. In the case of Aurora we do automatic heat management so we don’t have this hot parition issue. Second we don’t charge based on provisioned IO. It’s only the IO that you use. And that actually saves a lot of money. In this particular case this is a big social company, interaction company, I cannot tell the name, and they reduced their operational costs by 40% by moving from nosql to Aurora" [1]

[1] https://youtu.be/60QumD2QsF0?t=17m01s

Re: There are very few suitable use cases for DynamoDB

#118
post #2

When I started building my first app in 2011 MongoDB was the rage. So I build the back-end using the futuristic 'No-SQL' technology. It turned out to be slow (~1 min Query time), inconsistent, and missing an RDBMS layer. Move the thing to PHP/Mysql problems were gone. I still have not found a use case outside of web (comments/discussion) sites where the high integration with Javascript actually makes sense.

Why did you use MongoDB if your domain model wasn't suited ? I just don't understand people who complain about a technology and say it is useless for all of these use cases when they couldn't even spend a few hours to do a Spike/POC or some basic data domain design. MongoDB has very clear documentation about what you should or shouldn't use it for. MongoDB is unique in that it is one of the few document stores availa…

> Why did you use MongoDB if your domain model wasn't suited?

Because MongoDB's marketing sold it as the hot new datastore that made SQL legacy - "newer! faster! web scale!" not "only use this if your data isn't relational".

The reference documentation might be more accurate but the way it was publicised certainly wasn't.

Re: There are very few suitable use cases for DynamoDB

#119
post #102

Earlier quoted context omitted.

20TB in an RDBMS has very different characteristics than 100GB in an RDBMS. Once you can no longer store your dataset in memory, you start to see greater variance in latency. DynamoDB for us has given us very reliable latency for our datasets that are too large to hold in memory on an RDBMS.

You can get a box with 48TB RAM so 20TB has more to do with time it takes for DB to come up. To provision DynamoDB to the same performance as PG on i3.16xlarge instances you will be paying north of 50K/month vs 5K/month and that's with all limitations that come with DynamoDB.

An i3.16xlarge is 488GB, not 48TB. The x1.32xlarge (largest amazon offers) is 2TB.

Additionally, self-managed PG is a huge operational undertaking compared to DynamoDB. RDS is a bit closer, but the largest RDS offers is db.r3.8xlarge, which is 244GB.

The db.r3.8xlarge on RDS Postgres all in with 3000 provisioned IOPS (just provisioning for writes, all reads should be served from memory) and 1.5TB storage ends up around $6k. A 3 year reservation will get that down to $3k.

Comparable DynamoDB (3000 write units, 3000 read units, reserved IOPS) comes out to $1500.

The key here is consistent performance and low operational overhead. As I said, we're pretty happy with it.

Re: There are very few suitable use cases for DynamoDB

#120
post #98

Earlier quoted context omitted.

We have close to 25M monthly active users and DynamoDB works pretty well for our use case. We store user generated content in DynamoDB. Our largest table is over 1 TB. The mapping of users to their generated content is in Postgres. So doing work on behalf of a particular user will generally be distributed across multiple nodes. We've enjoyed the very predictable performance of DynamoDB as well as the operational simp…

How large is user's generated content in average? Have you considered S3 for your use case? Storage in DDB can be quite pricey (when combining storage cost and and read capacity usage cost)

We are a messaging application, so every chat message is stored in DynamoDB in an ordered stream. We frequently fetch slices of a stream, so the granularity of storage is a single message.

S3 does not do well (cost wise) with billions of files. Provisioning 1000 Write IOPS on Dynamo costs around $100/month. That's 2.5B writes. On S3 that's going to be on the order of $10k. Similarly 1000 Read IOPS on Dynamo is ~$20/month vs 2.5B reads on s3 costing $1000. Storage is cheaper on S3 by about 10x, but per TB that comes out to $250 vs $25, which is hardly a dominating cost factor.

Post reply on HN