Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are very few suitable use cases for DynamoDB

syslog.ravelin.com

Post reply on HN
Print view View original

101–110 of 146 posts

Re: There are very few suitable use cases for DynamoDB

#101

Post by dserban »

My background is in Cassandra and one company I worked for last year insisted that we use DynamoDB for a project.

Here are a few things that ended up being show stoppers.

1. Both the partition key and the sort key are capped at 1 field. In an attempt to "think Cassandra data model", the ugly workaround was to stringify and concatenate things at the application layer, then parse / split on the other side. This made the code unreadable.

2. DynamoDB-Spark integration is a second-class citizen. (Cassandra-Spark integration is first-class and well-maintained.)

3. The other thing that made code unreadable was the accidental complexity introduced by exception handling / exponential backoff we needed to implement to protect against accidental read capacity underprovisioning.

Although I made repeated pleas to switch to Cassandra, the (non-technical) CEO insisted that we keep using DynamoDB. I'm no longer at that company but I hear they have meanwhile switched to RedShift.

Re: There are very few suitable use cases for DynamoDB

#102

Post by qaq »

phamilton wrote: post #74
qaq wrote: post #55

Earlier quoted context omitted.

when your data set grows beyond what can reasonably stored in RDBMS e.g. 20+ TB

20TB in an RDBMS has very different characteristics than 100GB in an RDBMS. Once you can no longer store your dataset in memory, you start to see greater variance in latency. DynamoDB for us has given us very reliable latency for our datasets that are too large to hold in memory on an RDBMS.

You can get a box with 48TB RAM so 20TB has more to do with time it takes for DB to come up. To provision DynamoDB to the same performance as PG on i3.16xlarge instances you will be paying north of 50K/month vs 5K/month and that's with all limitations that come with DynamoDB.

Re: There are very few suitable use cases for DynamoDB

#103

Post by luhn »

I took a look at BigTable, as recommended by the article, because I was evaluating DynamoDB myself just yesterday. It looks like the minimum price for that is ~$1500 a month. Granted, you're getting what you pay for (3 nodes that support 10,000QPS each), but the pricing is out of reach for smaller projects.

Re: There are very few suitable use cases for DynamoDB

#105

Post by 5ersi »

luhn wrote: post #103

I took a look at BigTable, as recommended by the article, because I was evaluating DynamoDB myself just yesterday. It looks like the minimum price for that is ~$1500 a month. Granted, you're getting what you pay for (3 nodes that support 10,000QPS each), but the pricing is out of reach for smaller projects.

You should take a look at Google Cloud Database which is a better option and has per-operation pricing instead of per-node pricing.

Re: There are very few suitable use cases for DynamoDB

#106

Post by luhn »

5ersi wrote: post #105
luhn wrote: post #103

I took a look at BigTable, as recommended by the article, because I was evaluating DynamoDB myself just yesterday. It looks like the minimum price for that is ~$1500 a month. Granted, you're getting what you pay for (3 nodes that support 10,000QPS each), but the pricing is out of reach for smaller projects.

You should take a look at Google Cloud Database which is a better option and has per-operation pricing instead of per-node pricing.

That is very attractive pricing. It looks like you don't even need to preallocate capacity, just read and write as you please.

Re: There are very few suitable use cases for DynamoDB

#107

Post by pishpash »

NoSQL should stay as a quick and dirty solution for storing key-values. When you start to schematize it and use it as a "DB," you are going down the wrong path. That is because NoSQL is a glorified cache, not a DB. It is essentially using the memory on a large number of nodes to buffer bursty throughput, and using background processes to collate the data later onto disk. There is almost no case where an explicit distributed caching or queuing solution backed by a traditional DB isn't strictly better.

Re: There are very few suitable use cases for DynamoDB

#108

Post by pishpash »

dserban wrote: post #101

My background is in Cassandra and one company I worked for last year insisted that we use DynamoDB for a project. Here are a few things that ended up being show stoppers. 1. Both the partition key and the sort key are capped at 1 field. In an attempt to "think Cassandra data model", the ugly workaround was to stringify and concatenate things at the application layer, then parse / split on the other side. This made th…

Cassandra isn't any better in those regards. For 1, Cassandra just does the concatenation under the hood and 3 happens all the time on Cassandra. The broken thing is using NoSQL as a DB, not Cassandra or DynamoDB.

Re: There are very few suitable use cases for DynamoDB

#109

Post by ZGF4 »

sametmax wrote: post #91
ZGF4 wrote: post #79

There are a few misguided views in this article and in some of these comments. 1. Every shardable database (Cassandra, Dynamo, BigTable) has to worry about hot spots. Picking a UUID as a partition key is only step one. What happens if one user is a huge majority of your traffic? All of their reads/writes are going to a single partition and of course you are going to suffer from performance issues from that hot spot.…

It depends a lot of the write. If they can be batched, you can put them in a queue or in redis until it reach a threshold and write the update down in the RDBMS. It won't work for all the use cases, but more often that people think.

Yes microbatching is a great way to get a fixed write rate regardless of traffic, can even do it right at the application layer. The trade-off is that you can theoretically lose one interval of data when your service goes down. This might not matter for analytics workloads with a margin of error but some usecases require confirmed writes

Re: There are very few suitable use cases for DynamoDB

#110

Post by krallja »

TheAceOfHearts wrote: post #53

Many of the comments here are saying that the author's use-case wasn't a good one for DynamoDB. Can anyone share some simple approachable resources that talk about the kinds of use-cases where these tools make sense? Whenever I read about NoSQL systems, I'm always left a unsure about its use-cases. I've only worked on systems where a traditional RDBMS made the most sense. How do you identify when it's appropriate to…

> How do you identify when it's appropriate to reach for one of the many NoSQL tools?

Despite the flag on the post, this is a pretty good summary of why you might choose one of several: https://news.ycombinator.com/item?id=14697230

Post reply on HN