Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There are very few suitable use cases for DynamoDB

syslog.ravelin.com

Post reply on HN
Print view View original

51–60 of 146 posts

Re: There are very few suitable use cases for DynamoDB

#51

Post by Something1234 »

TheAceOfHearts wrote: post #44
ryandvm wrote: post #20

Earlier quoted context omitted.

I would imagine the use of AWS Lambda might be a pretty decent incentive to use DynamoDB. I'm guessing that stateless DynamoDB queries are faster to fire off than dealing with Postgres or MySQL connection setup and teardown.

I'm not familiarized with DynamoDB, but why would connecting to it be any faster than connecting to Postgres or MySQL?

Perhaps DynamoDB is based around HTTP, rather then requiring socket back and forth actions(uname + pass, auth, query, results vs just a single hit where query + auth details are packed together). Never actually looked at dynamodb, so not sure.

Re: There are very few suitable use cases for DynamoDB

#52

Post by cyberferret »

Hmm... We dabbled with DynamoDB on a couple of very small projects, but found some real oddities in how it stores JSON data, and manages keys. Querying the dataset was also a mammoth, frustrating task.

Ended up switching to RethinkDB and haven't looked back - far better query syntax/language (REQL) and we can organise the JSON content just how we want it.

Re: There are very few suitable use cases for DynamoDB

#53

Post by TheAceOfHearts »

Many of the comments here are saying that the author's use-case wasn't a good one for DynamoDB. Can anyone share some simple approachable resources that talk about the kinds of use-cases where these tools make sense?

Whenever I read about NoSQL systems, I'm always left a unsure about its use-cases. I've only worked on systems where a traditional RDBMS made the most sense. How do you identify when it's appropriate to reach for one of the many NoSQL tools?

I've had the suspicion that many applications that leverage NoSQL tools are usually used in conjunction with a relational database, and not in isolation. Based on my limited understanding, I can at least wrap my head around a few ways in which this could probably help. Am I off the mark here? One of the points I struggle with is that once you're storing data across multiple data stores, maintaining data integrity becomes much harder.

Re: There are very few suitable use cases for DynamoDB

#54

Post by qaq »

I can see the future:

article that outlines gotchas of Bigtable

article you probably shouldn’t use Bigtable

article the amount of money we could've saved using PG and not rewriting things 3 times.

anything up to 10-15TB there are very few reasons not to use something like PG

Re: There are very few suitable use cases for DynamoDB

#55

Post by qaq »

TheAceOfHearts wrote: post #53

Many of the comments here are saying that the author's use-case wasn't a good one for DynamoDB. Can anyone share some simple approachable resources that talk about the kinds of use-cases where these tools make sense? Whenever I read about NoSQL systems, I'm always left a unsure about its use-cases. I've only worked on systems where a traditional RDBMS made the most sense. How do you identify when it's appropriate to…

when your data set grows beyond what can reasonably stored in RDBMS e.g. 20+ TB

Re: There are very few suitable use cases for DynamoDB

#57

Post by idunno246 »

throwaway91111 wrote: post #25
brandur wrote: post #8

I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned. Fundamentally, the problem seems to be that choosin…

Why on earth don't they hash the key to improve the key distribution?

they do, thats why the partition key used to be called the hash key

Re: There are very few suitable use cases for DynamoDB

#58

Post by Jach »

choward wrote: post #31
throwaway91111 wrote: post #25

Earlier quoted context omitted.

Why on earth don't they hash the key to improve the key distribution?

How would that help? The same user id will still hash to the same thing.

I think I had the same confusion after reading the GP's last statement. The idea of hashing is that the resulting number will be uniformly distributed across the hash space, as opposed to monotonically increasing bit by bit like an auto-increment or timestamp, but it's not the partition key value itself that needs to be uniformly distributed but your access patterns of the partitions. As the docs note, a device id (and also a user id) could be good or bad to use as a partition key, it depends on the read/write patterns given the key.

Re: There are very few suitable use cases for DynamoDB

#59

Post by pmohan6 »

lwansbrough wrote: post #29
ryanworl wrote: post #28

Earlier quoted context omitted.

Each shard is 10GB. The unit of scale is a shard. You can have as many shards as you want.

It's one shard per node though, right? So you're talking about SERIOUS cost when you want to store _actual_ big data.

DynamoDB is a multi-tenant service. There is no dedicated node for you. Each shard is replicated across 3 nodes and those nodes contain replicas of other shards/tables.

Re: There are very few suitable use cases for DynamoDB

#60

Post by imauld »

throwaway91111 wrote: post #25
brandur wrote: post #8

I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned. Fundamentally, the problem seems to be that choosin…

Why on earth don't they hash the key to improve the key distribution?

They do.From their getting started docs:

> DynamoDB uses the partition key value as input to an internal hash function; the output from the hash function determines the partition where the item is stored.

Post reply on HN