There are very few suitable use cases for DynamoDB • Hacker News Board

Re: There are very few suitable use cases for DynamoDB

#31

Post by choward » Sat, Jul 08, 2017, 1:48 AM UTC

throwaway91111 wrote: post #25

brandur wrote: post #8

I've seen two large usages of DynamoDB at two different companies, and for what it's worth, in both cases we've had similar trouble as the author. In one case we ended up ripping it out and moving to a sharded Postgres scheme, and in the other we've left in place for now because a migration will be such a monumental effort, but it's pretty much universally maligned. Fundamentally, the problem seems to be that choosin…

Why on earth don't they hash the key to improve the key distribution?

How would that help? The same user id will still hash to the same thing.

Re: There are very few suitable use cases for DynamoDB

#32

Post by ryanworl » Sat, Jul 08, 2017, 1:50 AM UTC

lwansbrough wrote: post #29

ryanworl wrote: post #28

Earlier quoted context omitted.

Each shard is 10GB. The unit of scale is a shard. You can have as many shards as you want.

It's one shard per node though, right? So you're talking about SERIOUS cost when you want to store _actual_ big data.

You don't pay per node. In fact, the concept of a node is not exposed at all. Each shard is technically on three nodes for high availability. You pay for provisioned capacity and data storage per GB.

Re: There are very few suitable use cases for DynamoDB

#33

Post by jey » Sat, Jul 08, 2017, 1:52 AM UTC

So it's fine as long as I precompute the full transaction to apply to the key, then do it all at once? I expect large overall transaction volumes but am happy to only issue one DB change per user interaction.

Re: There are very few suitable use cases for DynamoDB

#34

Post by sheeshkebab » Sat, Jul 08, 2017, 1:56 AM UTC

Dynamodb has a nice js interface and is a good helper store for various small aws specific automation projects.

It's not a good choice for various read heavy enterprise apps (and frankly for write ones neither). It also doesn't scale - it doesn't even have a cross region active/active support (highly surprising for a key value store when cassandra supported it for ages).

Don't use it for anything serious.

Re: There are very few suitable use cases for DynamoDB

#35

Post by thinkloop » Sat, Jul 08, 2017, 2:11 AM UTC

kevan wrote: post #21

>This will not work at scale in DynamoDb. I don't think we're getting the whole story from the author. I'm not the biggest fan of Dynamo either for reasons I won't get into here, but this type of workload is exactly what Dynamo was built for: serving a website with millions of customers. Disclaimer: I work at Amazon, my views are my own.

So what about the issue of requiring heavy over-provisioning for common usage patterns, like a customer being logged in causing temporary bursts of access on specific keys?

Re: There are very few suitable use cases for DynamoDB

#36

Post by axaxs » Sat, Jul 08, 2017, 2:18 AM UTC

jsemrau wrote: post #2

When I started building my first app in 2011 MongoDB was the rage. So I build the back-end using the futuristic 'No-SQL' technology. It turned out to be slow (~1 min Query time), inconsistent, and missing an RDBMS layer. Move the thing to PHP/Mysql problems were gone. I still have not found a use case outside of web (comments/discussion) sites where the high integration with Javascript actually makes sense.

As someone being 'forced' to use Mongo for the first time, I'm curious as to when/where you hit that? I'm looking for pros/cons, but the internet is proving rather unhelpful as it seems to be people vehemently for it or against it.

Re: There are very few suitable use cases for DynamoDB

#37

Post by dijit » Sat, Jul 08, 2017, 2:19 AM UTC

jethro_tell wrote: post #23

dijit wrote: post #4

Earlier quoted context omitted.

I hope you don't mind if I piggyback on this to echo this sentiment. Although I dislike MySQL for its many gotchas (data corruption level stuff too!) I was looking for a _long_ _long_ time for a high consistency NoSQL database.. we basically need document storage of large binary data. Ironically literally nothing in NoSQL land does write-through to disk, they just write to vfs and hope it works; additionally, those t…

Isn't that the point of no_sql. You trade consistency for speed. Best used to mark temporary state in a web app. If you need atomic consistency in a database you should use a relational database that does atomic writes. That has always been the case. Why would it be so shocking to you? Atomic writes have already been built. Sounds like you were standing there with a hammer looking for a nail.

I always thought the term "document store" just meant arbitrary unstructured data that didn't need to be related.

That's how you do things like we did regarding deterministic sharding- it's very easy to scale this way if you don't ever "JOIN" tables etc;

Re: There are very few suitable use cases for DynamoDB

#38

Post by mrep » Sat, Jul 08, 2017, 2:22 AM UTC

thinkloop wrote: post #35

kevan wrote: post #21

>This will not work at scale in DynamoDb. I don't think we're getting the whole story from the author. I'm not the biggest fan of Dynamo either for reasons I won't get into here, but this type of workload is exactly what Dynamo was built for: serving a website with millions of customers. Disclaimer: I work at Amazon, my views are my own.

So what about the issue of requiring heavy over-provisioning for common usage patterns, like a customer being logged in causing temporary bursts of access on specific keys?

Aka, the status quo before AWS allowed you to dynamically scale certain parts of your infrastructure. It's not perfect for all parts of your stack, but do you have a better alternative for databases?

Re: There are very few suitable use cases for DynamoDB

#39

Post by coredog64 » Sat, Jul 08, 2017, 2:27 AM UTC

banku_brougham wrote: post #22

The first part of the article sounds convincing, but i'm pretty sure amazon is using DynamoDB extensively and successfully at massive scales.

Does the internal charge back model for Dynamo work the same as it does for AWS customers?

If I understand the author's point, you _can_ get great performance, but it requires over provisioning. So it's not so much a technical failing as much as it is a poor monetisation of the capability.

I see that elsewhere in AWS: They require huge instance types for large EBS volumes with ES. For our workload, said instances barely break a sweat.

Re: There are very few suitable use cases for DynamoDB

#40

Post by threeseed » Sat, Jul 08, 2017, 2:32 AM UTC

jsemrau wrote: post #2

When I started building my first app in 2011 MongoDB was the rage. So I build the back-end using the futuristic 'No-SQL' technology. It turned out to be slow (~1 min Query time), inconsistent, and missing an RDBMS layer. Move the thing to PHP/Mysql problems were gone. I still have not found a use case outside of web (comments/discussion) sites where the high integration with Javascript actually makes sense.

Why did you use MongoDB if your domain model wasn't suited ?

I just don't understand people who complain about a technology and say it is useless for all of these use cases when they couldn't even spend a few hours to do a Spike/POC or some basic data domain design. MongoDB has very clear documentation about what you should or shouldn't use it for.

MongoDB is unique in that it is one of the few document stores available today. So if you have use cases such as 360 Customer View or where you need to fetch a lot of nested information with a single id it is blisteringly fast.

If you have a relational data model than use a relational database.