Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The DynamoDB Paper

brooker.co.za

Post reply on HN
Print view View original

11–20 of 95 posts

Re: The DynamoDB Paper

#11

Post by whalesalad »

ciberado wrote: post #9
magic_hamster wrote: post #8

Earlier quoted context omitted.

If you have a database access layer then structuring your application shouldn't be that different. I wouldn't deal with the database directly unless I had a really good reason or the abstraction layer didn't support the query I was trying to run.

DynamoDB is amazing, but not very flexible once you have designed your database. No abstraction layer will allow you to run queries ad-hoc in a performant way.

It’s true. 400kb max item size, too. 1mb max query size I believe. Good luck grabbing a shit load of data at once without a parallel scan.

Dynamo is a precision tool and it’s great at those specific workloads but it’s not a one size fits all by any means.

Re: The DynamoDB Paper

#12

Post by manigandham »

magic_hamster wrote: post #8
bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

If you have a database access layer then structuring your application shouldn't be that different. I wouldn't deal with the database directly unless I had a really good reason or the abstraction layer didn't support the query I was trying to run.

An access layer doesn't change your access patterns, which is what actually determines the database model to use.

DynamoDB (and other similar key/value stores) make very big trade-offs for speed and scale that most applications don't need.

Re: The DynamoDB Paper

#13

Post by blantonl »

bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

Designing application specifically for DynamoDB will take _a lot_ of time and effort.

If you can write, read, and query a JSON document using an API in your application, it's literally that simple.

The only real time and effort is the architectural decisions you make up front, and that's about it. And there are some great guides out there that cover 99% of those architectural decisions.

As a user of both, I find MySQL replication and clusters to be far more complex and time and effort intensive.

Re: The DynamoDB Paper

#14

Post by manigandham »

fubbyy wrote: post #5
bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

I think it also depends on the system you’re using it on. I think one of the biggest advantages of DDB is that is scales so well (with good design to avoid hot partitions). Afaik, RDBMS simply cannot scale in the same way due to their design. Yes, they can scale somewhat, but as you said it requires lots of tuning, and you’ll still reach a hardish limit.

All databases scale the same way - by partitioning and sharding the dataspace. RDBMS have harder restrictions due to the features they provide and the performance expectations, but you can just as easily use a bunch of relational servers to partition a table (or several) across them by range or hashes of the primary key.

That's basically what key/value stores like DynamoDB do, and why DynamoDB was even built on MySQL (at least originally).

Re: The DynamoDB Paper

#15

Post by Dobbs »

magic_hamster wrote: post #8
bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

If you have a database access layer then structuring your application shouldn't be that different. I wouldn't deal with the database directly unless I had a really good reason or the abstraction layer didn't support the query I was trying to run.

DynamoDB usage is heavily based around correctly structuring your keys. Allowing you to do things like query sub-sets easily. This in turn means you need to know what your usage patterns will be like so you can correctly structure your keys.

God help you if you need to make major changes to this down the road.

Database Access Layer can't do this for you, that just isn't what they do.

Re: The DynamoDB Paper

#16

Post by i_love_limes »

blantonl wrote: post #13
bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

Designing application specifically for DynamoDB will take _a lot_ of time and effort. If you can write, read, and query a JSON document using an API in your application, it's literally that simple. The only real time and effort is the architectural decisions you make up front, and that's about it. And there are some great guides out there that cover 99% of those architectural decisions. As a user of both, I find MySQ…

Have to disagree on this one. Something as basic and out of the box as a migration / data backfill is not only complicated but also very expensive (both time and cost wise) on Dynamo. Not to mention all the other things that come nicely with an relational db (type checking, auto increments, uniform data)

Re: The DynamoDB Paper

#17

Post by kdazzle »

blantonl wrote: post #13
bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

Designing application specifically for DynamoDB will take _a lot_ of time and effort. If you can write, read, and query a JSON document using an API in your application, it's literally that simple. The only real time and effort is the architectural decisions you make up front, and that's about it. And there are some great guides out there that cover 99% of those architectural decisions. As a user of both, I find MySQ…

> The only real time and effort is the architectural decisions you make up front, and that's about it

And dont forget about the time spent fixing what could have been caught by types and regular old db constraints (for most applications)

Re: The DynamoDB Paper

#18

Post by ramraj07 »

bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

Could you elaborate on your (or a hypothetical) use case where dynamo db makes sense? I for one can never come up with something better served by rdbms or s3.

Re: The DynamoDB Paper

#19

Post by eatonphil »

ciberado wrote: post #9
magic_hamster wrote: post #8

Earlier quoted context omitted.

If you have a database access layer then structuring your application shouldn't be that different. I wouldn't deal with the database directly unless I had a really good reason or the abstraction layer didn't support the query I was trying to run.

DynamoDB is amazing, but not very flexible once you have designed your database. No abstraction layer will allow you to run queries ad-hoc in a performant way.

> No abstraction layer will allow you to run queries ad-hoc in a performant way.

Depends on the size of the data. Run analytics queries (i.e. things that return summary data not all rows) on 10GB of data through clickhouse or duckdb or datafusion and they'll generally return in milliseconds.

Re: The DynamoDB Paper

#20

Post by cebert »

codetiger wrote: post #4
bistablesulphur wrote: post #2

I'be been working with DynamoDB daily for a few years now, and whilst I like working with it and the specific scenario it solves for us, I'd still urge anyone thinking about using it to carefully reconsider whether their problem is truly unique enough that a traditional RDBMS couldn't handle it with some tuning. Theycan be unbelievably performant and give so much stuff for free. Designing application specifically for…

Is there a specific reason why you say "Designing application specifically for DynamoDB will take _a lot_ of time and effort". Are you talking about migrating from RDBMS to DynamoDB? Coz, my experience with DynamoDB designing was very similar to any other NoSQL DB.

You really need to consider you access patterns up front with DynamoDB. Any changes of those during application development can be very time consuming. There are limitations on how many local and global secondary indexes you can have. You also can’t easily add them to existing tables. However, you can use multiple databases to get the best of both worlds. At my employer, we typically store domain entities in DynamoDB as the source of truth. However, we replicate some entities to secondary databases like OpenSearch when we have access patterns that require adhoc querying.
Post reply on HN