Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon DynamoDB Transactions

aws.amazon.com

Post reply on HN
Print view View original

21–30 of 77 posts

Re: Amazon DynamoDB Transactions

#21

Post by scarface74 »

abalone wrote: post #11

Any experience with using Aurora in place of DynamoDB? A couple years ago there was an interesting tidbit at re:Invent about customers moving from DynamoDB to Aurora to save significant costs.[1] The Aurora team made the point that DynamoDB suffers from hotspots despite your best efforts to evenly distribute keys, so you end up overprovisioning. Whereas with Aurora you just pay for I/O. And the scalability is great.…

Whether NoSQL is the best approach and whether DynamoDB is the best approach are two separate issues. I find DynamoDB too limiting with the way that it handles indexing, read and write capacity, etc. compared to traditional NoSQL databases like ElasticSearch and Mongo.

That being said, one advantage of DynamoDB is that it is API based and you can make a true serverless web app where all of the logic is on the client, you use Web Federation for authentication to DynamoDB, and you host your JavaScript files, html and CSS on S3.

Another advantage until two days ago, was that with most of the data stores on AWS, you kept your databases behind a VPC and if you used lambda, your lambda also had to be in a VPC and that increased warm up time for the lambda.

Now, there is the Read Only Data API for serverless Aurora. You don’t have to worry about the traditional connection pooling or being in a VPC.

Re: Amazon DynamoDB Transactions

#22

Post by paragraft »

abalone wrote: post #17
coder543 wrote: post #14

Earlier quoted context omitted.

I personally recommend using a SQL database until you're absolutely positively sure you don't need one, for many reasons. But, as far as the "you end up overprovisioning" because of hotspots thing, DynamoDB does offer autoscaling these days, which should alleviate a lot of provisioning-related headaches and save you money compared to the provisioning you would have done with DynamoDB, from what I understand.

> DynamoDB does offer autoscaling these days, which should alleviate a lot of provisioning-related headaches The problem they noted isn't lack of autoscaling, it's that you have to provision the entire datastore to accommodate your hottest partition.

GP used the wrong term, think they meant adaptive capacity, which is a newer feature where shards will automatically lend capacity to each other in the case of hotspots.

Re: Amazon DynamoDB Transactions

#24

Post by abalone »

awinder wrote: post #19
abalone wrote: post #11

Any experience with using Aurora in place of DynamoDB? A couple years ago there was an interesting tidbit at re:Invent about customers moving from DynamoDB to Aurora to save significant costs.[1] The Aurora team made the point that DynamoDB suffers from hotspots despite your best efforts to evenly distribute keys, so you end up overprovisioning. Whereas with Aurora you just pay for I/O. And the scalability is great.…

https://aws.amazon.com/blogs/database/how-amazon-dynamodb-ad...

Oh cool. For those reading along this is titled "How Amazon DynamoDB adaptive capacity accommodates uneven data access patterns (or, why what you know about DynamoDB might be outdated)". Is this a new feature?

Re: Amazon DynamoDB Transactions

#25

Post by orthecreedence »

coder543 wrote: post #14
abalone wrote: post #11

Any experience with using Aurora in place of DynamoDB? A couple years ago there was an interesting tidbit at re:Invent about customers moving from DynamoDB to Aurora to save significant costs.[1] The Aurora team made the point that DynamoDB suffers from hotspots despite your best efforts to evenly distribute keys, so you end up overprovisioning. Whereas with Aurora you just pay for I/O. And the scalability is great.…

I personally recommend using a SQL database until you're absolutely positively sure you don't need one, for many reasons. But, as far as the "you end up overprovisioning" because of hotspots thing, DynamoDB does offer autoscaling these days, which should alleviate a lot of provisioning-related headaches and save you money compared to the provisioning you would have done with DynamoDB, from what I understand.

We use a hybrid. We process a lot of incoming data and dump most of it into dynamo (it's ephemeral so the TTL feature is nice) and if we get capacity errors (Dynamo takes a while to scale up sometimes) we just dump our objects in the DB. The end result is we keep a huge amount of writes off our DB for processing incoming largish objects. The amount of data it stores would cost an arm and a leg to put into redis.

Granted, I don't think I'd want to use Dynamo for anything other than temporary data. Lock-in makes me nervous, and the way it scales up/down really makes it difficult to use it for hourly workloads...by the time it scales up we're close to done needing more capacity, then it doesn't scale down for like 40m after. We set up caps and the DB overflow machanism keeps things from grinding to a halt.

Re: Amazon DynamoDB Transactions

#26

Post by monstrado »

otterley wrote: post #23
ryanworl wrote: post #10

“DynamoDB is the only non-relational database that supports transactions across multiple partitions and tables.” Uh... this is just not true.

Can you identify some others?

FoundationDB https://www.foundationdb.org/ not only supports transactions, they are mandatory. They also go one step further and support atomic operations, which are especially killer.

Re: Amazon DynamoDB Transactions

#27

Post by awinder »

abalone wrote: post #24
awinder wrote: post #19

Earlier quoted context omitted.

https://aws.amazon.com/blogs/database/how-amazon-dynamodb-ad...

Oh cool. For those reading along this is titled "How Amazon DynamoDB adaptive capacity accommodates uneven data access patterns (or, why what you know about DynamoDB might be outdated)". Is this a new feature?

Yeah I should have elaborated a bit. I believe adaptive capacity was announced at re:invent in 2017 and may have released shortly after / maybe early 2018. The feature is getting a lot more press & push from AWS lately though for sure.

Re: Amazon DynamoDB Transactions

#28

Post by stickfigure »

otterley wrote: post #23
ryanworl wrote: post #10

“DynamoDB is the only non-relational database that supports transactions across multiple partitions and tables.” Uh... this is just not true.

Can you identify some others?

The Google Cloud Datastore (formerly the "App Engine Datastore") has had cross-entity-group transactions since 2011:

https://googleappengine.blogspot.com/2011/10/app-engine-155-...

Re: Amazon DynamoDB Transactions

#29

Post by gmaster1440 »

otterley wrote: post #23
ryanworl wrote: post #10

“DynamoDB is the only non-relational database that supports transactions across multiple partitions and tables.” Uh... this is just not true.

Can you identify some others?

MongoDB (https://www.mongodb.com/transactions)

“Multi-document transactions can be used across multiple operations, collections, databases, and documents.”

Re: Amazon DynamoDB Transactions

#30

Post by psankar »

Postgresql gets native JSON support (at least since 9.2 onwards) to store schemaless free flowing text. Dynamodb gets transaction guarantees.

There is globalization and intermingling happening on technology too.

On a similar thought, a few years back, C# and Java got `Any` generic types, while Python/JS got static types (via python3 typings, typescript)

Post reply on HN