Every system I've built on DynamoDB just works. The APIs that use it have had virtually 100% uptime and have never needed database maintenance. It is not a replacement for a RDBMS, but for some use cases, it's a killer service.
DynamoDB 10 years later
41–50 of 225 posts
Re: DynamoDB 10 years later
#42Earlier quoted context omitted.
> The downside of on-demand is the pricing - it's more expensive if you have continuous load. True, although you don't have to make that choice permanently. You can switch from provisioned to on demand once every 24 hours. And you can also set up application autoscaling in provisioned mode, which'll allow you to set parameters under which it'll scale your provisioned capacity up or down for you. This doesn't require…
scaling down is limited to 4x a day
It’s up to 27 times a day, if you time it well: “4 decreases in the first hour, and 1 decrease for each of the subsequent 1-hour windows in a day”.
Re: DynamoDB 10 years later
#43We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…
> We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume.
If possible, put the json in Workers KV, and access it through Cloudflare Workers. You can also optionally cache reads from Workers KV into Cloudflare's zonal caches.
> To be honest, I'd hoped that it could be a bit more 'magic', like S3
You could opt to use the slightly more expensive DynamoDB On-Demand, or the free DynamoDB Auto-Scaling modes, which are relatively no-config. For a very ready-heavy workload, you'd probably want to add DynamoDB Accelerator (an write-through in-memory cache) in front of your tables. Or, use S3 itself (but a S3 bucket doesn't really like when you load it with a tonne of small files) accelerated by CloudFront (which is what AWS Hyperplane, tech underpinning ALB and NLB, does: https://aws.amazon.com/builders-library/reliability-and-cons...)
S3, much like DynamoDB, is a KV store: https://news.ycombinator.com/item?id=11161667 and https://www.allthingsdistributed.com/2009/03/keeping_your_da...
Re: DynamoDB 10 years later
#44We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…
Your impressions are cordect: DynamoDB is quite low-level and more like a DB kit than ready to use DB, for most applications it's better to use something else.
Re: DynamoDB 10 years later
#45The in-the-trenches, technical, battle-tested/battle-scarred comments on this thread are why I come to HN for the comments.
Re: DynamoDB 10 years later
#46Earlier quoted context omitted.
Your example really summarizes the challenge with the AWS paradigm: namely that they want you to believe that the thing to do is to spread the the backend of your application across a large number of distinct data systems. No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable quer…
> they bolt it onto Postgres I am working with a company that is redesigning an enterprise transactional system, currently backed by an Oracle database with 3000 tables. It’s B2B so loads are predictable and are expected to grow no more than 10% per year. They want to use DynamoDB as their primary data store, with Postgres for edge cases it seems to me the opposite would be more beneficial. At what point does DynamoD…
You can make Postgres scale, but there is an operational cost to it. DynamoDB does that for you out of the box. (So does Aurora, to be honest, but there is also an overhead to setting up an Aurora cluster to the needs of your business.)
I've found also that in Postgres the query performance does not keep up with bursts of traffic -- you need to overprovision your db servers to cope with the highest traffic days. DynamoDB, in contrast, scales instantly. (It's a bit more complicated that that, but the effect of it is nearly instantaneous.) And what's really great about DynamoDB is after the traffic levels go down, it does not scale down your table and maintains it at the same capacity at no additional cost to you, so if you receive a burst of traffic at the same throughput, you can handle it even faster.
DynamoDB does a lot of magic under the hood, as well. My favorite is auto-sharding, i.e. it automatically moves your hot keys around so the demand is evenly distributed across your table.
So DynamoDB is pretty great. But to get the the best experience from DynamoDB, you need to have a stable codebase, and design your tables around your access patterns. Because joining two tables isn't fun.
Re: DynamoDB 10 years later
#47We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…
Exactly. This has been my experience with several AWS technologies. Like with their ElasticSearch service, where I had to constantly fine-tune various parameters, such as memory. I was curious why they couldn't auto-scale the memory, why I had to do that manually. There are several AWS services that should be a bit more magical, but they are not.
Re: DynamoDB 10 years later
#48We're at early stages of planning an architecture where we offload pre-rendered JSON views of PostgreSQL onto a key value store optimised for read only high volume. Considering DynamoDB, S3, Elastic, etc. (We'll probably start without the pre-render bit, or store it in PostgreSQL until it becomes a problem). When looking at DynamoDB I noticed that there was a surprising amount of discussion around the requirement for…
I do not recommend starting off with a decision to use DynamoDB before you have worked with it directly for some time to understand it. You could spend months trying to shoehorn your use case into it before realizing you made a mistake. That said, DynamoDB can be incredibly powerful and inexpensive tool if used right.
Re: DynamoDB 10 years later
#49Earlier quoted context omitted.
Your example really summarizes the challenge with the AWS paradigm: namely that they want you to believe that the thing to do is to spread the the backend of your application across a large number of distinct data systems. No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable quer…
If this is not the only option, what would you suggest instead? How to simplify it?
The alternative is to go to GCP and use the big GCP selling point, which is Big Table/Big Query.
Those databases build most of that in, and it's all one fairly excellent distributed monolith.
Re: DynamoDB 10 years later
#50We tried to implement an application on DynamoDB about 2 years ago. We really struggled with implementing adhoc queries/search. For e.g:- select * from employees where name = X and city = Y. Any improvements in DynamoDB that make it easier to implement such queries?
That's not what DynamoDB is for. If you need to run queries like that, you should be using RDBMS. DynamoDB should only really be used for use cases where the queries are known up-front. There are ways to design your data model in Dynamo so that you could actually run queries like that, but you would have had to that work from day 1. You won't be able to retroactively support queries like that.