Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DynamoDB 10 years later

amazon.science

Post reply on HN
Print view View original

211–220 of 225 posts

Re: DynamoDB 10 years later

#211

Post by WatchDog »

greiskul wrote: post #154
AtlasBarfed wrote: post #134

Earlier quoted context omitted.

So they just rolled out global replication, and I can't for the life of me figure out how they resolve write conflicts without cell timestamps or any other obvious CRDT measures. Questions were handwaved away, and the usual Amazon black box non-answers which always smells like they are hiding problems. Any ideas how this is working? It seems bolt-on and not well thought out, and I doubt they'll ever pay for Aphyr to…

From: https://aws.amazon.com/dynamodb/global-tables/ Consistency and conflict resolution Any changes made to any item in any replica table are replicated to all the other replicas within the same global table. In a global table, a newly written item is usually propagated to all replica tables within a second. With a global table, each replica table stores the same set of data items. DynamoDB does not support partial…

So a write than doesn’t “win” just gets silently discarded?

Re: DynamoDB 10 years later

#212

Post by WatchDog »

jerf wrote: post #59
redwood wrote: post #18

Earlier quoted context omitted.

Your example really summarizes the challenge with the AWS paradigm: namely that they want you to believe that the thing to do is to spread the the backend of your application across a large number of distinct data systems. No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable quer…

It seems to me that what this is saying is that storage has become so cheap that if another database provides even slight advantages over another for some workload it is likely to be deployed and have all the data copied over to it. HN entrepreneurs take note, this also suggests to me that there may be a market for a database (or a "metadatabase") that takes care of this for you. I'd love to be able to have a "relati…

My understanding of the cockroach db architecture, it that it’s essentially two discrete components, a key value store that actually persists the data, and a SQL layer built on top. Although I don’t think it’s recommended or supported to access the key value store directly.

Re: DynamoDB 10 years later

#213

Post by jd_mongodb »

leetrout wrote: post #202
throwdbaaway wrote: post #194

Earlier quoted context omitted.

Interesting. MongoDB actually came to mind while I was reading the other comment here: > No one uses DynamoDB alone: they bolt it onto Postgres after realizing they have availability or scale needs beyond what a relational database can do, then they bolt on Elasticsearch to enable querying, and then they bolt on Redis to make the disjointed backend feel fast. And I'm just talking operational use cases; ignoring analy…

Not snark: did MongoDB ever go away? I've seen it used in many places over the years. Today I would choose JSON in Postgres before I would just jump to Monogo but it certainly serves a purpose for many shops and it is still widely used AFAIK. I _really_ miss RethinkDB.

What do you miss about RethinkDB?

Re: DynamoDB 10 years later

#214

Post by robertlagrant »

> Customers no longer want to just store and query the data in their databases. They then want to analyze that data to create value

Presumably create more value. I know it's a marketing post but still - storing and retrieving things is pretty valuable in and of itself.

Re: DynamoDB 10 years later

#215

Post by rhoulihan »

awsthro00945 wrote: post #77
gigatexal wrote: post #28

Earlier quoted context omitted.

At his rank at AWS I don’t know if money was such an issue. He strikes me as a person who cares deeply about the underlying tech. But I have no idea one way or the other.

I think I've seen you post something similar on r/aws about how Rick was "top DynamoDb person at AWS" (apologies if that wasn't you). I think you are overestimating Rick's "rank". I just looked him up (I had not heard of him before seeing his name mentioned on r/aws a few days ago) and he was an L7 TPM/Practice Manager in AWS's sales organization. That's not really a notably high position, and in the grand scheme of…

Actually, I was a direct report to Colin Lazier (https://twitter.com/clazier) who is the GM for DynamoDB, Keyspaces, and Glue Elastic Views. I was the original TPM for DocumentDB before joining the Professional Services team as a Senior Practice Manager to head up the NoSQL Blackbelt team which led the archtecture/design effort for Amazon's RDBMS->NoSQL migration. I was brought back to the service team by Jim Scharf to lead the technical solutions team for strategic accounts, but I maintained the org chart role of Senior Practice Manager until I left for MongoDB.

Compensation was a minor issue. I was an org chart aberration already and AWS pulled out all the stops to retain me. I will always appreciate the opportunity that AWS provided me and my time at DynamoDB will always hold a special place in my heart. I really do believe that MongoDB is poised to do great things and my decision had more to do with being a part of that than anything else.

Re: DynamoDB 10 years later

#216

Post by voxic11 »

didip wrote: post #101

To be honest, as a customer, it is hard for me to justify using DynamoDB. Some of this criticism can be out of date: 1. DynamoDB is not as convenient. There are a bit too many dials to turn. 2. DynamoDB does not have a SQL facade on top. 3. DynamoDB is proprietary, I believe there's no OSS API equivalent if you want to migrate out. 4. DynamoDB was kind of expensive. But it has been a while since I last check the pric…

> 2. DynamoDB does not have a SQL facade on top.

I think this is out of date https://aws.amazon.com/about-aws/whats-new/2020/11/you-now-c...

Re: DynamoDB 10 years later

#217

Post by garethmcc »

amzn-throw wrote: post #149
belter wrote: post #124

Earlier quoted context omitted.

You never heard of Rick Houlihan? He is the 90% of DynamoDB Evangelism... At the same time you are able to this internal lookups? Do you work with DynamoDB? AWS re:Invent 2018: Amazon DynamoDB Deep Dive: Advanced Design Patterns for DynamoDB (DAT401) https://youtu.be/HaEPXoXVf2k AWS re:Invent 2019: [REPEAT 1] Amazon DynamoDB deep dive: Advanced design patterns (DAT403-R1) https://youtu.be/6yqfmXiZTlM AWS re:Invent 20…

Do you expect the engineers on your team to know the top sales person at your company? This person might be responsible for the majority of evangelism and revenue for the company. Do you expect the SDEs to know about him? Again, no shot against against Rick - he is amazing, smart, technical, competent, and a deep owner. But the average SDE on the team won't know about these or watch these talks. There are too many de…

I have watched almost all those talks as they are technically dense and full of very good and very useful technical knowledge that I would be much poorer for not watching. These are not sales videos but highly complex instructional content meant for developers on the ground

Re: DynamoDB 10 years later

#218

Post by 0xbadcafebee »

AtlasBarfed wrote: post #208
0xbadcafebee wrote: post #161

Earlier quoted context omitted.

They did have 100 million to burn, but my mostly-wild-guess is it was closer to $1.5M/yr. But that gives you an in-house SaaS DB used across a hundred other teams/products/services, so it actually saved money (and nothing else matched its performance/CAP/functionality). Cassandra is too opinionated and its CAP behavior wasn't great for a service like this, so they built on top of Riak. (This also eliminated any thoug…

Because Riak uses vector clocks instead of cell timestamps? Cassandra's ONE/QUORUM/ALL consistency levels otherwise allow tuning for tolerance of CP vs AP, don't they?

To be honest I don't know, I wasn't there for the initial decision, but I know it wasn't just about CAP. It could have been as simple as Riak was easier to use (which I don't know either)

Re: DynamoDB 10 years later

#219

Post by wnolens »

0xbadcafebee wrote: post #163
wnolens wrote: post #147

Earlier quoted context omitted.

this is the answer. source: currently being burned out on an adjacent aws team..

That sucks, man. If they won't move you to another team, just get out of there. We don't benefit by suffering for them, and they're not gonna change.

Yea. I can probably move to a more chill team, but I wouldn't work on anything nearly as cutting edge. I mentally check out for weeks at a time, then get back into it and deliver something large. I'm low key job hunting, but don't entirely trust that it'll be different anywhere else (previous jobs were like this too)

Re: DynamoDB 10 years later

#220

Post by gigatexal »

rhoulihan wrote: post #215
awsthro00945 wrote: post #77

Earlier quoted context omitted.

I think I've seen you post something similar on r/aws about how Rick was "top DynamoDb person at AWS" (apologies if that wasn't you). I think you are overestimating Rick's "rank". I just looked him up (I had not heard of him before seeing his name mentioned on r/aws a few days ago) and he was an L7 TPM/Practice Manager in AWS's sales organization. That's not really a notably high position, and in the grand scheme of…

Actually, I was a direct report to Colin Lazier ( https://twitter.com/clazier ) who is the GM for DynamoDB, Keyspaces, and Glue Elastic Views. I was the original TPM for DocumentDB before joining the Professional Services team as a Senior Practice Manager to head up the NoSQL Blackbelt team which led the archtecture/design effort for Amazon's RDBMS->NoSQL migration. I was brought back to the service team by Jim Schar…

Whoa straight from the source!
Post reply on HN