Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon DocumentDB, with MongoDB compatibility

aws.amazon.com

Post reply on HN
Print view View original

271–280 of 323 posts

Re: Amazon DocumentDB, with MongoDB compatibility

#271

Post by talawahtech »

mrep wrote: post #167
talawahtech wrote: post #126

Earlier quoted context omitted.

It sounds like it is built on top of the Aurora storage subsystem that is used by both Aurora MySQl and Aurora Postgres[1]. I kinda expected them to build it on top of DynamoDb's backend and provide the same kind of "Serverless" on demand experience, but I guess the architecture didn't fit, or maybe this was just faster. 1. https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

Definitely because it was faster. Amazon's strategy is to launch new features ASAP and then rely on everyone having to be on-call to fix shit when it inevitably breaks in prod because they rushed to launch. I will admit that while their "operational excellence" is shit, the security engineers do have quite a bit of power to block launches so their security isn't as bad as the reliability. However, the fact that write…

It definitely sounds like it sucks from the perspective of an internal AWS developer or SRE, but if the AWS systems are architected such that these internal failures aren't seen by end users then AWS's reliability reputation remains fully intact.

Customers are paying AWS so that their SREs don't get called, they don't care if the AWS SREs do as long as the system keeps running.

Based on the supporting quotes at launch from Capital One, Dow Jones and WaPo it sound like enough customers are ok with vertical write scalability and (pretty awesome) horizontal read scalability for now because it fits their use case and is better than what they had before.

Also consider that since the cluster management overhead has been removed from the customer, they can essentially "shard" by using a separate cluster for each sufficiently large service/org/dept, which might actually work out better for them in some respects.

Perfect is the enemy of good enough, the architecture might be laughable to you, but it is probably miles ahead of what the customer was using before.

Re: Amazon DocumentDB, with MongoDB compatibility

#272

Post by talawahdotnet »

adaptiveValleys wrote: post #251
senderista wrote: post #182

Earlier quoted context omitted.

I was there for the launch of a major AWS service where they had an entire separate team working on the next iteration since well before launch (because the initial design wasn’t even intended to be sustainable). They are happy to incur technical risk (and in this case, to eat major losses in hardware costs) in order to be first to market.

I'm sure they are, so they a pass those lovely negative externalities onto the customer because they know it's in demand and only they provide that service. If only they had a competitor that could launch the same products a few months later but offered higher reliability off the bat, that could eventually force Amazon to improve their reliability or risk losing customers long term. Being first to market doesn't ensu…

You think AWS has a reliability problem for their database products? That's news to me. AWS often launches products with limited features, but security, durability and reliability tend to be the standard.

Reliability is the trickiest of the three because it requires the customer to architect their solution with multi-AZ support in mind, but AWS always provides the foundation for that architecture.

Could they, and should they provide more features and a better developer experience around building fault tolerant solutions? Absolutely! But I certainly don't think they have a bad reputation for reliability.

Re: Amazon DocumentDB, with MongoDB compatibility

#274

Post by bnjmn »

Looking through the supported APIs (https://docs.aws.amazon.com/documentdb/latest/developerguide...), it appears DocumentDB has no support for Mongo's oplog (https://docs.mongodb.com/manual/core/replica-set-oplog/), or change streams (https://docs.mongodb.com/manual/changeStreams), which I guess is no surprise because change streams were introduced in Mongo 4, whereas DocumentDB copied the 3.6 API. So DocumentDB seems much less useful as a reactive data store than MongoDB.

In other words, DocumentDB is only a drop-in replacement for MongoDB if you weren't using any of the features Amazon decided not to support.

Happy to be corrected if I'm misreading the documentation!

Re: Amazon DocumentDB, with MongoDB compatibility

#275

Post by aortenzi »

api wrote: post #155
abrookewood wrote: post #70

I was reading a post [0] by Brian Cantrill that predicted this would be the result of licences like the SSPL. I instinctively disagreed with him, but it turns out he was right: "The cloud services providers are currently reproprietarizing all of computing — they are making their own CPUs for crying out loud! — reimplementing the bits of your software that they need in the name of the service that their customers want…

Years ago I realized that a hidden driver for the growth of cloud is this. The cloud is DRM, and almost uncrackable DRM at that since you have neither the code nor the hardware.

Software and the control plane is the razor, compute resources are the blades. Amazon's software is its loss leader.

Re: Amazon DocumentDB, with MongoDB compatibility

#276

Post by penguinbroker »

gazzini wrote: post #137
iamed2 wrote: post #134

Earlier quoted context omitted.

> it requires an elastic IP which slows launch of lambdas since they have to connect to VPC Do you have any other info/links related to this?

There's an enlightening graph in this post: https://medium.freecodecamp.org/lambda-vpc-cold-starts-a-lat... I was surprised to learn this. When working in Lambda, you have to choose between a relational database & a responsive API. It seems inevitable that AWS will fix this soon, but apparently this is a significant architectural problem. As I understand it, RDS instances should (must?) be accessed from within a VPC,…

https://www.nuweba.com/AWS-Lambda-in-a-VPC-will-soon-be-fast...

Re: Amazon DocumentDB, with MongoDB compatibility

#277

Post by talawahtech »

evil-olive wrote: post #217
talawahdotnet wrote: post #190

Earlier quoted context omitted.

Interesting, how long ago was that? I would be curious to know if the WiredTiger switch ever happened, and what that support relationship looks like not given the contentious relationship between MongoDB and AWS. The old Wired Tiger Inc website[1] still lists AWS as a customer. Then again, the relationship between AWS and Oracle is even more contentious and Aurora MySQL is one of AWS's most popular products so I don'…

3+ years ago, so it's entirely possible that things have changed since I left. I don't have any more recent information on the state of the system. At least when I was there, the strong focus was always on adding new features (global & local secondary indexes, change streams, cross-region replication, and so on) to keep up with the Joneses (MongoDB et al). Meanwhile, a bunch of internal Amazon teams were taking a dep…

Given the scale and popularity of DynamoDB and the distributed nature you would think that they could hire multiple teams just to work on improving it, but I guess it isn't as simple as that.

I would love to get a behind the scenes look at the process of gradually improving the components of DynamoDB with better technologies, while still maintaining reliability and performance.

Re: Amazon DocumentDB, with MongoDB compatibility

#278

Post by ahachete »

bnjmn wrote: post #274

Looking through the supported APIs ( https://docs.aws.amazon.com/documentdb/latest/developerguide... ), it appears DocumentDB has no support for Mongo's oplog ( https://docs.mongodb.com/manual/core/replica-set-oplog/ ), or change streams ( https://docs.mongodb.com/manual/changeStreams ), which I guess is no surprise because change streams were introduced in Mongo 4, whereas DocumentDB copied the 3.6 API. So DocumentD…

I agree with you.

Having said that, when we were working on https://www.torodb.com we discussed how we'd implement the oplog. And actually, based on PostgreSQL's logical decoding (LD), it wouldn't have been a great deal (there are some gotchas, but LD brings much of what you need. So I won't be surprised if this would be implemented sooner than later.

Re: Amazon DocumentDB, with MongoDB compatibility

#279

Post by sfifs »

meddlepal wrote: post #148
Lazare wrote: post #127

Earlier quoted context omitted.

Ultimately, Cantrill put it well: > ...for those open source companies that still harbor magical beliefs, let me put this to you as directly as possible: cloud services providers are emphatically not going to license your proprietary software. I mean, you knew that, right? MongoDB Inc cannot make Amazon pay commercial license fees. That is not a thing that will happen. They have a lever in front of them with two posi…

Could Mongo or other companies use the Oracle v. Google precedent regarding API copyright to extract money from competitive vultures like Amazon?

Not likely because the Apache 2 licence version they are compatible with includes an explicit copyright and patent licence grant.

Re: Amazon DocumentDB, with MongoDB compatibility

#280

Post by indogooner »

bnjmn wrote: post #274

Looking through the supported APIs ( https://docs.aws.amazon.com/documentdb/latest/developerguide... ), it appears DocumentDB has no support for Mongo's oplog ( https://docs.mongodb.com/manual/core/replica-set-oplog/ ), or change streams ( https://docs.mongodb.com/manual/changeStreams ), which I guess is no surprise because change streams were introduced in Mongo 4, whereas DocumentDB copied the 3.6 API. So DocumentD…

Interesting. I think the sole purpose of this product is to wean existing Mongo customers (3.6-). And only those customers who are happy with Mongo API but not MongoDB itself. Is that such a huge market? Would be curious to see how this solution is adopted.
Post reply on HN