[flagged]
[deleted]
Amazon Aurora DSQL
131–140 of 146 posts
Re: Amazon Aurora DSQL
#132Earlier quoted context omitted.
This but unironically - if I’m in the market for a geo distributed hyper scale DB, I genuinely and sincerely have no interest in foreign key constraints. They are pretty much useless, if not actively harmful, at scale.
Disagree heavily that they’re useless at scale. Both PlanetScale and Citus support them (though the former doesn’t support it on sharded DBs yet), as well as TiDB (alpha), and Yugabyte. As far as performance hits go, I guarantee I could find a half dozen other, larger problems that would have a bigger impact than disabling FK constraints. It is of course possible to maintain referential integrity without them, but it…
I didnt necessarily mean scale as req/sec, although that is a factor, more system complexity & num of engineers. In a large enough companies tech ecosystem you tend to evolve a few characteristics:
- many data stores. even if you're not doing microservices persay, you're almost certainly going to end up with different domains/buis units/systems having their own data stores. FK constraints obviously dont work across system boundaries. They could be vendor systems too, not even just your own services.
- soft deletes. you rarely hard delete root objects, so you dont need the FK constraint to protect those dangling refs
- eventual consistentency/idempotency - you end up creating data in different systems as part of a workflow (eg signup), and you have to tolerate partial creation for resiliency.
- some sort of ORM or sql framework that makes it very difficult to create a leaf row with an invalid FK parent pointer.
So in a fundamentally distributed large scale tech ecosystem, FK constraints end up protecting against.... not much. Ive worked at three large scale (10m+ users) companies which all ended up deleting all their FK constraints. And... nothing happened, nothing ever went wrong due to their absence. Its theoretical protection, not practically needed IMHO.
Re: Amazon Aurora DSQL
#133I find it to be super limited, and I'm sort of struggling to see the point given all these constraints. No temporary tables, no foreign keys, no views, no more than 10k rows in a transaction. Except for some basic wire compatibility with the postgres protocol, I'd hardly call this a "database", and more a key-value store. https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
Seems like it's mostly AWS' answer to Cloudflare D2, but using Postgres as the frame of reference instead of sqlite. (I'm not really sure what to call Postgres in this situation since it's so limited it's clearly not any full version of Postgres in any respect. Postgres-ish)
> Cloudflare D2
Not that it matters but it's D1 but I noticed people sometimes call it D2, maybe by analogy with R2.
Re: Amazon Aurora DSQL
#134Earlier quoted context omitted.
So it’s basically a CQL/Cassandra alternative. It’s interesting there’s also no comparison page to Cassandra or Amazon Keyspaces. I also now don’t get why they are emphasizing PostgreSQL here when they should be emphasizing CQL compatibility. Being compatible with Postgres wire protocol but not being relational is strange.
What? Cassandra is a very different technology semantically. From data design to consistency models, to transactions, it’s entirely different, don’t know how you drew this conclusion.
My guess is from the customer's perspective, DSQL seems to have too many limitations, making it feel more like an enhanced version of a NoSQL database with SQL semantics, ACID transactions, and multi-region capabilities rather than a truly distributed version of a relational database. It seems the DSQL team doesn't fully understand why relational databases remain widely popular today. What makes them great is their flexibility, which lets them handle all kinds of use cases and adapt as things change. But all the limits on transaction size, column size, etc.., and too many missing capabilities pretty much take away all the big advantages relational databases usually offer.
Re: Amazon Aurora DSQL
#135- Yes, the storage is distributed kv.
- Yes, it's SQL on distributed transactional kv
- SQL layer is stateless, runs in multiple containers
- SQL layer would push down compute logic down to the storage(like, filter/predicate)
....Yes, that reminds me of TiDB
Re: Amazon Aurora DSQL
#136I find it to be super limited, and I'm sort of struggling to see the point given all these constraints. No temporary tables, no foreign keys, no views, no more than 10k rows in a transaction. Except for some basic wire compatibility with the postgres protocol, I'd hardly call this a "database", and more a key-value store. https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...
Major "No wireless. Less space than a Nomad. Lame." vibes.
This is the first release of something new and groundbreaking.
Re: Amazon Aurora DSQL
#137I won't even consider this without pricing, I can't imagine how anyone could. Serverless Aurora is stupid-expensive so I can't imagine this is going to be cheap but I keep hunting for that perfect DB that can scale to 0 (or very low) and be priced well. Right now neon.tech is what I'm using and I'm very happy but this looks like it would be interesting IF the pricing is good.
I'm also on Neon. Seems like the primary benefit of this would be multi-region support, which Neon doesn't have. But I do wonder if multi-AZ is actually just enough, has there ever really been a time when all AZs have gone down?
(neon employee)
we have multi-region disaster recovery and replicas coming in 2025. Switch over time will be greater than this active-active system but the overall latency for Neon should be much less on a consistent basis.
Re: Amazon Aurora DSQL
#138Earlier quoted context omitted.
The most interesting page is always quotas and limits: https://docs.aws.amazon.com/aurora-dsql/latest/userguide/CHA... Seeing the No to configurable for some of the settings is the most telling hard limits that we can see up front. Some Very Noteable ones are: Maximum size of all data modified within a write transaction: 10 MiB Maximum transaction time: 5 minutes
The 10MiB transaction size limit smells a lot like FoundationDBs transaction size limit
10MiB is an appealing number for humans. Can easily appear across different designs.
Re: Amazon Aurora DSQL
#139Earlier quoted context omitted.
* RDS is a managed relational database service. You got a database? We'll run it in the cloud for you. Exact same bits and bytes as you're running locally. * Aurora is Amazon's own relational database. You can't run it yourself, only with Amazon. It can pretend to be either Postgres or MySQL. And it'll be cheaper and faster and have higher availability. But it won't be the exact same bits and bytes as your own Postgr…
> f you're building a new Cloud native application and you don't have to worry about legacy migrations, you'll probably choose either DynamoDB or Aurora DSQL in 99.9% of cases. If cloud-native means spending your life in AWS, that is.
I meant an AWS cloud native application, yes.
But I use "cloud native" in a way that actually makes sense, not the way that GCP dubbed Kubernetes and all of it's ecosystem of friends "cloud-native".
"Native" development on a desktop OS means using primitives specific to that OS. As opposed to using something like Java Applets or Electron which will give you cross-platform compatibility, but you're not developing "natively" for the platform. That has the obvious pros and cons to each way.
Kubernetes is very much the Electron way: Rely on a generic high-level abstraction that will then create awkward bindings of running on abstract generic servers - whether they're in your data center or in the cloud.
Calling that "cloud-native" is some Orwellian post-truthness. I hate it.
If you want to develop in the cloud and make the most of it, use Azure, use GCP, use AWS, I don't care (not true: AWS is the best; use AWS). Just use the cloud provider's NATIVE primitives.
Otherwise, you might as well go back to running your own data centers, what are you even doing.
(Note: i'm just some guy. You don't have to listen to me. It's just what I think)
Re: Amazon Aurora DSQL
#140Earlier quoted context omitted.
>Aurora is Amazon's own relational database. You can't run it yourself, only with Amazon. It can pretend to be either Postgres or MySQL. And it'll be cheaper and faster and have higher availability. But it won't be the exact same bits and bytes as your own Postgres, so there's some risks. That doesn't ring true to me. From my understanding rds aurora mostly replaces the storage engines of these DBs and the frontent m…
You're spot on. GP is repeating the common misconception that "Aurora" is a distinct, from-the-ground-up relational database system which can "pretend" to be either MySQL or Postgres by reimplementing their protocols/front-end from scratch. But that isn't the case at all. In reality, Aurora MySQL is clearly heavily based on the MySQL codebase, but with major changes specifically to InnoDB's persistence and MVCC layer…
Oops, my bad.