Live data from Hacker News

Amazon Aurora DSQL

aws.amazon.com

81–90 of 146 posts

Re: Amazon Aurora DSQL

#81
post #76

Very interesting. My dream is to have something like this, a KV-store, a blob store, and pubsub all behind the same interface.

"like this" meaning you want wire PG compatibility, so you'd do something like this? UPDATE kv SET value = 'alpha' WHERE key = 'beta.charlie'; UPDATE s3 SET value = $b64$good luck$b64$ WHERE key = '/some/s3/path'; LISTEN whatever;

I don't know what I want exactly but I'm thinking along the lines that SQL is already doing a lot so it would make the most sense to start with a database interface and augment from there, to try to build a system to handle all the common forms of durable storage used by applications.

The type of situation I'm thinking about is for example storing a blob in S3, storing metadata and a reference to the blob's path in a database row, sending a message into a queue to trigger some async processing, and updating a cache. It would be nice to be able to do this through a single API or service, and it would be really nice to do all this within some type of transaction abstraction that would allow all operations to pass or fail collectively, really really nice if the whole thing could be pay-as-you and scale horizontally-ish on shared infrastructure without managing nodes or slots or whatever.

I'm not a Postgres user so I don't know how far you can get currently and I should probably look into it in detail. Coordinating blob/ject storage, database, and pubsub operations is a pain point for me presently. I think that overall system design is going to prevent a database-type system from being a good idea for blob storage but I would still like to see someone try to put three systems in a trenchcoat and try to make it work behind one interface.

Re: Amazon Aurora DSQL

#82
post #65

Can Amazon please get RDS product management ducks in a row? You got RDS For PostgreSQL, Aurora PostgreSQL (provisioned), Aurora Serverless v1, Aurora Serverless v2, Aurora PostgreSQL Limitless Database, Aurora Global Database, and now Aurora DSQL (with PostgreSQL compatibility). It's very confusing, the docs are confusing, and the even the tools are confused. I think this is pretty big contrast to something like S3…

> It's very confusing, the docs are confusing, and the even the tools are confused. Not to worry -- you can use Rufus...uh "Q"....uhh...Nova...to summarize the docs into a decision tree on which confusing choice to use!

Except you can't, because it will just misundertand you or say something hopelessly generic.

Even asking it something on the same page that it's advertised 'now supports asking about' completely failed for me - think it was ENIs or something; just said something completely generic, not at all about the instances I'd asked about which it advertised being able to do (and I happened to have a concurrent need for).

Re: Amazon Aurora DSQL

#84
post #65

Can Amazon please get RDS product management ducks in a row? You got RDS For PostgreSQL, Aurora PostgreSQL (provisioned), Aurora Serverless v1, Aurora Serverless v2, Aurora PostgreSQL Limitless Database, Aurora Global Database, and now Aurora DSQL (with PostgreSQL compatibility). It's very confusing, the docs are confusing, and the even the tools are confused. I think this is pretty big contrast to something like S3…

> It's very confusing

As a very naive person myself, isn’t “being very confusing” one of the main points of AWS? They know they are big and nobody will get fired for using AWS. So, the more confusing their products are, the more benefits for them (e.g., because you don’t know how much you will pay, because you are misusing their products, because perhaps you don’t actually need their products, etc)

So, any manager at AWS probably thinks this is all working very fine.

Re: Amazon Aurora DSQL

#85
post #34

Just tried a quick test cluster - Identifies as PG 16.5 No views/triggers/sequences No foreign key constraints No extensions No NOTIFY ("ERROR: Function pg_notify not supported") No nested transactions No json(b) Unsupported PG features are now online https://docs.aws.amazon.com/aurora-dsql/latest/userguide/wor...

what index types does it support?

Re: Amazon Aurora DSQL

#86
post #65

Can Amazon please get RDS product management ducks in a row? You got RDS For PostgreSQL, Aurora PostgreSQL (provisioned), Aurora Serverless v1, Aurora Serverless v2, Aurora PostgreSQL Limitless Database, Aurora Global Database, and now Aurora DSQL (with PostgreSQL compatibility). It's very confusing, the docs are confusing, and the even the tools are confused. I think this is pretty big contrast to something like S3…

Is it? Every solution has some advantages, switching between them is relatively easy (especially if you can tolerate some writing downtime), and IMO it's not that confusing if you have at least some experience in the space.

Otherwise if you're just staring from scratch, I'd pick provisioned aurora if your load is steady and serverless v2 if not. Then IO optimized if the db doesn't fit into memory and non-IO-optimized until it does.

Re: Amazon Aurora DSQL

#87
post #65

Can Amazon please get RDS product management ducks in a row? You got RDS For PostgreSQL, Aurora PostgreSQL (provisioned), Aurora Serverless v1, Aurora Serverless v2, Aurora PostgreSQL Limitless Database, Aurora Global Database, and now Aurora DSQL (with PostgreSQL compatibility). It's very confusing, the docs are confusing, and the even the tools are confused. I think this is pretty big contrast to something like S3…

> Aurora Serverless v1

This is deprecated.

Re: Amazon Aurora DSQL

#88
post #71

I won't even consider this without pricing, I can't imagine how anyone could. Serverless Aurora is stupid-expensive so I can't imagine this is going to be cheap but I keep hunting for that perfect DB that can scale to 0 (or very low) and be priced well. Right now neon.tech is what I'm using and I'm very happy but this looks like it would be interesting IF the pricing is good.

I'm also on Neon. Seems like the primary benefit of this would be multi-region support, which Neon doesn't have. But I do wonder if multi-AZ is actually just enough, has there ever really been a time when all AZs have gone down?

yes, every couple of years there's a region-wide outage. On us-east anyway.

Re: Amazon Aurora DSQL

#90
post #65

Can Amazon please get RDS product management ducks in a row? You got RDS For PostgreSQL, Aurora PostgreSQL (provisioned), Aurora Serverless v1, Aurora Serverless v2, Aurora PostgreSQL Limitless Database, Aurora Global Database, and now Aurora DSQL (with PostgreSQL compatibility). It's very confusing, the docs are confusing, and the even the tools are confused. I think this is pretty big contrast to something like S3…

Is it? Every solution has some advantages, switching between them is relatively easy (especially if you can tolerate some writing downtime), and IMO it's not that confusing if you have at least some experience in the space. Otherwise if you're just staring from scratch, I'd pick provisioned aurora if your load is steady and serverless v2 if not. Then IO optimized if the db doesn't fit into memory and non-IO-optimized…

It's not just selecting the service. It's also trying to decipher which parts what docs apply to when reading. Or how different services interact. Or getting any understanding what "Aurora" means in different contexts.

Just random example I was poking around AWS Backup recently, and there you have option to enable Backup for different services. Among the other options, you have separate toggles for "Aurora" and "RDS".

Overall the whole RDS API has lots of weirdness because there are all these different things (and combinations of things). Another random example, configuring Data HTTP API, you got this thing:

https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/AP...

> This operation applies only to Aurora Serverless v2 and provisioned DB clusters. To enable the HTTP endpoint for Aurora Serverless v1 DB clusters, use the EnableHttpEndpoint parameter of the ModifyDBCluster operation.

At least that specific case was nicely documented now. Not all the pitfalls are thst clear.

Post reply on HN