Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS cancels serverless Postgres service that scales to zero

datanami.com

Post reply on HN
Print view View original

81–90 of 101 posts

Re: AWS cancels serverless Postgres service that scales to zero

#81

Post by alanfranz »

pandastronaut wrote: post #60
qaq wrote: post #10

I guess a good opportunity for someone to build a good alternative. Although it's hard to get it right because if you scale to 0 people will complain about latency and if you don't it's not really serverless.

Corporate customers that runs AWS, GCP or Azure at not looking for alternative outside of their cloud providers. Most solution architecture are designed with the assumption that all your ressources will be inside the same vnet or subnet, in a given cloud provider. And I am not talking about any egress fees yet.

I challenge this. I work at Aiven and we provide alternate cloud-independent solutions, sometimes in the customer’s own account, and enterprise customers definitely see the value in using a fixed cost, no-lock-in service.

Re: AWS cancels serverless Postgres service that scales to zero

#82

Post by justinclift »

nikita wrote: post #67
qaq wrote: post #58

Earlier quoted context omitted.

It helps but does not fully solve it if I need PG to have 512 GB of memory buffers under load to have reasonable latency starting from 0 and going through a number of VM migrations to finally get to properly sized VM with 512 GB of hot data in memory gonna cause a ton of latency.

The buffers will be cached in remote storage - so it will just work likely. In the future we are considering separating the buffer pool from compute and turning off compute doesn't drop the buffer cache. It's a research project but PolarDB did that for AliCloud.

"Remote buffers" sounds like it'd introduce a base level of latency anyway.

Though computers are pretty fast these days (and RDMA exists), so if the pieces are in the same data centre and using good interconnects it's probably not going to make much of a practical difference.

Re: AWS cancels serverless Postgres service that scales to zero

#83

Post by belter »

dragonwriter wrote: post #79
belter wrote: post #49

Earlier quoted context omitted.

Almost any SQL workload you can move to DynamoDB

DynamoDB lacks consistency (in the ACID sense, it has been it in the CAP sense which is more related to Atomicity in ACID), so you can move an SQL workload to DynamoDB by moving the consistency parts out of the DB into the code of any application that hits the DB.

You hardly get the ACID on a relational database :-)

"When is "ACID" ACID? Rarely." - http://www.bailis.org/blog/when-is-acid-acid-rarely/

Re: AWS cancels serverless Postgres service that scales to zero

#84

Post by belter »

e12e wrote: post #71
belter wrote: post #49

Earlier quoted context omitted.

Almost any SQL workload you can move to DynamoDB

https://aws.amazon.com/blogs/database/how-to-determine-if-am... > Some unsuitable workloads for DynamoDB include: > Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome. > Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally requ…

I said moving the workload to DynamoDB, not doing the same query pattern with DynamoDB ... :-)

For each of these workloads you switch the query pattern to accommodate for DynamoDB and provide a solution for your workload and/or app. That is actually the secret of NoSQL. You do the work upfront.

So your reading of the recommendations is correct on the surface but incorrect on the fundamental usage aspect of it. Lets have a look at each:

> Services that require ad hoc query access. Though it’s possible to use external relational frameworks to implement entity relationships across DynamoDB tables, these are generally cumbersome.

A: Dont do this. Don't do ad hoc query access. Define a series of access patterns and lay out your data to support them. You dont get ad hoc query access for Netflix, Amazon or your Airline Travel website...

> Online analytical processing (OLAP)/data warehouse implementations. These types of applications generally require distribution and the joining of fact and dimension tables that inherently provide a normalized (relational) view of your data.

These are not most SQL Patterns. This is OLAP and was always or should be done with MPP systems, not with your relational database like your SQLServer or your Oracle. So this is going on an edge...

> Binary large object (BLOB) storage. DynamoDB can store binary items up to 400 KB, but DynamoDB is not generally suited to storing documents or images. A better architectural pattern for this implementation is to store pointers to Amazon S3 objects in a DynamoDB table.

Your Relational Database will not store BLOBs more efficiently than S3 anyway....

Re: AWS cancels serverless Postgres service that scales to zero

#85

Post by dragonwriter »

belter wrote: post #83
dragonwriter wrote: post #79

Earlier quoted context omitted.

DynamoDB lacks consistency (in the ACID sense, it has been it in the CAP sense which is more related to Atomicity in ACID), so you can move an SQL workload to DynamoDB by moving the consistency parts out of the DB into the code of any application that hits the DB.

You hardly get the ACID on a relational database :-) "When is "ACID" ACID? Rarely." - http://www.bailis.org/blog/when-is-acid-acid-rarely/

That's an article about isolation, not consistency; and even there, both the RDBMS’s relevant to thus discussion (those on Aurora Serverless that it is suggested in this subthread be replaced with DynamoDB) supported serializable isolation even in their current versions at the time of that article a decade ago.

So your post is both irrelevant as a response to mine about consistency and more generally irrelevant to the entire discussion here.

Re: AWS cancels serverless Postgres service that scales to zero

#86

Post by belter »

dragonwriter wrote: post #85
belter wrote: post #83

Earlier quoted context omitted.

You hardly get the ACID on a relational database :-) "When is "ACID" ACID? Rarely." - http://www.bailis.org/blog/when-is-acid-acid-rarely/

That's an article about isolation, not consistency; and even there, both the RDBMS’s relevant to thus discussion (those on Aurora Serverless that it is suggested in this subthread be replaced with DynamoDB) supported serializable isolation even in their current versions at the time of that article a decade ago. So your post is both irrelevant as a response to mine about consistency and more generally irrelevant to th…

The article is about ACID, both consistency and isolation not just about consistency and you can't isolate them (pun intended). The underlying of the discussion was not on consistency, instead on the possibility of using DynamoDB where you would use a typical relational engine.

From the article: "The textbook definition of ACID Isolation is serializability (e.g., Architecture of a Database System, Section 6.2), which states that the outcome of executing a set of transactions should be equivalent to some serial execution of those transactions. This means that each transaction gets to operate on the database as if it were running by itself, which ensures database correctness, or consistency."

" This means that each transaction gets to operate on the database as if it were running by itself, which ensures database correctness, or consistency. A database with serializability (“I” in ACID), provides arbitrary read/write transactions and guarantees consistency (“C” in ACID), or correctness, of the database. Without serializability, ACID, particularly consistency, is generally not guaranteed"

I am of course ignoring the Consistency you are certainly aware that exist in DynamoDB with Strong Consistency and DynamoDB Transactions.

Moving consistency management outside of DynamoDB as you propose doesn't circumvent the CAP theorem's limitations.

Re: AWS cancels serverless Postgres service that scales to zero

#88

Post by qaq »

pandastronaut wrote: post #60
qaq wrote: post #10

I guess a good opportunity for someone to build a good alternative. Although it's hard to get it right because if you scale to 0 people will complain about latency and if you don't it's not really serverless.

Corporate customers that runs AWS, GCP or Azure at not looking for alternative outside of their cloud providers. Most solution architecture are designed with the assumption that all your ressources will be inside the same vnet or subnet, in a given cloud provider. And I am not talking about any egress fees yet.

Well just one counterexample Snowflake

Re: AWS cancels serverless Postgres service that scales to zero

#89

Post by Ao7bei3s »

belter wrote: post #49
qvrjuec wrote: post #11

Earlier quoted context omitted.

Because you need SQL. Dynamo can be quite powerful for querying if you use all of its features but it won't be as flexible as just using SQL

Almost any SQL workload you can move to DynamoDB

DynamoDB can't even represent SELECT * FROM items LIMIT 25 OFFSET 100. It's just not designed for that. It's not meant to be a relational DB replacement.

How would you do it? Assume we want proper pagination, and not rewrite the app for cursor based "Load more" style pagination. Why? Because the React Admin provider API insists. https://github.com/marmelab/react-admin/issues/1510

Re: AWS cancels serverless Postgres service that scales to zero

#90

Post by [deleted] »

justinclift wrote: post #82
nikita wrote: post #67

Earlier quoted context omitted.

The buffers will be cached in remote storage - so it will just work likely. In the future we are considering separating the buffer pool from compute and turning off compute doesn't drop the buffer cache. It's a research project but PolarDB did that for AliCloud.

"Remote buffers" sounds like it'd introduce a base level of latency anyway. Though computers are pretty fast these days (and RDMA exists), so if the pieces are in the same data centre and using good interconnects it's probably not going to make much of a practical difference.

[deleted]
Post reply on HN