31–40 of 71 posts
This is some seriously impressive engineering. Spanner is the only DB that can achieve this.
I can't wait until people find its limits!
I can't wait until people find its limits... in production!
Make sure to prune old data from your tables. This one got to this limit because it eventually got too large that queries to delete old data would time out... so it just kept growing.
> Join the preview of Amazon Aurora Limitless Database would be the non-editorialized submission, and would have advised potential readers that one cannot currently play with it without a ton of hoopjumpery one will also want to watch out for this, buried 7 paragraphs in: > The preview runs in a new Aurora PostgreSQL cluster with version 15 in the AWS US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Paci…
Keep in mind that the California region has been devalued for a while. AWS would turn it off if they could. https://www.lastweekinaws.com/blog/us-west-1-the-flagship-aw...
As if data egress charges aren't gouging customers everywhere!
This sounds very exciting, but I would have greatly appreciated some actual technical details. From this statement > Sharded tables – These tables are distributed across multiple shards. Data is split among the shards based on the values of designated columns in the table, called shard keys. It sounds like this is very much managed CitusDB on top of Aurora, but without any details about the implementation its impossi…
Aurora Limitless Database is based on our own investments in database-optimized virtualization (Caspian), in scale-out log-first database storage (Grover)[2], and in a custom approach to cross-shard transactions that makes use of the high-quality hardware clocks available in EC2[1]. We'll be talking more about the internals over the coming months. [1] https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-ti... [2…
Super exciting announcement, and I am really looking forwards to learning more!
> a new capability supporting automated horizontal scaling to process millions of write transactions per second and manage petabytes of data in a single Aurora database. Limitless horizontal scaling is really cool and all, but does anyone not running a Fortune-500 Tech company actually need this kind of firepower?
Earlier quoted context omitted.
I can't wait until people find its limits... in production!
Did you know that Postgres has a max table size of 32TB? Its really really fun to find that out the Wednesday evening before Thanksgiving. Make sure to prune old data from your tables. This one got to this limit because it eventually got too large that queries to delete old data would time out... so it just kept growing. https://www.postgresql.org/docs/current/limits.html
Earlier quoted context omitted.
Aurora Limitless Database is based on our own investments in database-optimized virtualization (Caspian), in scale-out log-first database storage (Grover)[2], and in a custom approach to cross-shard transactions that makes use of the high-quality hardware clocks available in EC2[1]. We'll be talking more about the internals over the coming months. [1] https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-ti... [2…
I appreciate the context, but this just adds so many more questions. How are you integrating hardware clocks into the Postgres transaction model? Is this just postgres wire compatible like Cockroach or Yugabyte, or have you modified Postgres MVCC implementation to not use the standard TID. Does this support all of postgres at the Transaction coordinator ? Super exciting announcement, and I am really looking forwards…
1. Scales to zero, no cost when not using
2. Allows for SQL over API like aurora v1. This point is important, it allows for faster access from other serverless technologies, i.e. lambda