I think the upcoming CedarDB is HTAP? https://cedardb.com/
Clickhouse performance for Postgres workloads?
HTAP is Dead
71–80 of 95 posts
Re: HTAP is Dead
#72You cannot say HTAP is dead when the alternative is so much complexity and so many moving parts. Most enterprises are burning huge amounts of resources literally just shuffling data around for zero business value. The dream is a single data mesh presenting an SQL userland where I can write and join data from across the business with high throughput and low latency. With that, I can kill off basically every microservi…
> The dream is a single data mesh presenting an SQL userland where I can write and join data from across the business with high throughput and low latency. That exists, and has for years: an extremely large DB loaded to the gills with RAM and local NVMe drives. Add some read replicas if you need them, similarly configured. Dedicate one for OLAP.
This doesn’t work quite as well as people assume. The first limit is simply size, you can only cram a few petabytes of NVMe in a server (before any redundancy) and many operational analytic workloads are quite a bit larger these days. One of the major advantages of disaggregated storage in theory is that it allows you to completely remove size limits. Many operational analytic workloads don’t need a lot of compute, just sparse on-demand access to vast amounts of data. With good selectivity (another open issue), you could get excellent performance in this configuration.
Ignoring the storage size limits, the real issue as you scale up is that the I/O schedulers, caching, and low-level storage engine mechanics in a large SQL database are not designed to operate efficiently on storage volumes this large. They will work technically, but scale quite a bit more poorly than people expect. The internals of SQL databases are (sensibly) optimized for working sets no larger than 10x RAM size, regardless of the storage size. This turns out to be the more practical limit for analytics in a scale-up system even if you have a JBOD of fast NVMe at your disposal.
Re: HTAP is Dead
#73Earlier quoted context omitted.
I think you may have misunderstood the GP and are perhaps misusing terminology. You cannot meaningfully scale vertically to improve write availability, and if you care about availability a single machine (and often a primary/secondary setup) is insufficient. Even if you only care about scaling reads, eventually the 1:N write:read replica ratio will become too costly to maintain, and long before you reach that point y…
> You cannot meaningfully scale vertically to improve write availability Disagree. Even if you limit yourself to the cloud, r7i/r8g.48xl gets you 192 vCPU / 1.5 TiB RAM. If you really want to get silly, x2iedn.32xl is 128 vCPU / 4 TiB RAM, and you get 3.8 TiB of local NVMe storage for temp tablespace. The money you’ll pay ($16.5K - $44K month, depending on specific class) would pay for a similarly spec’d server in th…
The big caveat about these configurations is the amount of time it takes to rebuild a replica due to the quantity of storage per node that has to be pushed over the network. This is one of the low-key major advantages of disaggregated storage.
I prefer to design my own hardware infrastructure but there are many operational tradeoffs to consider.
Re: HTAP is Dead
#74Earlier quoted context omitted.
> The dream is a single data mesh presenting an SQL userland where I can write and join data from across the business with high throughput and low latency. That exists, and has for years: an extremely large DB loaded to the gills with RAM and local NVMe drives. Add some read replicas if you need them, similarly configured. Dedicate one for OLAP.
This doesn’t work quite as well as people assume. The first limit is simply size, you can only cram a few petabytes of NVMe in a server (before any redundancy) and many operational analytic workloads are quite a bit larger these days. One of the major advantages of disaggregated storage in theory is that it allows you to completely remove size limits. Many operational analytic workloads don’t need a lot of compute, j…
> many operational analytic workloads are quite a bit larger these days.
What are the use cases where such workloads come up, aside from Google-level operations? Just trying to understand what we are talking about.
Re: HTAP is Dead
#75On the data warehousing side, I think the story looks like this: 1) Cloud data warehouses like Redshift, Snowflake, and BigQuery proved to be quite good at handling very large datasets (petabytes) with very fast querying. 2) Customers of these proprietary solutions didn't want to be locked in. So many are drifting toward Iceberg tables on top of Parquet (columnar) data files. Another "hidden" motive here is that Clou…
This is essentially what Crunchydata does with their Crunchydata Warehouse product. It’s really cool.
Their product looks promising. It looks like the PostGres schema and writes have to be "Iceberg-aware": special work to get around the fact that a small write results in a new, small Parquet file. That's not the end of the world - but perhaps ideally, you wouldn't be aware of Iceberg much at all when using PostGres. That might be a dream though.
Fully using PostGres without awareness of Iceberg would require full decoupling, and a translation layer in between (Debezium, etc). That comes with its own problems.
So perhaps some intimacy between the PostGres and Iceberg schemas is a good thing - especially to support transparent schema evolution.
DuckLake and CrunchyBridge both support SQL queries on the backing Iceberg tables. That's a good option. But a big part of the value of Iceberg comes in being able to read using Spark, Flink, etc.
Re: HTAP is Dead
#76Earlier quoted context omitted.
This doesn’t work quite as well as people assume. The first limit is simply size, you can only cram a few petabytes of NVMe in a server (before any redundancy) and many operational analytic workloads are quite a bit larger these days. One of the major advantages of disaggregated storage in theory is that it allows you to completely remove size limits. Many operational analytic workloads don’t need a lot of compute, j…
> many operational analytic workloads are quite a bit larger these days. What are the use cases where such workloads come up, aside from Google-level operations? Just trying to understand what we are talking about.
Sensor and telemetry analytics workloads in boring industrial sectors are all at this scale, even at companies that aren’t that large revenue-wise. TBs to PBs of new data per day.
Re: HTAP is Dead
#77Earlier quoted context omitted.
> many operational analytic workloads are quite a bit larger these days. What are the use cases where such workloads come up, aside from Google-level operations? Just trying to understand what we are talking about.
Sensor and telemetry analytics workloads in boring industrial sectors are all at this scale, even at companies that aren’t that large revenue-wise. TBs to PBs of new data per day.
What are these used for, to all have to be in a single unified database?
Re: HTAP is Dead
#78Earlier quoted context omitted.
This is essentially what Crunchydata does with their Crunchydata Warehouse product. It’s really cool.
Their product looks promising. It looks like the PostGres schema and writes have to be "Iceberg-aware": special work to get around the fact that a small write results in a new, small Parquet file. That's not the end of the world - but perhaps ideally, you wouldn't be aware of Iceberg much at all when using PostGres. That might be a dream though. Fully using PostGres without awareness of Iceberg would require full dec…
I'd argue the bigger value is keeping the data in one storage place and bringing the compute to it. Works especially well for Big Corp use cases where entire divisions of the corp go their own way. Throw in M&A activity and it is a good hedge for the unknown (I.e you might be an Databricks and Azure shop and you just bought a Snowflake & AWS company). Keep the data in an open table format, and let everyone query using their preferred engine to their hearts desire.
Re: HTAP is Dead
#79Earlier quoted context omitted.
Sensor and telemetry analytics workloads in boring industrial sectors are all at this scale, even at companies that aren’t that large revenue-wise. TBs to PBs of new data per day.
What are these used for, to all have to be in a single unified database?
A large part of those workloads is stitching together a single derived model of operational reality and how different entities interact over time from the samples you get from each individual source. You need a running log of all entity behavior and interactions over time to look back on in order to contextualize what you see at the current point in time. Most of this is not pre-computable because the combinatorial state space is too large so every analytic query needs to be able to see across every relationship between sources that can be inferred.
It is essentially a spatial and/or graph analytic model evolving over time. Any non-trivial data models that capture dynamics in the physical world looks like this.
In fairness, all popular analytics platforms handle these workloads poorly regardless of if they are vertically or horizontally scaled. These workloads usually cannot be cached even in theory, so performance and scalability comes down to the sophistication of your scheduler design.
Re: HTAP is Dead
#80Earlier quoted context omitted.
can you explain this please "not having it means your CDC updates will clobber your unchanged TOAST values" ?
They’re referring to this: https://debezium.io/blog/2019/10/08/handling-unchanged-postg...
Funny timing, just took a fresh look at this topic in this new post earlier this week: https://www.morling.dev/blog/backfilling-postgres-toast-colu....