Viewing profile — AdamProut

Agents should be driving up the quality of software if we focus them on finding and fixing bugs.

Very cool! pg_duckdb itself is missing fully integrated storage - it can query data stored externally (say in S3) in delta/iceberg formats, but it can't write out data in those for…

Had a similar thought. Azure Postgres has something similar to pg_parquet (pg_azure_storage), but we're looking into replacing it with pg_duckdb assuming the extension continues to…

Checksums can detect a torn page, but not always repair them. It's likely a good part of the database page is gone (i.e., an amount of data that matches the disk / file system atom…

Maybe good to mention torn pages somewhere too? Both MySQL and Postgres jump through some hoops to both detect them and repair them [1][2]. So, even the scenario in the post where …

I'm not sure why you're getting downvoted. I think the pressure to appease Gartner usually starts when companies bring in CEOs whose primary background is enterprise sales. They te…

Based on the writings of 2 Turing awards winners for work on databases; No MongoDB doesn't have ACID transactions.

Control theory is also used by databases (probably not as often as it should be). It's great for "self tuning" [1], for example tuning the various cache sizes a database has to max…

RE: "Distributed SQL offers elasticity (no downtime resize"). I'm not sure this is as much of an advantage of distributed databases vs single host databases anymore. Some of the te…

Databricks has kept their Photon[1][2] query engine for Spark closed sourced thus far. Unless EMR has made equivalent changes to the Spark runtime they use Databricks should be muc…

yeah, that is part of the trade off. Using an ephemeral SSD (for a database) means the database needs to have another means of making the data durable (replication, storing data in…

Its a popular design for SQL Data warehouses. I think almost all of them (snowflake, redshift, etc.) store cold data in S3 and hot data on local disk[1][2]. It works well if the da…

Yeah for workloads with any long running write transactions a single writer design is a pretty big limitation. Say some long running data load (or a big bulk deletion) running alon…

This may depend on if your use case is only vector search in isolation (ANN lookups). In this scenario pgvector is potentially not the best option ( https://ann-benchmarks.com/ ) T…

Yep, SingleStoreDB (formerly MemSQL) is not open source (probably never will be), but it does have many paying customers who have had workloads in production for over a decade at t…

Most SQL Analytical databases don't discourage joins (think BigQuery, Redshift, Snowflake, etc.) and all of them are columnstores. I think discouraging joins is something very spec…

Yep, for writes network bandwidth usage is independent of separation of storage of compute in some sense. Any database that provides high availability is writing over the network s…

There are hybrid designs for separation of storage and compute that are aimed at mixed workloads[1]. They avoid writes to remote storage on transaction commit (i.e., act like a sha…

Columnstores can do row level access by trading off a bit in terms of compression. If you organize the columnstore files as an LSM tree and use incremental compression schemes (so …

I read the article. And yes, this was probably a mistake to engage with you on it. Yandex N.V. (A dutch holding company of Yandex Russia) is listed as an investor of Clickhouse Inc…

Because half truths like your statements bother me? Yandex is a large investor in clickhouse Inc. (at last as far as public record shows). Have they forced Yandex to divest their h…

Yandex (Google of Russia) is a large share holder of Clickhouse Inc[1]. Yes, this maybe a European Yandex subsidiary.. I really don't care to dig, but I think its dishonest to say …

Almost no one does "official" TPC results these days (maybe other then Oracle and some of the Chinese vendors). Most other cloud DWs have public TPC-H or TPC-DS results that are ea…