Pg_lakehouse: Query Any Data Lake from Postgres
1–10 of 73 posts
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#2Paradedb is doing a lot of good work with postgres. Pg_analytics, and now pg_lakehouse...
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#3As somebody who writes a lot of Postgres extensions, I can say this is quite interesting!
I think I can see some parallels to Supabase's wrappers project.
Keep up the good work!
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#4Neat that you plan to support both Delta Lake and Apache Iceberg
I'm curious about HN's position between these two formats? I'm having a hard time deciphering which might be the industry winner (or perhaps they both have a place, no "winner" necessary)
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#5Nice. I wish timescaledb open-sourced their s3 storage thing.
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#6Neat that you plan to support both Delta Lake and Apache Iceberg I'm curious about HN's position between these two formats? I'm having a hard time deciphering which might be the industry winner (or perhaps they both have a place, no "winner" necessary)
This is anecdotal, but I feel that we (ParadeDB) have received more requests for Iceberg integration vs. Delta Lake. We were actually hesitant to launch pg_lakehouse without Iceberg support, but pulled the trigger on it because the iceberg-rust crate is still in its early days. We will probably be contributing to iceberg-rust to make it work with pg_lakehouse.
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#7Neat that you plan to support both Delta Lake and Apache Iceberg I'm curious about HN's position between these two formats? I'm having a hard time deciphering which might be the industry winner (or perhaps they both have a place, no "winner" necessary)
This is anecdotal, but I feel that we (ParadeDB) have received more requests for Iceberg integration vs. Delta Lake. We were actually hesitant to launch pg_lakehouse without Iceberg support, but pulled the trigger on it because the iceberg-rust crate is still in its early days. We will probably be contributing to iceberg-rust to make it work with pg_lakehouse.
> We will probably be contributing to iceberg-rust to make it work with pg_lakehouse
That's great news, thanks for your contributions to open source (here, and all the other extensions)
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#8Very cool!
Could you share the key difference between this and the previous pg_analytics, and motivation of making it a separate plugin?
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#9Neat that you plan to support both Delta Lake and Apache Iceberg I'm curious about HN's position between these two formats? I'm having a hard time deciphering which might be the industry winner (or perhaps they both have a place, no "winner" necessary)
There isn't a winner and there likely won't be one (at least not for a long time). Tabular will likely be acquired by Snowflake and the two industry behemoths now back their own formats, and each will treat their own as a first class citizen.
Re: Pg_lakehouse: Query Any Data Lake from Postgres
#10Neat that you plan to support both Delta Lake and Apache Iceberg I'm curious about HN's position between these two formats? I'm having a hard time deciphering which might be the industry winner (or perhaps they both have a place, no "winner" necessary)
I'm also building in the lakehouse space and anecdotally have seen more excitement around Iceberg over delta lake just because of its completely open source origins. Iceberg has evolved faster and has had more contributions from a more diverse set of contributors than Delta Lake. Not sure if this will change with a Snowflake Tabular acquisition but I'd easily bet on Iceberg if current trends continue.