Live data from Hacker News

Pg_lakehouse: Query Any Data Lake from Postgres

github.com

21–30 of 73 posts

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#21
post #19

This is great work! Could you please comment on the choice of your license. Lost Postgres extension that achieve wide adoption use Postgres, MIT or Apache license.

All ParadeDB extensions are released under AGPL-3.0. We've found that it strikes the right balance between being open-source and enabling the community to adopt for free, while also protecting us from hyperscalers and enabling us to build a sustainable business. Perhaps the topic of a blog post someday :)

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#22

Neat that you plan to support both Delta Lake and Apache Iceberg I'm curious about HN's position between these two formats? I'm having a hard time deciphering which might be the industry winner (or perhaps they both have a place, no "winner" necessary)

There isn't a winner and there likely won't be one (at least not for a long time). Tabular will likely be acquired by Snowflake and the two industry behemoths now back their own formats, and each will treat their own as a first class citizen.

Agreed, this is why we want to support both. Maybe even Apache Hudi down the line. But I hope the industry converges to a main standard rather than Snowflake/Databricks fighting for their own formats. They can differentiate on much more meaningful features

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#24
post #19

This is great work! Could you please comment on the choice of your license. Lost Postgres extension that achieve wide adoption use Postgres, MIT or Apache license.

All ParadeDB extensions are released under AGPL-3.0. We've found that it strikes the right balance between being open-source and enabling the community to adopt for free, while also protecting us from hyperscalers and enabling us to build a sustainable business. Perhaps the topic of a blog post someday :)

It looks like hyper scalers can still host it as long as they are publishing changes to the source code ? Am I reading the license right ?

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#25

The name seems to be an allusion to the author P.G. Wodehouse, creator of the character Jeeves. https://en.wikipedia.org/wiki/P._G._Wodehouse Very clever naming!

Sorry, what do you base that on? To me it just seems like a straightforward inspiration from the "data lake" -> "lakehouse" terminology that Databricks started (?) using.

https://www.databricks.com/product/data-lakehouse

edit: ah, but in a different comment someone noted that it's not actually a lakehouse, so who knows!? :)

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#27
post #25

The name seems to be an allusion to the author P.G. Wodehouse, creator of the character Jeeves. https://en.wikipedia.org/wiki/P._G._Wodehouse Very clever naming!

Sorry, what do you base that on? To me it just seems like a straightforward inspiration from the "data lake" -> "lakehouse" terminology that Databricks started (?) using. https://www.databricks.com/product/data-lakehouse edit: ah, but in a different comment someone noted that it's not actually a lakehouse, so who knows!? :)

Based on pure speculation. I may be reaching.

My best guess is that Databricks and Pg_lakehouse both independently coined "lakehouse" from "data lake", and that for the latter team, it was partly a pun on Wodehouse. But the creators are welcome to chime in and confirm/deny!

(Or to say, like, "Sure...uh, we totally meant that...yes we are very literary.")

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#28
post #24

Earlier quoted context omitted.

All ParadeDB extensions are released under AGPL-3.0. We've found that it strikes the right balance between being open-source and enabling the community to adopt for free, while also protecting us from hyperscalers and enabling us to build a sustainable business. Perhaps the topic of a blog post someday :)

It looks like hyper scalers can still host it as long as they are publishing changes to the source code ? Am I reading the license right ?

For aws to make it available on rds Aurora, would it be safe to assume there would have to be some changes to the extension source to make it compatible with the Aurora engine? If we assume aws doesn’t want todo that, then their licensing provides some protection there.

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#29
post #24

Earlier quoted context omitted.

It looks like hyper scalers can still host it as long as they are publishing changes to the source code ? Am I reading the license right ?

For aws to make it available on rds Aurora, would it be safe to assume there would have to be some changes to the extension source to make it compatible with the Aurora engine? If we assume aws doesn’t want todo that, then their licensing provides some protection there.

Yeah exactly. In practice, we've inspired ourselves from the likes of Citus and others who have adopted the AGPL-3.0 license as a good compromise and have found success. It's rather comment nowadays for infra startups to use AGPL-3.0. Other noteworthy examples: MinIO, Quickwit, Lago, etc.

Re: Pg_lakehouse: Query Any Data Lake from Postgres

#30
post #25

Earlier quoted context omitted.

Sorry, what do you base that on? To me it just seems like a straightforward inspiration from the "data lake" -> "lakehouse" terminology that Databricks started (?) using. https://www.databricks.com/product/data-lakehouse edit: ah, but in a different comment someone noted that it's not actually a lakehouse, so who knows!? :)

Based on pure speculation. I may be reaching. My best guess is that Databricks and Pg_lakehouse both independently coined "lakehouse" from "data lake", and that for the latter team, it was partly a pun on Wodehouse. But the creators are welcome to chime in and confirm/deny! (Or to say, like, "Sure...uh, we totally meant that...yes we are very literary.")

I wish we were that clever, but it's really just the combination of "data lake" and "data warehouse", which isn't even coined by us :)
Post reply on HN