Veni, Vidi, Vici
Nice
SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
71–78 of 78 posts
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#72Earlier quoted context omitted.
for one thing: https://github.com/pg-sharding/spqr/blob/1.3.0/LICENSE (BSD2) https://github.com/citusdata/citus/blob/v12.1.2/LICENSE (AGPLv3)
I still have a hard time understanding where the AGPL copyleft feature kicks in on a product like Citus. Are you safe just deploying it as part of your backend? Like, do you have to actually touch Citus's own code to trigger it? What if you start using some sprocs or sample connectivity code? Anybody know a good set of guidelines?
This is the entirety of the text which was added to the AGPL causing it to differ from the plain GPL:
> Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing access to the Corresponding Source from a network server at no charge, through some standard or customary means of facilitating copying of software. This Corresponding Source shall include the Corresponding Source for any work covered by version 3 of the GNU General Public License that is incorporated pursuant to the following paragraph.
In other words, if you deploy a modified version of Citus, your modified version is AGPL licensed, and thus you must provide the source code of this modified version to all users who interact with it remotely (e.g. through your web application).
What it does not state is that you must provide the source code of your entire web application just because you deployed a modified version of Citus, nor that your web application becomes an AGPL-licensed derived work of Citus because you used it over a network. The AGPL also does not require anything at all from you other than the plain GPL's basic requirements if the version of Citus you deploy is unmodified. (These are all incredibly common misconceptions on the internet, by people who've never read the license nor the GNU website.)
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#73Earlier quoted context omitted.
I recently did some db maintenance on a write heavy workload and I found that eventually it will bloat over time with a table with 500 million records. Switching it to use a proper partitioning scheme helped a lot. So people should not read this and assume you can just dump massive workloads into pg and they will be screamingly performant without some tuning and thoughtful design (I don’t think this is what you are i…
is there a chance you run some older version of PG? They reduced bloating significantly in last few releases.
Yes! PG12 in this case. Thanks, this definitely motivates my inflight project to get these db’s up to PG16. Very happy to hear that
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#74Earlier quoted context omitted.
I still have a hard time understanding where the AGPL copyleft feature kicks in on a product like Citus. Are you safe just deploying it as part of your backend? Like, do you have to actually touch Citus's own code to trigger it? What if you start using some sprocs or sample connectivity code? Anybody know a good set of guidelines?
This is the entirety of the text which was added to the AGPL causing it to differ from the plain GPL: > Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing a…
This is a different understand that I had on AGPL 3 quote FSF
"The AGPLv3 does not adjust or expand the definition of conveying. Instead, it includes an additional right that if the program is expressly designed to accept user requests and send responses over a network, the user is entitled to receive the source code of the version being used."
Have I misread this? As I understand AGPL3 is an extension of GPL 3. Therefore means if I use an AGPL3 licensed service I have rights to receive the code for this service.
ref: https://www.fsf.org/bulletin/2021/fall/the-fundamentals-of-t...
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#75Earlier quoted context omitted.
What I talk about is blocking of limited resources. Processes/connections are expensive and therefore you want/have to limit the max amount of them. Each query/transaction requires its own process and blocks it for everyone else until it is done. I/O or compute constraints are another issue, if your CPU or disc is already saturated you get probably no additional benefit. But if you wait for something (locks, I/O) the…
What I meant is, even if processes had no cost, you will probably bottleneck on the I/O. So it doesn't matter a lot how many Postgres connections you have.
Sure, it’s not the most common problem to experience, just something that’s useful to know once you have more and more clients. It did happen to me, all connections used up for long running transactions (for multiple reasons) and then nothing else was able to run, CPU and I/O nearly idling at <5%, no significant lock contention.
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#76Earlier quoted context omitted.
is there a chance you run some older version of PG? They reduced bloating significantly in last few releases.
Yes! PG12 in this case. Thanks, this definitely motivates my inflight project to get these db’s up to PG16. Very happy to hear that
If you need more motivation check out the feature matrix, there have been a fair number of pretty big advances since 12.
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#77Earlier quoted context omitted.
What I meant is, even if processes had no cost, you will probably bottleneck on the I/O. So it doesn't matter a lot how many Postgres connections you have.
Sure, it’s not the most common problem to experience, just something that’s useful to know once you have more and more clients. It did happen to me, all connections used up for long running transactions (for multiple reasons) and then nothing else was able to run, CPU and I/O nearly idling at <5%, no significant lock contention.
Ah, long-running transactions wreak havoc with connection pools.
Re: SPQR 1.3.0: a production-ready system for horizontal scaling of PostgreSQL
#78Earlier quoted context omitted.
This is the entirety of the text which was added to the AGPL causing it to differ from the plain GPL: > Notwithstanding any other provision of this License, if you modify the Program, your modified version must prominently offer all users interacting with it remotely through a computer network (if your version supports such interaction) an opportunity to receive the Corresponding Source of your version by providing a…
This is a different understand that I had on AGPL 3 quote FSF "The AGPLv3 does not adjust or expand the definition of conveying. Instead, it includes an additional right that if the program is expressly designed to accept user requests and send responses over a network, the user is entitled to receive the source code of the version being used." Have I misread this? As I understand AGPL3 is an extension of GPL 3. Ther…
The key word there is "the version being used". If you're using an unmodified version of Citus, the user is free to go get the source code from GitHub.
That's why the text of the clause itself makes a distinction about "if you modify the Program".