Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Citus 12: Schema-based sharding for PostgreSQL

citusdata.com

Post reply on HN
Print view View original

21–30 of 47 posts

Re: Citus 12: Schema-based sharding for PostgreSQL

#21

Post by AlexB138 »

mariocesar wrote: post #18

I'm having trouble getting CitusDB to work with RDS, even though I'd really like to use it with AWS. Whenever I try to research how to make it happen, I get stuck in a lot of challenges and end up concluding that it might not be possible. While I could use Azure instead, I'm hesitant because I have a lot of resources and infrastructure in AWS and it wouldn't make sense to move the database layer. The idea of having t…

Microsoft isn't its "partner", they were entirely bought by Microsoft. The goal is to make the technology an advantage for Azure, as I understand it. It's very unlikely you'll see them offer first-party support for RDS, or CloudSQL.

Re: Citus 12: Schema-based sharding for PostgreSQL

#23

Post by mulander »

skunkworker wrote: post #22

Having used schema based sharding in postgres before, I hope citus has a way around connection pooling and pgbouncer, as if you use pgbouncer transaction pooling your search_path could disappear at any time.

Great observation! :)

We worked upstream to have `search_path` properly handled (tracked per client) by pgbouncer.

https://github.com/pgbouncer/pgbouncer/commit/8c18fc4d213ad4...

Check config.md in that commit for a verbose, humanized description.

Re: Citus 12: Schema-based sharding for PostgreSQL

#25

Post by brightball »

metadat wrote: post #14

Does Oracle support anything like this? Or any other DBMS in widespread use, for that matter? The promise behind this approach to DB sharding has great potential. Simultaneously impressive, novel, and badass. I wish this had been available ten years ago at a few of my startups!

I watched a Citus demo a few months back and it blew me away. I hear upgrades are the big challenge.

Re: Citus 12: Schema-based sharding for PostgreSQL

#26

Post by rad_gruchalski »

nine_k wrote: post #20
asah wrote: post #12

love citus! this is useful. one gotcha: schemas are a weird old thing in SQL that are kinda the worst of all worlds, basically more like prefixing your SQL object names than a real level of indirection. Schema objects can't be manipulated as a batch, they don't provide the isolation / access control of databases, and can't be manipulated in DML and require stored procedures to manipulate.

Schemas are namespaces, a convenience feature, not a security feature.

That's exactly the reason I went for YugabyteDB tablespaces implementation on one of past projects. Dedicated database locked down to selected nodes for full isolation.

Re: Citus 12: Schema-based sharding for PostgreSQL

#27

Post by __s »

What's advantage over having tenant id as distribution column? Seems like you make schema name the distribution column. Maybe gross setups where same name function definition varies between schemas (been there done that, don't want to do it again)

Seems like article only offers ease of use. Guess I've never used microservices enough to consider that use case

Couldn't the microservice case be handled by having distributed tables with no distribution column? ie today I'd create a distribution column & fill it with the same value on every row

Can one have a reference schema which can be efficiently used alongside every other schema? Guess that's public schema with create_reference_table/create_distributed_function

Re: Citus 12: Schema-based sharding for PostgreSQL

#28

Post by janee »

AlexB138 wrote: post #21
mariocesar wrote: post #18

I'm having trouble getting CitusDB to work with RDS, even though I'd really like to use it with AWS. Whenever I try to research how to make it happen, I get stuck in a lot of challenges and end up concluding that it might not be possible. While I could use Azure instead, I'm hesitant because I have a lot of resources and infrastructure in AWS and it wouldn't make sense to move the database layer. The idea of having t…

Microsoft isn't its "partner", they were entirely bought by Microsoft. The goal is to make the technology an advantage for Azure, as I understand it. It's very unlikely you'll see them offer first-party support for RDS, or CloudSQL.

Yeah it's a bit of a shame they were bought by one of the cloud providers.

Would've been great if something like Cruncy Bridge, PgAnalyze and Citus were merged into one company so that you have managed Postgres over the 3 major providers with the best pg tech out the box.

Having your DB and apps in different providers is a stuff up, the latency and data transfer costs killed us, so Citus was a non start (we don't want to manage it ourselves in EC2).

Re: Citus 12: Schema-based sharding for PostgreSQL

#29

Post by stevefan1999 »

What happens if one node lost their shards due to external event? (e.g. Disk corruption, physcially destroyed like OVH Cloud) I do understand we still have to actively backup but I rather not serve any data than serving wrong data at the time of severe outage. Also I want to see any forward error correction code (FECC) would be implemented in Citus so we can do this on the fly rather than relying on RAID, e.g. RAID10, RAIDZ1, RAIDZ2

Re: Citus 12: Schema-based sharding for PostgreSQL

#30

Post by stevefan1999 »

mariocesar wrote: post #18

I'm having trouble getting CitusDB to work with RDS, even though I'd really like to use it with AWS. Whenever I try to research how to make it happen, I get stuck in a lot of challenges and end up concluding that it might not be possible. While I could use Azure instead, I'm hesitant because I have a lot of resources and infrastructure in AWS and it wouldn't make sense to move the database layer. The idea of having t…

Checkout stackgres: https://stackgres.io/install/

They are offering Citus integration

Post reply on HN