Citus 12: Schema-based sharding for PostgreSQL
citusdata.com
Citus 12: Schema-based sharding for PostgreSQL
1–10 of 47 posts
Re: Citus 12: Schema-based sharding for PostgreSQL
#2Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here?
My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Re: Citus 12: Schema-based sharding for PostgreSQL
#3Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Schema is a proper noun in Postgres: https://www.postgresql.org/docs/current/ddl-schemas.html
Re: Citus 12: Schema-based sharding for PostgreSQL
#4Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Schemas are groupings of tables and other entities that can be defined within a database. You can think of them like of a namespace in programming languages. You can have the same table definition, within the same database, defined multiple times (each in a different schema) and each holding different data.
By large and small we are referring to the amount of data each schema holds currently. They can grow over time and some of them may become very big while others will remain small (storage wise).
Re: Citus 12: Schema-based sharding for PostgreSQL
#5Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
I've seen MySQL clients/docs also use "schema" to refer to "this collection of tables", basically what SQLite would call "a database".
The words get really fuzzy here when you try to get a concrete and ubiquitous definition, though.
Best I can do is to say, in these queries:
use foo;
select id, name from foo.people where id Re: Citus 12: Schema-based sharding for PostgreSQL
#6Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Not to be rude...but the same things as every other ANSI SQL database. Nothing PostgreSQL specific here.
Schema is the thing you get from `CREATE SCHEMA`...a namespace of tables/functions/views/etc.
Re: Citus 12: Schema-based sharding for PostgreSQL
#7Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Schemas are namespaces (actually called that internally in Postgres).
The SQL standard defines a two level namespace hierarchy. A single "instance" of contains multiple catalogs and each catalog contains multiple schemas (and each schema then contains objects like tables, views, types, functions etc).
Many database products use the term "database" instead of "catalog" e.g. in Postgres and SQL Server. But "schema" is used quite uniformly. MySQL's "databases" are in fact "schemas" though.
Re: Citus 12: Schema-based sharding for PostgreSQL
#8Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Not to be rude...but the same things as every other ANSI SQL database. Nothing PostgreSQL specific here. Schema is the thing you get from `CREATE SCHEMA`...a namespace of tables/functions/views/etc.
Ah I see! Yeah I guess it's just a concept I've never run into before; unfortunate that it shares a name with the "other" concept of schema used by e.g. https://json-schema.org
Thank you!
Re: Citus 12: Schema-based sharding for PostgreSQL
#9Nice to see this on HN :)
The high-level is: You enable a setting and every CREATE SCHEMA creates a new shard. All the tables in the schema will be co-located so you can have efficient joins & foreign keys between the tables.
On top of that, you can also have reference tables that are replicated to all nodes, again for fast joins & foreign keys with all schemas.
Everything else is about making every PostgreSQL feature work as seamlessly as if there was no sharding. You can still do things like transactions across schemas, create and use custom types, access controls, work with other extensions, use procedures, etc.
Re: Citus 12: Schema-based sharding for PostgreSQL
#10Hm, question for people a bit more familiar with Postgres -- what is meant by "schema" here? My definition is "the columns and column types of a table", but, that doesn't seem to make sense with what they're talking about here ("large" and "small" schemas probably aren't referring to wide and narrow tables for example, and I don't see how sharding by my definition of "schema" could even make sense anyways)
Schemas are namespaces (actually called that internally in Postgres). The SQL standard defines a two level namespace hierarchy. A single "instance" of contains multiple catalogs and each catalog contains multiple schemas (and each schema then contains objects like tables, views, types, functions etc). Many database products use the term "database" instead of "catalog" e.g. in Postgres and SQL Server. But "schema" is…
Whew boy, that's a more sizable hierarchy than I thought was going on under the hood lol, I guess I've got some reading to do. TY for the pointers!