Live data from Hacker News

Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

thenile.dev

31–40 of 123 posts

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#31

Earlier quoted context omitted.

Legacy. If you have thousands of lines of code relying on Oracle the cost to migrate would be enormous.

Amazon has a great post on this topic. https://aws.amazon.com/blogs/aws/migration-complete-amazons-... I thought it was cool they retrained their Oracle DBAs into other roles as part of the project.

I work with a few former-Oracle DBAs in a PostgreSQL-flavored consultancy now and they are aces. All the root-cause analysis and organization skills transfer handily.

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#32
post #25

Is having to write "SELECT [...] WHERE user_id= " really considered a security hole? Isn't that how like every service in existence operates? Coming up with complicated auth systems and patterns just because you are scared you will accidentally skip that WHERE clause seems bizarre to me.

I think this is mainly an issue when you're using RAW SQL statements. If you're using an ORM, there are many ways to add a where clause to the statements automatically without having to update your code every where.

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#34
Row-level security is always a tricky and hard to enforce assumption as this not how we relational databases really.

Much bigger fan of the approach described here:

Scalability, Allocation, and Processing of Data for Multitenancy

https://stratoflow.com/data-scalability-allocation-processin...

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#35

We're currently using the schema-per-tenant, and it's working very well for us: * No extra operational overhead, it's just one database * Allows to delete a single schema, useful for GDPR compliance * Allows to easily backup/restore a single schema * Easier to view and reason about the data from an admin point of view * An issue in a single tenant doesn't affect other tenants * Downtime for maintenance is shorter (e.…

There are some other cons:

Memory usage and I/O can be less efficient. Postgres handles table data in 8kb pages, so even if you're just reading a single row, that reads 8kb from disk and puts 8kb in the Postgres buffer cache, with that row and whatever happens to be next to it in the physical layout of the underlying table. Postgres does this because of locality of reference: it's cheaper to bulk-load data from disk, and, statistically speaking, you may need the adjacent data soon. If each user is touching separate tables, you're loading a page per row for each user, and you're missing out on some of the locality benefits.

Another problem is monitoring (disclosure: I work for pganalyze, which offers a Postgres monitoring service). The pg_stat_statements extension can track execution stats of all normalized queries in your database, and that's a very useful tool to find and address performance problems. But whereas queries like "SELECT * FROM posts WHERE user_id = 123" and "SELECT * FROM posts WHERE user_id = 345" normalize to the same thing, schema-qualified queries like "SELECT * FROM user_123.posts" and "SELECT * FROM user_345.posts" normalize to different things, so you cannot easily consider their performance in aggregate (not to mention bloating pg_stat_statements by tracking so many distinct query stats). This is the case even when you're using search_path so that your schema is not explicitly in your query text.

Also, performance of tools like pg_dump is not great with a ton of database objects (tables and schemas) and, e.g., you can run into max_locks_per_transaction [1] limits, and changing that requires a server restart.

I wouldn't say you should never do schema-based multi-tenancy (you point out some good advantages above), but I'd be extremely skeptical of using it in situations where you expect to have a lot of users.

[1]: https://www.postgresql.org/docs/current/runtime-config-locks...

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#36
>Another issue we caught during testing was that some requests were being authorized with a previous request’s user id.

This is the terrifying part about RLS to me: having to rely on managing the user id as part of the database connection session seems like an easy way to shoot yourself in the foot (especially when combined with connection pooling). Adding WHERE clauses everywhere isn't great, but at least it's explicit.

That said, I've never used RLS, and I am pretty curious: it does seem like a great solution other than that one gotcha.

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#37
post #25

Is having to write "SELECT [...] WHERE user_id= " really considered a security hole? Isn't that how like every service in existence operates? Coming up with complicated auth systems and patterns just because you are scared you will accidentally skip that WHERE clause seems bizarre to me.

From my perspective, it isn’t the security aspects that are limiting, but the usability.

If you want to have any access controls that isn’t a simple user_id==123, SQL WHERE clauses can get complicated.

Users, groups, or any kind of fine grained access control can make simple queries non-trivial. It’s even worse if a user can be authorized to view data across different accounts.

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#38
post #16
post #12

Using RLS to implement multi-tenancy is a terrible idea. Just deploy a database per tenant. It's not hard. Why overcomplicate it?

Deploying a database per tenant is not that easy. You have a lot of new overhead, migrations become a pain in the ass (already are) and a lot of other little problems... I would say a database per tenant is overcomplicating it.

A database per tenant makes the rest of the workflow significantly easier though. No need to add clauses to SQL WHERE statements for users/groups. Queries are faster (less data). And data can be moved much easier between servers.

Yes, it does add extra overhead at account creation, during DB migrations, and for backups.

But if you don’t need cross-account or public data access, it can make life much easier.

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#39
post #4

Earlier quoted context omitted.

Every B2B client who asked us how we handle multi-tenancy also asked how we ensure their data is erased at the end of the contract. Using a shared database with RLS means you have to go through all DB backups, delete individual rows for that tenant, then re-generate the backup. That’s a non-starter, so we opted for having one DB per tenant which also makes sharding, scaling, balancing, and handling data-residency cha…

I filled out a ton of enterprise questionnaires on this stuff before and we just told people that it would be deleted when the backups expired after X days because we didn't have the capability to delete specific rows from our backups. Nobody ever argued. There's not a single customer I've ever run across who's going to halt a contract because you can't purge their data from your backups fast enough. They're signing…

i did the same with the same results

Re: Shipping Multi-Tenant SaaS Using Postgres Row-Level Security

#40

Earlier quoted context omitted.

Amazon has a great post on this topic. https://aws.amazon.com/blogs/aws/migration-complete-amazons-... I thought it was cool they retrained their Oracle DBAs into other roles as part of the project.

I work with a few former-Oracle DBAs in a PostgreSQL-flavored consultancy now and they are aces. All the root-cause analysis and organization skills transfer handily.

Postgres is functionally and conceptually extremely similar to Oracle. There are a few oddities (in particular, oracle's "nulls are never in indexes" is kinda weird) but the redo log is similar to the WAL, etc. In most cases, similar approaches will perform similarly and experience pretty much transfers over with a few months of experience.
Post reply on HN