Live data from Hacker News

Realtime Postgres Row Level Security

supabase.com

1–10 of 34 posts

Re: Realtime Postgres Row Level Security

#2
Hey HN, Supabase is an open source Firebase alternative. We're building the features of Firebase using enterprise-grade open source tools. We're particularly focused on scalability. We take proven tools like Postgres, and we make them as easy to use as Firebase.

Today, Supabase is adding Row Level Security (RLS) to our Realtime engine.

The linked blog post goes into depth around the technical implementation, so I’ll just give a recap for the people who jump straight to comments (like me).

Supabase was launched here on HN when we open sourced our Realtime engine[0] - an Elixir server which clients (i.e. website visitors/users) can connect to via websockets and receive a stream of PostgreSQL changes.

The server receives those changes via a logical replication slot - the same system that PostgreSQL uses for replicating to other databases.

To achieve RLS we added a few SQL functions, the main one is apply_rls[1] which the stream is filtered through. For every user connected to the Elixir server, the Postgres function checks if they have access to the database change and appends an array of allowed user IDs. The Realtime server then delivers the change to the user only if the connected user is matched in this array.

This one has been a long time coming, and it's one of the reasons why we have maintained our "beta" badge for so long. A few of the team will be here to answer any questions - my cofounder @awalias and @steve-chavez from PostgREST, @inian, @wenbo and @1_over_n

[0] Realtime Show HN: https://news.ycombinator.com/item?id=22114560)

[1] SQL function: https://github.com/supabase/realtime/blob/master/server/priv...

Re: Realtime Postgres Row Level Security

#3
We currently make use of Supabase and it's been fantastic. It's enabled us to completely get away from having a traditional backend/API by utilizing RLS and it's realtime nature to have data directly in the UI. Rough numbers are showing we cut development time by a third vs a traditional approach.

Re: Realtime Postgres Row Level Security

#4
post #3

We currently make use of Supabase and it's been fantastic. It's enabled us to completely get away from having a traditional backend/API by utilizing RLS and it's realtime nature to have data directly in the UI. Rough numbers are showing we cut development time by a third vs a traditional approach.

I'm on a POC project that's using PostgREST and it's been extremely fast to get a big complicated data model working with an API in front of it. But I guess I don't get how to really use this thing in reality? What does devops look like? Do you have sophisticated db migrations with every deploy? Is all the SQL in version control?

I also don't really get where the users get created in postgres that have all the row-level permissions. The docs are all about auth for users that are already in there.

Re: Realtime Postgres Row Level Security

#5
The row level security people should talk to the capability security people.

Probably the main problem with RDBMS is also it's biggest benefit: you write code that naively looks some bad polynomial time, and then query planner does the magic and use indices. The problem is, of course, when the index doesn't exist or the query planner doesn't do a good job.

What would be nice is a way to mandate that queries must go through certain indices. But this is in fact good for row level security too!

Functional programming is deeply related to capability-based security, in that both try to get rid of all ambient authority and mediate all actions through some sort of reference. Rust, with it's many types of reference makes the relationship even clearer. From these vantage points, it is very natural to use data structure both to make things performant and enforce certain invariant.

Indices are the closes data structures in RDBMSs, because they do promote only certain access patterns. With the rule that queries must go through the indices, they enforce those access pattern, and thus we have the same proven tool. Combine them with views for filtering out data, (which we can think in conjunction with more indices of as sort of a curried indexes Map> even if they aren't implemented that way), and now we have very powerful tools for guaranteeing performance and security.

As an icing on the cake, I think enforcing limited access patterns could get us closer to that long-sought goal of a RMBDS that automatically knows how to distribute itself. The limited access patterns open up possible options, because we which of the pathological joins that every sharding possibility run into are no longer allowed. Reusing the same query planner tricks, "cardinality analysis" type stuff can be used to choose between those possibilities.

Here's a worked out concrete example:

Tables:

   User: { k: UserID PK, t: TennentId, n: Name, isAdmin: Bool },
   Todo: { k: TodoID PK, u: UserID, d: Date, m: Msg } 
Indices and Views:

# exists_classical means proof of existence but we don't care which.

    Map
    Map
^ that's a joined view with compression syntax

# exists_constructive means need to show existence and we do see what item witness it

    Map>

    Map>

The more worked out system would probably allow tupling stuff to not repeat the "MapA few things to note:

1. When we do a comprehension syntax, we are "returning the whole table" i.e. an unconstrained view that can be queried in an arbitrary way. A joined table has to be joined in the given way, but is otherwise unconstrained.

2. exist_constructive is the same as a { ... } comprehension, but the cardinality must be 1 as proven by the unique constraints. exists_classical is a comprehension for an arbitrary number of stuff, that is then truncated in the type theory sense. https://xenaproject.wordpress.com/2021/05/19/the-trace-of-an... is a good discussion of an example of this truncation.

3. Normally one puts the TennentID on every table, even though it is denormalizing. Now the access patterns enforce this.

Clearly this is a few PhD theses away from actually existing :), but all the embryonic theory comes from good well-established things, which makes me confident.

Re: Realtime Postgres Row Level Security

#6
post #4
post #3

We currently make use of Supabase and it's been fantastic. It's enabled us to completely get away from having a traditional backend/API by utilizing RLS and it's realtime nature to have data directly in the UI. Rough numbers are showing we cut development time by a third vs a traditional approach.

I'm on a POC project that's using PostgREST and it's been extremely fast to get a big complicated data model working with an API in front of it. But I guess I don't get how to really use this thing in reality? What does devops look like? Do you have sophisticated db migrations with every deploy? Is all the SQL in version control? I also don't really get where the users get created in postgres that have all the row-le…

In Supabase we use a separate Auth server [0]. This stores the user in an `auth` schema, and these users can login to receive a JWT. Inside the JWT is a "role", which is, in fact, a PostgreSQL role ("authenticated") that has certain grants associated to it, and the user ID (a UUID).

Inside your RLS Policies you can use anything stored inside the JWT. My cofounder made a video [1] on this which is quite concise. Our way of handling this is just an extension of the PostgREST Auth recommendations: https://postgrest.org/en/v9.0/auth.html

[0] Auth server: https://github.com/supabase/gotrue

[1] RLS Video: https://supabase.com/docs/learn/auth-deep-dive/auth-row-leve...

Re: Realtime Postgres Row Level Security

#7
post #3

We currently make use of Supabase and it's been fantastic. It's enabled us to completely get away from having a traditional backend/API by utilizing RLS and it's realtime nature to have data directly in the UI. Rough numbers are showing we cut development time by a third vs a traditional approach.

When I think of postgrest, supabase and other tools that allows you skip the backend completely and go straight to the DB; is how do you handle business logic that doesn't make sense to have in either the frontend or the DB ?

Re: Realtime Postgres Row Level Security

#8
I've been developing an app on postgrest (which supabase uses) for a while, and my solution to this was to have a NOTIFY trigger and a small program that broadcasted the changed ID's over SSE to clients. The clients would then fetch the changes.

As for the RLS aspect I considered it "good enough" to allow all clients to know the changed ID (an UUID) as long as they got no other info, and planned to have the trigger also send the allowed groups in the NOTIFY so that the program could filter without needing a separate PG connection per each subscriber. In my testing it scales very well.

Anyone from supabase that can comment why LISTEN/NOTIFY triggers were not used, and why WS was used over SSE when the communication is not bi-directional? It'd be interesting to hear your thoughts.

Re: Realtime Postgres Row Level Security

#9
post #3

We currently make use of Supabase and it's been fantastic. It's enabled us to completely get away from having a traditional backend/API by utilizing RLS and it's realtime nature to have data directly in the UI. Rough numbers are showing we cut development time by a third vs a traditional approach.

When I think of postgrest, supabase and other tools that allows you skip the backend completely and go straight to the DB; is how do you handle business logic that doesn't make sense to have in either the frontend or the DB ?

You can do business logic in stored procedures, functions and views.

Re: Realtime Postgres Row Level Security

#10
post #8

I've been developing an app on postgrest (which supabase uses) for a while, and my solution to this was to have a NOTIFY trigger and a small program that broadcasted the changed ID's over SSE to clients. The clients would then fetch the changes. As for the RLS aspect I considered it "good enough" to allow all clients to know the changed ID (an UUID) as long as they got no other info, and planned to have the trigger a…

> LISTEN/NOTIFY triggers were not used

We started with this implementation, however there are 2 issues with this approach. 1) NOTIFY has a limit of 8000 bytes, so large rows are silently dropped. 2) you need to attach the notify trigger to every table (vs a Publication, which can listen to an entire database in a single line of code).

> why WS was used

The Server is Elixir (Erlang) so it's very good at WS. Over time we will add other multiplayer useful features (like presence)

Post reply on HN