Supabase Wrappers: A framework for building Postgres foreign data wrappers

So much great postgres stuff comming out from supabase. I'm really impressed with how much focus and direction the team have. So many companies would try to broaden their product/feature suite, but fail at understanding what made the core sucsessful and stretch the company in many directions. It really seems like they are just making them self better and better experts at postgres. And how to use it well. Really looking forward to see where this is going.

On a sidenote, is not the wrappers for Airtable, BigQuery and ClickHouse opensourced? Or why did they skip that column in the second table?

Is all of supabase opensourced now? I'm meaning I heard something about function not being opensourced but I can be remembering wrong. Most stuff is on GitHub I see

The blog post, repo and your comment still leave me a little confused about what Supabase's wrappers project is, the intent and its current state.

So this "wrappers" project is just a framework that simplifies the development and manages the curation and distribution of Postgres Foreign Data Wrappers? At the end of the day, are these just FDWs that run on top of any Postgres instance? Do I have to run Supabase to take advantage of the FDWs created through this framework?

Additionally one of your examples is Snowflake, but I don't see a matching SF FDW in your repo? Is there a generic JDBC/SQL/etc FDW for any SQL based database like Snowflake, Oracle, etc? Or is this just to create a spark in someone's mind (guilty!) about what is possible and yet to be implemented?

Despite my confusion, this sounds like a very exciting project to follow.

How well do foreign data wrappers work with Postgres's query planner? Does't the planner need to estimate the cardinality of results from each table? Are the Supabase wrappers providing this info from the wrappers even though it might be hard to get from the data source?

Also, is the Firebase connector for Realtime Database or Firestore?

I don't think so... in your example those are functions we just call in SQL right? I'm talking about exposing arbitrary data as a table like an FDW, but without having to compile a whole module.

So more like, (this is a bit contrived but bear with me):

Let's say I wanted to make a table that lists all my keybase proofs (so sad Keybase is basically dead, but I digress).

I'd build a graphql endpoint where you can query those proofs.

Then I'd have to define a metadata endpoint that maybe returns a JSON schema?

Then you can tell the FDW to define a virtual table that (as much as possible) uses the JSON schema to generate columns in this virtual table.

You'd define the FDW to say, here's the virtual table (like you'd do now), and then put in an option to say "you'll get the mappings for this table with the JSON schema returned by this metadata endpoint" and then tell the FDW "send select queries to this REST/graphql endpoint" which would be something in our example that returns keybase proofs).

I wouldn't need to build a whole new keybase FDW. I'd just use the generic one to hit that existing endpoint.

Is that making sense? It's kind of a dynamic FDW that can be configured for the long tail of API endpoints.

| At the end of the day, are these just FDWs that run on top of any Postgres instance?

Yes, the FDWs developed by Wrappers can be used on any Postgres db.

| Do I have to run Supabase to take advantage of the FDWs created through this framework?

No, you don't need to. Those are just normal FDWs can be used on any Postgres db.

| Is there a generic JDBC/SQL/etc FDW for any SQL based database like Snowflake, Oracle, etc?

The Snowflake FDW isn't developed yet, the example is just for concept demo. It has ClickHouse and BigQuery FDWs although they are not JDBC based. Generic JDBC FDW will need JVM embedded so it is not in plan at this moment.

Disclaimer: I am Supabase developer.