Live data from Hacker News

Dbdev – A database package manager for PostgreSQL trusted language extensions

database.dev

11–20 of 57 posts

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#11

Can I use this on AWS RDS?

RDS does have the underlying native extension that is required to install user defined packages (pg_tle) from database.dev. It doesn't currently have support for the extension that is necessary to make the in-database client work to install packages directly from the registry.

That's great feedback though: We'll add a new page to the website for each package that provides the snippet you can run on an RDS instance to load the extension and then run `create extension`.

We're also in talks with RDS team and experimenting with a few options that would allow users to query the registry directly. The most promising one we have found so far is using AWS Lambda with RDS's aws_lambda extension to do that http part, and pass the contents back to the database. Still a WIP though!

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#12

Can I use this on AWS RDS?

No. RDS has a strict list of approved extensions. (Since AWS is on the hook for the system stability, security, replication, etc.)

Partially correct, this doesn't work with RDS as you can't install pgsql-http (a requirement for automatic install). However all the extensions listed are "Trusted Language Extensions for PostgreSQL" which was developed by AWS to enable installing extensions written in "Safe" languages. So I believe that all the extensions listed could be installed manually.

https://github.com/aws/pg_tle

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Postg...

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#13

How does `index_advisor` work?

index_advisor is a project that almost made the cut for LW7. We decided to launch it as a pgtle instead to get some feedback before we (spoiler) probably release it in LW8 It uses the hypopg package to very quickly generate an index for each column referenced in the query and then re-run's an explain plan to measure the estimated "total cost". If the expected "total_cost" reduces, we add the indexes to the list of re…

Cool idea for an extension!

I had a whole set of comments about concerns about doing this to a large table in prod, and statistics and such - then I read about hypopg.

What a neat package, learned something new today.

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#14
At KotlinConf today I gave a talk on designing apps with two-tier architecture, where you implement your entire app without the web stack appearing anywhere at all. Instead you publish desktop and mobile apps that connect directly to an RDBMS like PostgreSQL via its native protocol, and use server extensions for any logic that is inconvenient to do with SQL.

This approach might seem horrifyingly outside-the-box but has a lot of advantages, and some of the reasons we didn't do things this way historically have been solved in recent years.

Because it was KotlinConf the demo uses PL/Java, which is pretty nice because there's such a healthy ecosystem of stuff based around JDBC and because deploying JVM stuff doesn't require any sort of cross-compilation. PL/Java also supports (for now) trusted extensions using sandboxing, although of course the sandbox can just get in the way and normally you trust your own server anyway so this is a double edged sword.

The demo code can be found here (it's a prototype and nobody reviewed it yet so be gentle)

https://github.com/hydraulic-software/bugzino

I'll write up a blog post version of the talk, but for now I had to mention that DBaaS providers don't actually enable this sort of design because they like to wall off the full power of the RDBMS behind custom APIs. But in two-tier design you really lean into the database and use all of its features. So, it'd be nice if:

a. database.dev were to support PL/Java extensions.

b. Supabase were to allow direct connections, as the native DB protocol supports a lot of features that otherwise have to be sort of hacked on top of HTTP. Ultimately, HTTP is designed to fetch hypertext whereas the PG native protocol is designed to work with data, and that difference shines through in a bunch of ways.

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#15
post #8

Is this like pgxn but specific to trusted languages so presumably less installation issues?

yeah, exactly. pgxn also houses native extensions, some of which may be TLE compatible. database.dev is trusted language extension specific

We've reached out to David, who runs pgxn to see how we might collaborate and allow pgxn to be used as another registry that users can install TLEs from

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#16

From https://supabase.com/blog/dbdev > dbdev fills the same role for PostgreSQL as npm for JavaScript, pip for Python and cargo for Rust in that it enables publishing libraries and applications for repeatable deployment. We'll be releasing the tooling necessary for third-parties to publish pglets to the registry once we’ve collected some community feedback and incorporate any great new ideas. Our goal is to create an…

pgxn does not (yet) support the notion of TLEs, but it certainly could with minimal changes. We're currently in discussion with the pgxn maintainers about how that would look.

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#17

At KotlinConf today I gave a talk on designing apps with two-tier architecture, where you implement your entire app without the web stack appearing anywhere at all. Instead you publish desktop and mobile apps that connect directly to an RDBMS like PostgreSQL via its native protocol, and use server extensions for any logic that is inconvenient to do with SQL. This approach might seem horrifyingly outside-the-box but h…

Are there any performance issues with connection handling? Not sure how/where a connection pooler fits in.

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#18
post #10

What's stopping me from adding an extension to this repository that adds something valuable while simultaneously exfiltrating entire databases back to me? It's a cool idea, but I won't be the first to try it. You're placing an awful lot of trust in these modules.

Like any package or plugin for any language or framework, you are responsible for understanding what you are installing. Since TLEs do not come with any compiled code, this is relatively easy for you to verify yourself with code inspection.

We expect a lot of DBAs to say "no way" to something like TLEs, that's fine, for similar reasons people don't use pip, npm, or cpan. You make your own risk/benefit analysis.

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#19
I'd love to see plv8 added to this.

Jerry S. has done an amazing job of wrangling the insane v8 build process into something that mostly works across platforms, but installation is still not particularly simple.

Unfortunately, due to the problems with that build process, Ubuntu and other flavors dropped the prebuilt apt packages.

Making plv8 a simple install would be a huge boost to the community.

Re: Dbdev – A database package manager for PostgreSQL trusted language extensions

#20

Earlier quoted context omitted.

No. RDS has a strict list of approved extensions. (Since AWS is on the hook for the system stability, security, replication, etc.)

Partially correct, this doesn't work with RDS as you can't install pgsql-http (a requirement for automatic install). However all the extensions listed are "Trusted Language Extensions for PostgreSQL" which was developed by AWS to enable installing extensions written in "Safe" languages. So I believe that all the extensions listed could be installed manually. https://github.com/aws/pg_tle https://docs.aws.amazon.com/A…

Oh cool!
Post reply on HN