The more I read about WASM the more magical it becomes...
Postgres WASM
91–100 of 190 posts
Re: Postgres WASM
#92I was wondering if anyone had thought about using this to experiment with the planner.
The engineering and support teams at Greenplum, a fork of Postgres, have a tool (minirepro[0]) which, given a sql query, can grab a minimal set of DDLs and the associated statistics for the tables involved in the query that can then be loaded into a "local" GPDB instance. Having the DDL and the statistics meant the team was able to debug issues in the optimizer (example [1]), without having access to a full set of data. This approach, if my understanding is correct, could be enabled in the browser with this Postgres WASM capability.
[0] https://github.com/greenplum-db/gpdb/blob/6X_STABLE/gpMgmt/b...
[1] https://github.com/greenplum-db/gpdb/issues/5740#issuecommen... (has an example output)
Re: Postgres WASM
#93No post body was provided.
Re: Postgres WASM
#94Aside from folks who already have Postgres infrastructure they want to replicate, what sort of applications would find this a better fit than using SQLite on the client side?
Re: Postgres WASM
#95The use cases here are going to be really wide spread in my opinion, just a few ideas off the cuff. Obviously the 30mb size means it won't really be for regular consumer apps, but for enterprise or specific tasks it can make a lot sense. 1. Training websites 2. Interview challenges involving SQL 3. Client side tooling that loads data into your local machine and displays into a SaaS web app without the SaaS app ever h…
> Obviously the 30mb size means it won't really be for regular consumer apps
You know that it will end up being used for regular consumer apps. And once everyone is doing it, regular web pages being over 30MB and including an enterprise-grade SQL server engine will simply be accepted as normal, and everyone not doing it is a luddite.
Re: Postgres WASM
#96Interesting. I have written a translation layer for the MongoDB wire protocol [0] that persists to PostgreSQL. I am wondering if I could make this layer also compile to wasm and run in the browser. --- [0] http://oxidedb.com
Wow, this is awesome! So as I understand it, this uses Postgres and JSON fields to emulate the features of MongoDB, sort of as an abstraction?
Thank you! It's exactly that: basically you don't need to keep an additional MongoDB instance if you already use and maintain a Postgres server somewhere.
I translate the JSON-based query interface into the corresponding SQL statements, leveraging the excellent JSON support that PostgreSQL offers.
Re: Postgres WASM
#97Peter from Snaplet here. A month ago I saw the CrunchyData post and wanted to play around with the code that made it happen, it wasn't OSS so I asked for help: > If anyone out there wants to work on an open source version of this full-time please reach out to me. [0] Paul reached out and we started working on it almost immediately. Check out the repo here: https://github.com/snaplet/postgres-wasm We have a blog post…
Have you looked at compressing that ~16MB wasm file with something like Brotli?
When I was actively doing stuff with wasm (~2019), Brotli was the best compression approach. eg 16MB uncompressed -> 2.4MB compressed
https://github.com/golang/go/wiki/WebAssembly#reducing-the-s...
Re: Postgres WASM
#98Hey HN, we’re excited about this launch. This was a collaborative effort with the team at Snaplet [0]. postgres-wasm is an embeddable Linux VM with Postgres installed, which runs inside a browser. It provides some neat features: persisting state to browser, restoring from pg_dump, logical replication from a remote database, etc. The idea was inspired by CrunchyData’s HN post about a month ago [1]. We love the possibi…
> an embeddable Linux VM with Postgres installed, which runs inside a browser.
Wow! I feel like this is the lede. How much work was done supporting the VM and OS privatives (eg networking) vs PG specific work? I feel like a minimal Linux in the browser opens up a LOT more opportunities than just a database.
When figma got bought out, a lot of articles were written about “where’s the wasm applications”, and I feel like throwing Linux into a browser really shows potential. One commenter already wondered if it could be used to compile microcontrollers (so creative, i now want that too), I wonder if it can be used similar to Repl.it, with packaging test environments.
To be very, very clear, I would LOVE a write up about just the linux portion of this interesting project.
Re: Postgres WASM
#99Peter from Snaplet here. A month ago I saw the CrunchyData post and wanted to play around with the code that made it happen, it wasn't OSS so I asked for help: > If anyone out there wants to work on an open source version of this full-time please reach out to me. [0] Paul reached out and we started working on it almost immediately. Check out the repo here: https://github.com/snaplet/postgres-wasm We have a blog post…
Nice, I wonder if anyone has got Django with postgres working in this environment?
Re: Postgres WASM
#100The first thing to point out is that our implementation isn't pure WASM. We attempted to compile Postgres for WASM directly from source, but it was more complicated that we anticipated. Crunchy's HN post provided some hints about the approach they took, which was to virtualize a machine in the browser. We pursued this strategy too, settling on v86 which emulates an x86-compatible CPU and hardware in the browser. I’m…
The problem is you need operating system features to run Postgres as-is, e.g. mapping memory, forking processes, manipulating files. What is missing is a WASM kernel that skips the x86 emulation but implements enough of the other stuff. For example, for just one of many hairy problems, consider that Postgres uses global variables in each backend for backend-local state (global state as such is in shared memory). How…
Thanks, those are specific requirements I could definitely see WASM struggling to meet.
In my experience in a large+mature enough codebase (particularly one that is already multi-platform, like Postgres appears to be) many of those requirements are wrapped in an abstraction layer to allow targeting new platforms, but some requirements (like memory mapping) could definitely be dealbreakers if the target platform doesn't naturally support them.
This solution still seems awfully complex (and probably not very efficient) but I certainly see why it's probably the "easiest" option.