Live data from Hacker News

Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

github.com

81–87 of 87 posts

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#81
post #52

Very interesting project! The source is inspiring, it seems the author has made an explicit choice to minimize dependencies. Even transpilation from ES modules into commonjs is done by the author himself: https://github.com/porsager/postgres/blob/master/transpile.c... I’m not sure if I’d made the same choice, but it’s fun the see it can work out great :)

Manipulating source code via regex... I wonder what could possibly go wrong.

With NPM being such a security nightmare everytime you rope in another dependency, and myself being a long-time user of postgres.js (and much of porsagers' other work in the Node community), I do think this was the right decision.

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#82
post #78

Earlier quoted context omitted.

> The performance gains would have to either come from a faster (JIT'd) protocol parser, or from eliminating additional FFI calls (that it isn't obvious why they'd exist). or, as I noted above, and will try to simplify here: let's suppose you have a buffer which contains 5 key/value pairs, and you want to convert that into an array of 5 javascript objects in v8. 1. obtain a pointer to the isolate 2. create an array o…

Hey Jerry! I'll respond here so we don't have split threads :). I'm also quite familiar with at least the older versions of the node internals (I too maintained a popular db binding for a number of years) and I'm very confused by the way you're positioning the operation of the v8 vm in these 2 scenarios. Sure, the c++ is going to require you to do some sanitizing as you force your data into v8, but as we noted that's…

> Sure, the c++ is going to require you to do some sanitizing as you force your data into v8

it's not just sanitizing, there's a lot more to the object creation inside v8 itself. but, even if it were just sanitizing, that mechanism has become a lot more complicated than it ever was in v8 3.1 (timeframe around node 0.4) or 3.6 (timeframe around node 0.6). when interacting with c++, v8 makes no assumptions, whereas when interacting with javascript, a large number of assumptions can be made (e.g. which context and isolate is it being executed in, etc).

> but as we noted that's inevitable no matter how you slice it.

yes, from c++ to javascript and back, but when you need to make that trip multiple times, instead of once, that interchange adds up to quite a bit of extra code executed, values transformed, values checked, etc. sure, banging your head against a wall might not hurt once, but do it 40 times in a row and you're bound to be bloodied.

> Now maybe in some cases the v8 internals offer some advantages the generic c++ api can't access

by a fairly large margin, as it turns out, especially as v8 has evolved from the early 3.1 days to the current 9.8: 11 years. there has been significant speedup to javascript dealing with javascript objects compared to c++ dealing with javascript objects. see below.

> My memories of the redis client is different than yours so I'd be quite interested to see those conversations / benchmarks.

super easy to find, all of that was done in public: https://github.com/redis/node-redis/pull/242 - there are multiple benchmarks done by multiple people, and the initial findings were 15-20% speedup, but were improved upon. the speedup was from the decoding of the binary packet, which was passed as a single buffer, as opposed to parsing it externally and passing in each object through the membrane.

> As a simple thought experiment, in the scenario you're describing we should see a javascript implementation of a JSON parser to beat the pants off the v8 engine implementation, but this doesn't seem to the case.

that's a bit of a straw man argument. especially given that JSON.parse() is a single call and does not require any additional tooling/isolates/contexts to execute, it's just straight c++ code with very fast access into the v8 core:

    Local result = Local::New(isolate, JSON.Parse(jsonString));
but, let's take your straw man a little further. let's suppose that all of the actual parsing is done for you already, and all you're doing is iterating through the data structure, creating objects through the c++ api, and calling it good. that should be faster than calling the c++ JSON.parse(), shouldn't it? since we don't have to actually parse anything, right? no, it's actually much slower. you can see this in action at https://github.com/plv8/plv8/blob/r3.1/plv8_type.cc#L173-L60...

again, we're not talking about whether javascript in an interpreter is faster than c++, we're talking about whether v8's api causes enough slowdown that some workloads that require a lot of data between c++ and javascript are slower than the same workload that requires very little data between c++ and javascript ... because passing through v8's c++/javascript membrane is slow.

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#83
post #51

Earlier quoted context omitted.

This can’t be overstated. Crossing the JS native barrier is inherently a perf hit. I’ve measured it trying a large variety of high performance messaging protocols and postMessage with structured clone nearly always wins. If that sounds like a bold claim, there are troves of Node and Deno issues where they’ve improved perf by staying in JS specifically because calling into native and back has been the bottleneck. Ther…

I responded to the OP, but I'll add here too for the conversation. Where are these projects that are implementing network protocols, where the nodejs versions are faster than the native ones? As I noted in my other comment, in -any- of those implementations, you're still going to be required to traverse from libc somewhere to nodejs (even if it's just to read the network data out of the socket and send it to v8). The…

I can’t link to any projects, implementing network protocols per se isn’t what I focused on researching this. My focus was on optimizing postMessage between threads. They’re not that different but the important distinction is that the user-facing API is in JS, not a network boundary.

My hypothesis was that:

1. Converting to binary data

2. Using facilities for shared memory

Would yield better performance in a language well suited to de/serialize binary data, with a message encoding designed for performance. The latter does perform better! If you need to spend a lot of CPU time wrangling binary data, it’s a clear win. If you have a workload with many small messages, they invariably slow down compared to structuredClone. If your workload can avoid crossing the boundary that’s a clear win still. But the moment JS VM values need to get shuttled around, structuredClone is specifically optimized for structuredClone-able values, and optimizing the JS/native boundary for data which benefits from that is an extremely narrow edge case. The only way I’ve found to win is not to play.

That said I don’t have the cleverness a lot of performance geeks have and maybe there’s some technique I’ve missed! But I honestly can’t imagine how I’d optimize JS/native interop better than the JIT without finding myself getting surprising new career opportunities on a VM team.

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#84

Earlier quoted context omitted.

> Any plans for Knex support? That question doesn't really make sense. I'm not familiar with this library, but the developer interface is very similar to slonik. The whole point of this is that SQL tagged template literals replace the need for a query builder like Knex. Here is a good blog post from the author of slonik explaining: https://gajus.medium.com/stop-using-knex-js-and-earn-30-bf41...

I'm broadly a fan of just writing SQL, but Knex does have one important advantage: it automatically declares the return types from a query based on my DB structure. If I write SQL by hand, I have to annotate the query with the return types, and those annotations can get out of sync with the query.

Postgres.js does the same, transparently. No advantages of Knex here.

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#85
post #62

Earlier quoted context omitted.

Some people really love the simplicity that comes from zero compilation. I like that too, so I always start with just JS, but then two files in realize I really want my typings. Now I use esbuild instead of tsc, and I have the best of both worlds.

> Now I use esbuild instead of tsc, and I have the best of both worlds. I'm interested in this. I know esbuild can compile TypeScript to JS, but that it doesn't serve as an actual typechecker. Without tsc as a dev dependency, do you just rely on your IDE's intellisense to tell you when there's a type error?

I still install tsc, but I don’t actually do the type checking except at release time (and whatever typechecking the IDE provides through the language server).

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#86

Excellent work as usual, Rasmus! :) Loved my time with postgres.js in past projects, looking forward to checking out v3 in an upcoming project of mine. The speed and lack of any dependencies, while being so full-featured is really attractive.

Hey keb_ :) Thanks a lot!

Re: Show HN: Postgres.js – Fastest Full-Featured PostgreSQL Client for Node and Deno

#87
post #77

I spent some time today trying to replace pg but I ran into an issue. When using an rds proxy with iam authentication, it seems to repeatedly retry authentication and eventually my lambda functions time out. If I switch to using regular credentials it works fine. Using the exact same options with pg+iam authentication also works fine which leads me to believe it's an issue with this project. I'll open an issue on Git…

Be very interesting to know if it is the rds proxy that is causing the problem. It is one things that annoys me about rds and aws services.

It was indeed the RDS Proxy. It appers to be very strict about the client_encoding parameter. Postgres.js was sending 'utf-8' which PostreSQL will understand, but RDS Proxy would just hang until this was changed to UTF8 (uppercase with no dash).

https://github.com/porsager/postgres/issues/288

Post reply on HN