21–30 of 46 posts
[1] https://developers.cloudflare.com/workers/tutorials/query-po...
Earlier quoted context omitted.
HTTP does not (easily) allow for using answers of earlier queries in the transaction. E.g. BEGIN; INSERT INTO my_table (...) RETURNING (id); SELECT count(\*) AS my_count FROM my_table; INSERT INTO historical (new_id, value_derived_from_count, now()); COMMIT; is difficult (or potentially impossible) to do transactionally using single HTTP queries. Sure, you can rewrite your queries to use single-statement queries if y…
That I understand. But of course for the user of the driver it might be fine if that is 2 or more HTTP requests. I expect that is how PlanetScale does it in their transaction implementation, https://github.com/planetscale/database-js#transactions , and I know that is how Prisma Data Proxy handles it - the transaction is identified with an ID which is returned to the Client and then included in further requests for th…
I’m Neon CEO. Happy to answer questions. We also have an interesting roadmap for the driver where we hope to keep driving latency and number of round trips down
https://community.fly.io/t/preview-managed-upstash-redis-wit...
They don't have a managed postgres offering yet ("this is not a managed postgres" - https://fly.io/docs/postgres/getting-started/what-you-should...)
Earlier quoted context omitted.
Right: at a basic level, using WebSockets lets us change as little as possible from the user perspective. You get an ordinary Postgres session with an ordinary Postgres driver, full control over transactions, and so on. At this point, your serverless function establishes a new Postgres connection on each call. We do pooling on the server side with pgBouncer, which means we can handle lots of simultaneous connections…
Ok, so to make that explicit: If I want to do 5 parallel queries on my serverless function I should still have a connection pool size of 5 in my application, which will be fine as PgBouncer ensures there are plenty connections to open and use from the database server side. Correct?
Earlier quoted context omitted.
HTTP does not (easily) allow for using answers of earlier queries in the transaction. E.g. BEGIN; INSERT INTO my_table (...) RETURNING (id); SELECT count(\*) AS my_count FROM my_table; INSERT INTO historical (new_id, value_derived_from_count, now()); COMMIT; is difficult (or potentially impossible) to do transactionally using single HTTP queries. Sure, you can rewrite your queries to use single-statement queries if y…
That I understand. But of course for the user of the driver it might be fine if that is 2 or more HTTP requests. I expect that is how PlanetScale does it in their transaction implementation, https://github.com/planetscale/database-js#transactions , and I know that is how Prisma Data Proxy handles it - the transaction is identified with an ID which is returned to the Client and then included in further requests for th…
Yes, and there's a catch there that people might not notice: HTTP-based queries utilize this ID to identify which active transaction to use, but this is vulnerable to concurrent HTTP requests on the same transaction ID, thus allowing query injection (early COMMIT, SELECT sleep(1000000), etc. by attackers) due to requests being re-routed every time you send the query.
Keeping a direct connection (albeit proxied) to the PostgreSQL instance prevents this kind of attack, while also allowing for better state keeping control in systems that don't have access to raw TCP sockets, and reducing per-query overheads.
I’m Neon CEO. Happy to answer questions. We also have an interesting roadmap for the driver where we hope to keep driving latency and number of round trips down
Hmm. Wouldn't this also be useful for browser-hosted application code that wants to talk to PG directly?
The "How it works" section surprised me - I did not expect to see Websockets there. Other serverless drivers or database APIs (PlanetScale Serverless Driver, AWS RDS Proxy or Prisma Data Proxy) usually use HTTP to replace the stateful TCP connection. The post explains a bit why this might be beneficial (can use existing tools, and use connection state) - but what about the other benefits and use cases of going with H…
Earlier quoted context omitted.
Ok, so to make that explicit: If I want to do 5 parallel queries on my serverless function I should still have a connection pool size of 5 in my application, which will be fine as PgBouncer ensures there are plenty connections to open and use from the database server side. Correct?
In principle, yes, but as things stand you'll be starting 5 separate TLS connections to the server that way. My feeling is that this would be an unusual way to use serverless functions. Is it something you think you'd do?
What would be the alternative? Only execute the queries in sequence, one by one?