Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

pg_durable: Microsoft open sources in-database durable execution

github.com

Post reply on HN
Print view View original

81–90 of 119 posts

Re: pg_durable: Microsoft open sources in-database durable execution

#81

Post by oofbey »

moron4hire wrote: post #40
TuringNYC wrote: post #22

I'm trapped on Azure at work and we're constantly waiting for Azure pg to catch up with modernity. For example, you cant use this: https://www.paradedb.com/blog/hybrid-search-in-postgresql-th... Also for example, you dont get ultra-wide high dimensionality vectors. It is nice they are open sourcing pg_durable, but how about adopting table stakes I'd get with AWS?

I'm sorry, I'm sure you've considered this, but why couldn't you create a bare VM with Postgres vCurrent installed?

You could. But then you’re also building from scratch HA failover, backups, replica management, monitoring, etc - cloud vendor managed RDBMS come with lots of niceties. All of which are possible to set yourself. But a hassle, and difficult to make bullet proof.

Re: pg_durable: Microsoft open sources in-database durable execution

#82

Post by gitscope_ai »

levkk wrote: post #2

2026 is the year of the Postgres queue! (DBOS[0], pgQue[1]) It's awesome that the community is contributing this and giving us the option to use it. As an ex-app engineer though, I kind of prefer my queue logic to be in code, in Git, but maybe with the right tooling, you can change my mind. :) [0]: https://www.dbos.dev/ [1]: https://github.com/NikolayS/pgque

[flagged]

Re: pg_durable: Microsoft open sources in-database durable execution

#83

Post by oofbey »

guhidalg wrote: post #43
gdecandia wrote: post #14

Earlier quoted context omitted.

Contributor here. At Microsoft, our Postgres customers seem to split pretty evenly into 2 camps, those that want to do as much as they can in the database, and those that agree with your take - want to keep apps and compute outside the DB.

I bet this is correlated with how much they like/know Postgres already. When people don’t understand their database’s features, they want it to behave like something else they do understand (code). They’re leaving a lot of performance on the table by not leveraging everything their database can do.

Understanding and deciding to rely upon are not the same. Complexity is one of the biggest challenges in real world software systems. Keeping a subsystem’s responsibilities simple can do a lot for a system’s reliability. Most teams have extremely mature systems for managing complexity in their code - tests, CICD etc. Sure you CAN build all those things for your database too, but it’s more work. Most teams I’ve worked on choose to minimize database migrations because it’s a lot of work to make that part of the system as robust against mistakes as code is. Choosing to ignore a feature in your database is often a very rational pragmatic choice aimed at keeping complexity under control.

Re: pg_durable: Microsoft open sources in-database durable execution

#84

Post by nextaccountic »

levkk wrote: post #2

2026 is the year of the Postgres queue! (DBOS[0], pgQue[1]) It's awesome that the community is contributing this and giving us the option to use it. As an ex-app engineer though, I kind of prefer my queue logic to be in code, in Git, but maybe with the right tooling, you can change my mind. :) [0]: https://www.dbos.dev/ [1]: https://github.com/NikolayS/pgque

I like pgmq https://github.com/pgmq/pgmq

Re: pg_durable: Microsoft open sources in-database durable execution

#86

Post by otterley »

dpark wrote: post #77
junto wrote: post #76

This smells like stored procedures. You can’t unit test it. You can’t version it. Business logic in the database, (hidden brain problem), harder to isolate noisy workloads, no observability, scaling pressure lands solely in Postgres, lack of IO, especially API calls. Good for local database only jobs though. Niche use cases.

> This smells like stored procedures. You can’t unit test it. You can’t version it Say what? Stored procedures are awesome when used correctly. Versioning is straightforward. You stick any sort of monotonically increasing id at the end of the name. Whenever you need a breaking change, you bump the id. You also leave the old version with the old id, retiring it only after it’s no longer used. You do need a real story…

Why would using a stored procedure reduce I/O? I can see it reducing network round trips, but not storage reads and writes.

Re: pg_durable: Microsoft open sources in-database durable execution

#87

Post by eddythompson80 »

FuriouslyAdrift wrote: post #44
TuringNYC wrote: post #22

I'm trapped on Azure at work and we're constantly waiting for Azure pg to catch up with modernity. For example, you cant use this: https://www.paradedb.com/blog/hybrid-search-in-postgresql-th... Also for example, you dont get ultra-wide high dimensionality vectors. It is nice they are open sourcing pg_durable, but how about adopting table stakes I'd get with AWS?

Wouldn't Azure Cosmos DB be better suited for vector searches?

Never ever use Azure Cosmos DB. The entire point is to lock you in. This isn’t some paranoid shit either. We use azure a lot, and I have worked with many people designing systems on Azure. Always avoid cloud providers lock in services. That’s their bread and butter. They want you to use them. They want you using Azure Cosmos DB, Azure Event Hubs, Azure Apps, Azure DataLake, etc. Same with AWS. Don’t be naive. Use Azure VMs, Azure Postgres, Azure Redis. Those are fine. You’re just paying someone for the operational cost of a service, but you can migrate of. There is no migration from Cosmos or DataLake. They tell you you can abstract your code, but that never works. They know you will be locked in. That’s the entire business model. Also resist the temptation of the offers they’ll through at you to link those services with all their other crap. Don’t be naive.

Re: pg_durable: Microsoft open sources in-database durable execution

#88

Post by pjmlp »

junto wrote: post #76

This smells like stored procedures. You can’t unit test it. You can’t version it. Business logic in the database, (hidden brain problem), harder to isolate noisy workloads, no observability, scaling pressure lands solely in Postgres, lack of IO, especially API calls. Good for local database only jobs though. Niche use cases.

Good databases like Oracle and SQL Server have great IDEs for stored procedures development.

You can certainly unit test them, good databases have telemetry and metrics.

Version control is no different from using containers instead of VMs.

Any database change goes through CI/CD pipelines and regular devs cannot edit code directly on the DB.

In fact the biggest issue with databases is like debugging, some devs rather not learn how to use them properly.

In one they never go beyond printf, in the other, they only know what an ORM looks like, and the command line applications for basic SELECTs.

Re: pg_durable: Microsoft open sources in-database durable execution

#89

Post by dpark »

otterley wrote: post #86
dpark wrote: post #77

Earlier quoted context omitted.

> This smells like stored procedures. You can’t unit test it. You can’t version it Say what? Stored procedures are awesome when used correctly. Versioning is straightforward. You stick any sort of monotonically increasing id at the end of the name. Whenever you need a breaking change, you bump the id. You also leave the old version with the old id, retiring it only after it’s no longer used. You do need a real story…

Why would using a stored procedure reduce I/O? I can see it reducing network round trips, but not storage reads and writes.

I was referring to network I/O. But disk I/O should be at least as good for a stored procedure and often better. It’s classic “bring the computation to the data”. Putting the computation into the DB means that use of disk caching (up to and including CPU caching) is maximized.

Pulling the data out of the DB to do computation in a higher layer cannot be more efficient in terms of any I/O unless your stored procedure is just poorly written. It might be a win if your DB is compute bound, though.

Post reply on HN