81–90 of 181 posts
Postgres administration is such a PITA. I'd rather use sqlite for everything.
Taking graph databases, for example, Postgres (via Apache AGE) stores graph data in relational tables with O(log(n)) index lookups and O(k ⋅ log(n)) traversal*
Whereas a true graph database like Neo4J stores graph data in adjacency lists, which means O(1) index lookups and O() traversal.
That's a massive difference in traversal complexity for most graphs.
*k is the degree of the node (number of edges connected to a node).
[flagged]
PGQueuer is a lightweight job queue for Python, built entirely on PostgreSQL. It uses SKIP LOCKED for efficient and safe job processing, with a minimalist design that keeps things simple and performant. If you’re already using Postgres and want a Python-native way to manage background jobs without adding extra infrastructure, PGQueuer might be worth a look: GitHub - https://github.com/janbjorge/pgqueuer
Check out the Similar Projects section in the docs for a whole bunch of Postgres-backed task queues. Haven't heard of pgqueuer before, another one to add!
PGQueuer is a lightweight job queue for Python, built entirely on PostgreSQL. It uses SKIP LOCKED for efficient and safe job processing, with a minimalist design that keeps things simple and performant. If you’re already using Postgres and want a Python-native way to manage background jobs without adding extra infrastructure, PGQueuer might be worth a look: GitHub - https://github.com/janbjorge/pgqueuer
PGQueuer is a lightweight job queue for Python, built entirely on PostgreSQL. It uses SKIP LOCKED for efficient and safe job processing, with a minimalist design that keeps things simple and performant. If you’re already using Postgres and want a Python-native way to manage background jobs without adding extra infrastructure, PGQueuer might be worth a look: GitHub - https://github.com/janbjorge/pgqueuer
I always wondered about the claim that SKIP LOCKED is all that efficient. Surely there are lots of cases where this is a really suboptimal pattern. Simple example: if you have a mixture of very short jobs and longer duration jobs, then there might be hundreds or thousands of short jobs executed for each longer job. In such a case the rows in the jobs table for the longer jobs will be skipped over hundreds of times. T…
This way you won't need to skip over the long jobs that are in state "processing".
Great repo! Thanks! I'm wondering, any approach for bitemporal dbs, like xtdb?
You definitely don’t need a special database for bitemporal data. Just a datetime and as of data time column, your value column and whatever metadata you want (or a jsonb col for metadata if you want more flexibility at the cost of some speed of filtering by metadata)
Earlier quoted context omitted.
If you don't have any discipline it becomes hell. Not to mention that a random team writing a migration that locks a key shared table (or otherwise chokes resources) now causes outages for everyone .
Not a problem until it's a problem.
Overengineering is a plague amongst SWEs, and almost as dangerous as failing to sell the product in the market.