111–120 of 181 posts
I was recently annoyed to find postgres indexes don't support skipping [1] you also can't have the nul character in a string (\u0000) [2]. Its great, but it has some strange WTF gaps in places. [1] https://wiki.postgresql.org/wiki/Loose_indexscan [2] https://stackoverflow.com/questions/28813409/are-null-bytes-...
Let me introduce you to blob…
You could, but it is going to force technical compromises. Even tools that can do everything aren't the best at everything.
I was recently annoyed to find postgres indexes don't support skipping [1] you also can't have the nul character in a string (\u0000) [2]. Its great, but it has some strange WTF gaps in places. [1] https://wiki.postgresql.org/wiki/Loose_indexscan [2] https://stackoverflow.com/questions/28813409/are-null-bytes-...
What is a reasonable use for a null character in a string? My first instinct is that strings with nulls in them should absolutely be rejected.
That said, I’m with you. And if someone wants nulls inside their “strings” then they probably want blobs.
I was recently annoyed to find postgres indexes don't support skipping [1] you also can't have the nul character in a string (\u0000) [2]. Its great, but it has some strange WTF gaps in places. [1] https://wiki.postgresql.org/wiki/Loose_indexscan [2] https://stackoverflow.com/questions/28813409/are-null-bytes-...
What is a reasonable use for a null character in a string? My first instinct is that strings with nulls in them should absolutely be rejected.
Earlier quoted context omitted.
I always wondered about the claim that SKIP LOCKED is all that efficient. Surely there are lots of cases where this is a really suboptimal pattern. Simple example: if you have a mixture of very short jobs and longer duration jobs, then there might be hundreds or thousands of short jobs executed for each longer job. In such a case the rows in the jobs table for the longer jobs will be skipped over hundreds of times. T…
Good point about SKIP LOCKED inefficiencies with mixed-duration jobs. In PGQueuers benchmarks, throughput reached up to 18k jobs/sec, showing it can handle high concurrency well. For mixed workloads, strategies like batching or partitioning by job type can help. While a separate "running" table reduces skips, it adds complexity. SKIP LOCKED strikes a good balance for simplicity and performance in many use cases. One…
Generally what you need to do there is have some column that can be sorted on that you can use as a high watermark. This is often an id (PK) that you either track in a central service or periodically recalculate. I've worked at places where this was a timestamp as well. Perhaps not as clean as an id but it allowed us to schedule when the item was executed. As a queue feature this is somewhat of an antipattern but did make it clean to implement exponential backoff within the framework itself.
Earlier quoted context omitted.
What is a reasonable use for a null character in a string? My first instinct is that strings with nulls in them should absolutely be rejected.
There are two kinds of programmers: Those who think of strings as text, and those who think of strings as a sequence of bytes. The second group doesn’t care about the special case where a byte is all zeroes.
Earlier quoted context omitted.
You don't have to plan this very early; most companies won't get to 100+ engineers. Let's ship first and worry about this, much much much much later. Overarchitecting stuff makes life hard; coming into companies that have 40 servers running with everything architected for 1000000 engineers and billions of visitors while in reality there aren't even 2 users and there is 1 overworked engineer. Stop doing that and stop…
> Let's ship first and worry about this, much much much much later. Overarchitecting stuff makes life hard; coming into companies that have 40 servers running with everything architected for 1000000 engineers and billions of visitors while in reality there aren't even 2 users and there is 1 overworked engineer. Even if you try to draw boundaries between different bits of the system, you are unlikely to end up with 40…
And you are lucky to not see an org where everything is a microservice that uses some unusual database, because the poeple responsible wanted to use some fancy new technology. Also seems you were lucky to not see messy development where some data is in a legacy system, some in new (which doesnt quite work), some in "cool" mongoDB that uses math.random to report just 10% of errors and rest is plugged via CSV files coming from ERP edited in Excel...
Just don't use a single Postgres DB for everything as you scale up to 100+ engineers. You'll inevitably get database-as-the-API. Now if you have the actual technical leadership [1] to scale your systems by drawing logical and physical boundaries so that each unit has its own Postgres? Yeah Postgres for everything is solid. [1] Surprisingly rare I've found. Lots of "successful" CTOs who don't do this hard part.
database as API works fine if you have properly abstracted things with sprocs and views. It will be also far less brittle than 100 services exposed as GraphQL
Earlier quoted context omitted.
There are two kinds of programmers: Those who think of strings as text, and those who think of strings as a sequence of bytes. The second group doesn’t care about the special case where a byte is all zeroes.
In that second case the string is better represented as "bytea", which has most (but not all) of the features of the "text" type.