Live data from Hacker News

Choosing a Postgres primary key

supabase.com

31–40 of 163 posts

Re: Choosing a Postgres primary key

#31
Disclaimer: not a dba so my terms might not be appropriate

I’ve seen uuid4 which replaces the first 4 bytes with a timestamp. It was mentioned to me that this strategy allows postgres to write at the end of the index instead of arbitrarily on disk. I also presume it means it has some decent sorting.

[inspiration](https://github.com/tvondra/sequential-uuids/blob/master/sequ...)

Re: Choosing a Postgres primary key

#32
Hybrid Logical Clock [1] could be of interest for readers. This is a monotonically increasing clock based on a physical clock.

Combined with a machine identifier you can obtain globally unique identifiers that are totally ordered.

[1] https://cse.buffalo.edu/~demirbas/publications/hlc.pdf

Re: Choosing a Postgres primary key

#33

Disclaimer: not a dba so my terms might not be appropriate I’ve seen uuid4 which replaces the first 4 bytes with a timestamp. It was mentioned to me that this strategy allows postgres to write at the end of the index instead of arbitrarily on disk. I also presume it means it has some decent sorting. [inspiration]( https://github.com/tvondra/sequential-uuids/blob/master/sequ... )

I use ULID, 128 bits, time and great sorting

https://github.com/ulid/spec

Re: Choosing a Postgres primary key

#34
post #6

My opinion. Always if in any way possible pick a semantic key. There is usually something defining the thing you are working on. If there isnt work on your normalisation. Main benefits to this: Avoids accidental duplication (happens so much). Avoids additional round trips to fetch the id to make a mutation. Of course if you work on something where you don’t know what it is yet (actually humans are a good example for…

For relational purists, e.g. Joe Celko, this approach is pretty much mandatory. However as others have stated in reply it's very difficult to find a genuinely immutable identifying attribute in many domains, including human beings.

Re: Choosing a Postgres primary key

#35

Disclaimer: not a dba so my terms might not be appropriate I’ve seen uuid4 which replaces the first 4 bytes with a timestamp. It was mentioned to me that this strategy allows postgres to write at the end of the index instead of arbitrarily on disk. I also presume it means it has some decent sorting. [inspiration]( https://github.com/tvondra/sequential-uuids/blob/master/sequ... )

It also has the advantage that the page being written to, the right most leaf at the end of the index, is likely to always be available in the page cache. With random you may need to constantly go to disk to fetch the page.

Re: Choosing a Postgres primary key

#36
post #9

While this is a good overview of the options for primary key generation, there's no silver bullet here. Most projects that are using SQL should just use the gold standard: an auto-incrementing integer for an internal primary key. And then decouple the public-facing primary key from it into a separate column, whether it be ULID, UUID, or a random-project-slug-123. Also, during debugging, it's a lot nicer to look at sh…

Why would you use an integer primary key and a public facing UUID? That seems like it's the worst of both worlds: ugly externally visible identifiers, record bloat, a database that you can't easily merge in the event of backups or DR, and having to roundtrip to the DB before you know the ID of a record. I personally stick to UUIDs in pretty much all cases, with the exception of where there are justified and benchmark…

You're right to point at performance as the main motivator for this setup.

The primary key is included in all indexes, including non-clustered indexes, so in some cases there can be quite a large difference between UUID and integer PKs in terms of index size.

UUID PKs are also more susceptible to fragmentation.

Re: Choosing a Postgres primary key

#38
post #30

Earlier quoted context omitted.

But if you’re handing out a random ID, you still need to have an index over that ID to have efficient lookups of random ID -> internal ID right? One advantage is that you only need to resolve it once at the edge and then internally you use the external facing value. Are there any others I’m missing? I agree, I would have loved a deeper dive in xid since it seemed to clearly outperform everyone else.

Not necessarily. The idea here is that the id can be exposed publically because it is random. The problem it solves is someone sees a page /accountdetails?id=123 and can easily look for /accountdetails?id=124 and assume it is likely to be valid. If you use a random id, you cannot quickly know what other ids exist which makes looking for unauthorized access to objectids much harder.

This of course assumes that all the records backed by these IDs are not public information. In a lot of cases, this may not be true.

Re: Choosing a Postgres primary key

#39
post #23

Honestly that's a poor blog post. Randomly concludes "the best time-based ID seems to be xid" without saying why or comparing to others e.g. ksuid, UUIDv7 etc ("xid" is only mentioned twice in the entire blog, first in the above statement and second a link to the reference implementation). Equally unfortunate that they picked "xid" as their supposed "best" because Postgres has an internal identifier that is also call…

You really care about exposing a serial number scheme for a list of books your company publishes, or the identifier for each of the various hotels you own, or the cities you have an office in or something?

Probably not, but you definitely should care about not issuing sequential credit card numbers. Probably not the best example, but I don't think it's hard to imagine some scenario in between the two that still presents a concern.

Re: Choosing a Postgres primary key

#40
post #30

Earlier quoted context omitted.

But if you’re handing out a random ID, you still need to have an index over that ID to have efficient lookups of random ID -> internal ID right? One advantage is that you only need to resolve it once at the edge and then internally you use the external facing value. Are there any others I’m missing? I agree, I would have loved a deeper dive in xid since it seemed to clearly outperform everyone else.

Not necessarily. The idea here is that the id can be exposed publically because it is random. The problem it solves is someone sees a page /accountdetails?id=123 and can easily look for /accountdetails?id=124 and assume it is likely to be valid. If you use a random id, you cannot quickly know what other ids exist which makes looking for unauthorized access to objectids much harder.

If there needs to be an auth check for other IDs, then it shouldn't matter whether the IDs are random or not.

Depending on how highly aesthetic URLs are valued, it's not unlikely that after being in business for a while, the density of your keyspace will mean that even random IDs are found.

A better rationale for disconnecting public and private IDs is to make certain types of database migration a little bit easier. I don't think it's a huge win though, it's a chunk of work to maintain multiple IDs, do the required inderictions all the time, ensure public IDs are always sent to front end, and so on, while adding a translation layer as part of a database migration delays this kind of work until it's actually needed, if ever.

Post reply on HN