Live data from Hacker News

Unexpected downsides of UUID keys in PostgreSQL

cybertec-postgresql.com

141–150 of 215 posts

Re: Unexpected downsides of UUID keys in PostgreSQL

#141

Earlier quoted context omitted.

It is not a matter of a couple milliseconds. The loss of locality for reads is bad, especially for data sets that don't fit into cache / RAM (while the active set would). Where it really bites you is writes, because it can trigger pretty massive write amplification. Imagine you have 128 GB index on UUID column, that's ~16M pages (8kB) and insert 1M random values. Congrats! You've probably just wrote 8GB to the WAL, b…

Out of my scope, but why are UUIDs even discussed? ULIDs ~~(and i think Nanoids?)~~ don't suffer these same problems. Locality and ordering alone make me[1] think ordered ULIDs (and friends) are the only thing worth discussing. Is there some value to UUIDs over ULIDs that make these discussions largely revolve around Autoincrement vs UUIDs rather than Autoincrement vs ULIDs(and friends)? [1]: Again, totally out of my…

ULIDs have made our MySql caches breathe a sigh of relief.

One place we're avoiding ULIDs (and other counters) is in publicly-facing IDs. Preferring random to help keep them unguessable. (say what you will about security-through-obscurity).

So we do ULIDs for private IDs. Random UUIDs for public IDs. Seems to work well.

Re: Unexpected downsides of UUID keys in PostgreSQL

#142

I can't think of many use cases where I would sacrifice the beauty and elegance of UUIDs to optimize access times by a millisecond or two. UUID is totally worth the cost. UUID actually performs much better than I thought based on the author's example with a COUNT query. COUNT queries aren't very efficient because typically, all records are traversed; here we're talking about a 50% slowdown on 10 million records... Ho…

It is not a matter of a couple milliseconds. The loss of locality for reads is bad, especially for data sets that don't fit into cache / RAM (while the active set would). Where it really bites you is writes, because it can trigger pretty massive write amplification. Imagine you have 128 GB index on UUID column, that's ~16M pages (8kB) and insert 1M random values. Congrats! You've probably just wrote 8GB to the WAL, b…

Mmmhhh should that not be highly efficient for flash based storage?

Re: Unexpected downsides of UUID keys in PostgreSQL

#144

Earlier quoted context omitted.

Out of my scope, but why are UUIDs even discussed? ULIDs ~~(and i think Nanoids?)~~ don't suffer these same problems. Locality and ordering alone make me[1] think ordered ULIDs (and friends) are the only thing worth discussing. Is there some value to UUIDs over ULIDs that make these discussions largely revolve around Autoincrement vs UUIDs rather than Autoincrement vs ULIDs(and friends)? [1]: Again, totally out of my…

ULIDs have made our MySql caches breathe a sigh of relief. One place we're avoiding ULIDs (and other counters) is in publicly-facing IDs. Preferring random to help keep them unguessable. (say what you will about security-through-obscurity). So we do ULIDs for private IDs. Random UUIDs for public IDs. Seems to work well.

I wonder if just encrypting them for public usage would be enough here. Then application can convert between public and private representation at will.

Re: Unexpected downsides of UUID keys in PostgreSQL

#145
post #118

Earlier quoted context omitted.

I agree that UUIDv7 will usually be the best choice. The one downside I can think of is that it may sometimes be necessary to let people know the identity of a thing without also telling them exactly when that thing was created.

Encrypt it. Block ciphers are bijective so they will turn sequential IDs into unique random IDs in a way you can easily reverse. You can also do that to get opaque identifiers from an auto-increment primary key.

Seems awfully complicated to me.

Re: Unexpected downsides of UUID keys in PostgreSQL

#146

Earlier quoted context omitted.

Yes exactly, with proper sharding, raw performance is not as important; to some extent, you trade it away for improved concurrency. In fact, I struggle to see how one would implement sharding with auto-incrementing integers (you would get ID collisions for different resources across different shards/database instances); there needs to be a way to uniquely refer to resources across potentially multiple databases and U…

We did auto-increment integers with multiple servers like 20 years ago: The caveat is that you have to know how many servers are in the set in advance. Each server increments by the population size, and their starting number is their position within the pool. Not hyper scale, but good enough for failover or a 3-5 node setup.

IIRC that's what MySQL Galera is doing by default

    mysql> CREATE TABLE animals (
        ->      id MEDIUMINT NOT NULL AUTO_INCREMENT,
        ->      name CHAR(30) NOT NULL,
        ->      PRIMARY KEY (id)
        -> );
    Query OK, 0 rows affected (0.34 sec)
    
    mysql> INSERT INTO animals (name) VALUES
        ->     ('dog'),('cat'),('penguin'),
        ->     ('lax'),('whale'),('ostrich');
    Query OK, 6 rows affected (0.01 sec)
    Records: 6  Duplicates: 0  Warnings: 0
    
    mysql> SELECT * FROM animals;
    +----+---------+
    | id | name    |
    +----+---------+
    |  3 | dog     |
    |  6 | cat     |
    |  9 | penguin |
    | 12 | lax     |
    | 15 | whale   |
    | 18 | ostrich |
    +----+---------+
    6 rows in set (0.00 sec)

Re: Unexpected downsides of UUID keys in PostgreSQL

#147

Earlier quoted context omitted.

I personally find the numeric id extremely valuable for internal data analysis and sharing. I can refer to rows by the numeric id, including a range of rows, and seeing the ids gives exactly that intuitive information about it's relation in the set that we are hiding from end users. Numeric ids can also be used for the same reason in an admin-only UI. On the efficiency side, joining and querying by id is generally mo…

This was generally the reason I went with numeric ID as PK originally. It makes working with and analyzing the data as well as cross referencing relations easier. For all my tables I have a base schema that looks something like this. id: integer sequence PK uuid: uuidV4 created_at: datetime updated_at: datetime The concern I have is when I have to distribute my system when scaling. Those numeric IDs will have to be r…

Everything breaks at scale. In my experience most tables don't end up with more than a few million rows and will work fine with this. If you did want to transition a large table to be UUID only, the nice thing about this approach is that you could do it with no down time. If you are using a DB that only scales writes vertically though (most DBs, including distributed DBs) then how are you actually going to scale the DB layer horizontally? Pretty much just CRDB (PG) or TiDB (MySQL) are the options there- look at their docs for how to setup your ids.

Re: Unexpected downsides of UUID keys in PostgreSQL

#148
post #71

For me, the visual noise is a downside for UUIDs. A lot of time investigating data issues means glancing at query results and deciding if something looks unexpected. I just can't parse a UUID with my eyes that quick. I've come up against this at my current job a little too often and I'm cursing the decision to switch to UUIDs. I know, it's a balancing act of competing concerns. But for us moving to UUIDs was future p…

[deleted]

Re: Unexpected downsides of UUID keys in PostgreSQL

#149
post #137

Earlier quoted context omitted.

Out of my scope, but why are UUIDs even discussed? ULIDs ~~(and i think Nanoids?)~~ don't suffer these same problems. Locality and ordering alone make me[1] think ordered ULIDs (and friends) are the only thing worth discussing. Is there some value to UUIDs over ULIDs that make these discussions largely revolve around Autoincrement vs UUIDs rather than Autoincrement vs ULIDs(and friends)? [1]: Again, totally out of my…

I’ve worked with ULIDs a bit but honestly haven’t operated at a scale where you might run into issues. And the main reason for choosing ULID was because I could only find experimental support for UUIDv6 or UUIDv7, and KSUID (another alternative) only had time precision down to a second. And the reason for that was to have lexicographically sortable IDs (even if not monotonic, which would require an extra server) so w…

Out of curiosity, why do your keys need time precision less than a second? Are they carrying some secondary expectations beyond uniqueness and k-sortable?

Re: Unexpected downsides of UUID keys in PostgreSQL

#150
post #71

For me, the visual noise is a downside for UUIDs. A lot of time investigating data issues means glancing at query results and deciding if something looks unexpected. I just can't parse a UUID with my eyes that quick. I've come up against this at my current job a little too often and I'm cursing the decision to switch to UUIDs. I know, it's a balancing act of competing concerns. But for us moving to UUIDs was future p…

Interesting take. I do agree there's some visual noise there, but not how you described it. UUID just takes so much screen space compared to its integer counterpart. If you don't have a good UI sorting things out it's quite annoying. Otherwise it's pretty great.

It's kind of a tooling problem; no reason tooling can't do the same as git and display "1509af9" instead of "1509af9af2634d16f8d9b98e01a0166a49185474".

Also I wish these sort of things would get encoded in base-36 (0-9 a-z) instead of base-16; that would help too.

Post reply on HN