Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding UUIDs, ULIDs and string representations

sudhir.io

Post reply on HN
Print view View original

21–30 of 104 posts

Re: Understanding UUIDs, ULIDs and string representations

#21

Post by stevesimmons »

jdknezek wrote: post #9

There's also a proposal for UUIDv6-8, lexicographically sortable variants. https://datatracker.ietf.org/doc/html/draft-peabody-dispatch...

To summarise the differences:

* UUIDv6 - sortable, with a layout matching UUIDv1 for backward compatibility, except the time chunks have been reordered so the uuid sorts chronologically

* UUIDv7 - sortable, based on nanoseconds since the Unix epoch. Simpler layout than UUIDv6 and more flexibility about the number of bits allocated to the time part versus sequence and randomness. The nice aspect here is the uuids sort chronologically even when created by systems using different numbers of time bits.

* UUIDv8 - more flexibility for layout. Should only be used if UUIDv6/7 aren't suitable. Which of course makes them specific to that one application which knows how to encode/decode them.

UUIDv7 is thus the better choice in general.

(I recently wrote Python and C# implementations - https://github.com/stevesimmons/uuid7 and https://github.com/stevesimmons/uuid7-csharp)

Re: Understanding UUIDs, ULIDs and string representations

#22

Post by stevesimmons »

hn_go_brrrrr wrote: post #17
skunkworker wrote: post #12

I'm liking ULIDs more and more recently, as a UUIDv4 is random, insert performance is going to be subpar compared to bigserial. But going to a ULID which includes the time allows you slightly quicker insert performance. Also allowing for some tiered storage architectures, where if you know the ULID, you know where to look (approximately).

Depends on your storage system. For the one I work with most, a common prefix on your primary keys will hurt performance because it causes hotspotting. A UUID primary key would be the best case because it optimally shards writes. A ULID would be the worst case -- I would need to store it with the bits reversed.

And conversely, if storage is chunked (e.g. parquet files on S3), having time-ordered uuids may turn it into essentially an append-only log-structured store.

Here hotspotting is the aim, since it lets you efficiently prune query plans from index scans to direct reads of the right chunk.

Re: Understanding UUIDs, ULIDs and string representations

#24

Post by atoav »

gorgoiler wrote: post #20

All this stuff about collisions and avoiding them, even though they will never happen, feels like a PHB compliance issue. “Great work Geoff! One question: what’s the probability of two transactions having the same ID?” “It is very low” “Hmmm. But it’s not zero?” “It’s so low that it practically is zero.” “But it’s not technically zero? This company wasn’t built on taking chances, son! Come back when your product comp…

This is when any tech person worth their salt should lean over and say:

"Okay boss, we _could_ do that, but so you know what that would mean?"

And then you tell them about cosmic rays, bitflips and redundant computing and what that would mean for the cost of IT at your company.

"... or, we could just use UUIDs like nearly everybody else. I will spend a few days thinking about what would happen in case of a UUID collision and create a mechanism that adverts the worst consequences if you want. That should be enough in my judgement, we could also ask $collegue what if they agree with that conclusion"

On a side note: bosses who think they just need to be convincing enough in order to change physics are the worst. Some bosses expect NASA-level solutions for no or minor resource cost at all.

Re: Understanding UUIDs, ULIDs and string representations

#25

Post by elevader »

I really liked this article but I feel it misses one somewhat important point about using incremental numbers: They are trivially guessable and one needs to be very cautious when exposing them to the outside world.

If you encounter some URL like https://fancy.page/users/15 chances are that the 15 is a numeric ID and 1 to 14 also exist. And the lower numbers tend to be admin accounts as they are usually created first. This might be used by an attacker to extract data or maybe gain access to something internal. One could argue that using UUIDs only hides a security hole in this case but thats better than nothing I guess.

Re: Understanding UUIDs, ULIDs and string representations

#26

Post by 9dev »

elevader wrote: post #25

I really liked this article but I feel it misses one somewhat important point about using incremental numbers: They are trivially guessable and one needs to be very cautious when exposing them to the outside world. If you encounter some URL like https://fancy.page/users/15 chances are that the 15 is a numeric ID and 1 to 14 also exist. And the lower numbers tend to be admin accounts as they are usually created first.…

More trivially, it also gives insights into your business if I can determine the upper bound of your resources by trial-and-error guessing. If the highest user ID is 94, that may be an (hopefully unwarranted!) red flag to potential customers or investors.

Re: Understanding UUIDs, ULIDs and string representations

#27

Post by 9dev »

atoav wrote: post #24
gorgoiler wrote: post #20

All this stuff about collisions and avoiding them, even though they will never happen, feels like a PHB compliance issue. “Great work Geoff! One question: what’s the probability of two transactions having the same ID?” “It is very low” “Hmmm. But it’s not zero?” “It’s so low that it practically is zero.” “But it’s not technically zero? This company wasn’t built on taking chances, son! Come back when your product comp…

This is when any tech person worth their salt should lean over and say: "Okay boss, we _could_ do that, but so you know what that would mean?" And then you tell them about cosmic rays, bitflips and redundant computing and what that would mean for the cost of IT at your company. "... or, we could just use UUIDs like nearly everybody else. I will spend a few days thinking about what would happen in case of a UUID colli…

Or be helpful and do the rounding for your boss instead. The difference between „zero“ and „practically zero“ is only interesting to academia in this case.

Re: Understanding UUIDs, ULIDs and string representations

#28

Post by dlbucci »

sudhirj wrote: post #2

Author here, self-posted. AMA.

Great article! I hadn't heard of this, but it sounds like something I might suddenly be using in the near future! Also, I loved this line: "Given the way we're going, humanity in its present form isn't likely to exist them, so when this becomes an issue it'll be somebody else's problem. More likely something else's problem."

Re: Understanding UUIDs, ULIDs and string representations

#29

Post by JimDabell »

voidfunc wrote: post #10

Most people are familiar with time and random UUIDs but and I had sort of known about v5 UUIDs but recently used them to get consistent identifiers from an input value… super useful because you can do `uuidv5($namespace_uuid, data)` and get the same UUID everytime.

Isn’t that just a hash?

Re: Understanding UUIDs, ULIDs and string representations

#30

Post by drewcoo »

elevader wrote: post #25

I really liked this article but I feel it misses one somewhat important point about using incremental numbers: They are trivially guessable and one needs to be very cautious when exposing them to the outside world. If you encounter some URL like https://fancy.page/users/15 chances are that the 15 is a numeric ID and 1 to 14 also exist. And the lower numbers tend to be admin accounts as they are usually created first.…

This is the problem with incremental numbers:

https://en.wikipedia.org/wiki/German_tank_problem

Post reply on HN