I have always been a bit hesitant to use UUIDs with timestamps as it can be a security issue if the IDs are public. For example getting the age of a user account just from the id. I will say, however, that I have not heard of any major incidents stemming from this.
ULID: Universally Unique Lexicographically Sortable Identifier
21–30 of 58 posts
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#22I love the aesthetics. The cryptographic strength tradeoffs (against UUIDv7) seem rough for a lot of applications, though.
Not sure what you mean by cryptographic strength - they are both Unique ID generators, not meant for anything related to cryptography. UUIDv7 has 62 bits of random data, ULID uses 80 bits, so if anything ULID is "stronger" (meaning less chances of generating the same id within the same millisecond)
UUIDv7 has 74 bits of randomness, not 62, you forgot rand_a portion, so the difference is just 6 bits and only matters within the same millisecond.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#23Earlier quoted context omitted.
You're right, not hosts or processes in that case. I forgot about random part as it's been a while since I looked at it. However, a single instance of a ULID generator must support this mode, which means that on multi-threaded architectures, it must lock the sequence as it still uses a single random value. That again, kills the purpose of a client-side, lock-free generation of universal identifiers as you said.
You only need to lock sequence if you care about IDs being ordered within a millisecond. That generally only matters when you create a batch of IDs at once, in that case you don't need to lock anything: generate ULID, keep incrementing sequence in that batch either by doing on the same thread, or by moving it from thread to thread. Kinda like creating an iterator and zip'ing it with iterator of thing you need IDs for…
I hope that's not literally incrementing a sequence. Because it would lead to trivial neighbor ID guessing attacks.
I've implemented this thing, though not called it ULID. I've dedicated some bits for timestamp, some bits for counter within millisecond and rest for randomness. So they always ordered and always unpredictable.
Another approach is to keep latest generated UUID and if new UUID requested within the same timestamp - generate random part until it's greater than previous one. I think that's pretty good approach as well.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#24Whenever ULID comes up, I need to remind that it has a sequential ID generation mode in its spec which is prone to conflicts on multi-threads, processes or hosts which kills the purpose of a "universal" identifier. If you need a sequential ID, just use an integer, preferably one that's autoincremented by the database. It's best to stick to UUIDv7 because of such quirks of ULID.
> If you need a sequential ID, just use an integer
Are monotonic/sequential ULIDs as easily enumerated as integers? It's the ease of enumerability that keeps a lot of folks away from using sequential integers as IDs
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#25Whenever ULID comes up, I need to remind that it has a sequential ID generation mode in its spec which is prone to conflicts on multi-threads, processes or hosts which kills the purpose of a "universal" identifier. If you need a sequential ID, just use an integer, preferably one that's autoincremented by the database. It's best to stick to UUIDv7 because of such quirks of ULID.
ULID's initial segment is timestamp generated, with a random suffix at the end. This kind of collision you're concerned about is not an issue at all, across multi-threads, processes or hosts.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#26Interesting article noting how ULIDs solve database index fragmentation caused when using UUIDv4. However, for extremely high-volume writes, ULIDs create "hot spots" at the current timestamp index location, potentially causing contention. The article notes that UUID v7 (newly standardized) adopts the same time-ordered approach, validating ULID's design.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#27Earlier quoted context omitted.
Under what circumstances is it prone to conflicts? On separate threads/hosts/processes, id's created within the same millisecond would be differentiated by the 80 bits of randomness (more than UUID v7).
No, ULID has a "monotonic" feature, where if it detects the same millisecond timestamp in back to back calls, it just increments the 80 bit "random" portion. This means it has convoying behavior. If two machines are generating ids independently and happen to choose initial random positions near each other, the probability of collision is much higher than the basic birthday bound. I think this "sort of monotonic but n…
> If two machines are generating ids independently and happen to choose initial random positions near each other, the probability of collision is much higher than the basic birthday bound.
But the chance of the initial random positions being near each other is very very low.
If you pick a billion random numbers in an 80 bit space, the chance you have a collision is one in a million. (2^80 / (2^30)^2)
If you pick a thousand random starting points and generate a million sequential numbers each, the chance your starting points are sufficiently close to each other to cause an overlap is one in a trillion. ((2^80 / 2^20) / (2^10)^2)
In that one in a trillion case, you'll likely end up with half a million collisions, which might matter to you. But if you care about 0 collisions versus 1+ collisions, pick the monotonic version.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#28Earlier quoted context omitted.
You're right, not hosts or processes in that case. I forgot about random part as it's been a while since I looked at it. However, a single instance of a ULID generator must support this mode, which means that on multi-threaded architectures, it must lock the sequence as it still uses a single random value. That again, kills the purpose of a client-side, lock-free generation of universal identifiers as you said.
You only need to lock sequence if you care about IDs being ordered within a millisecond. That generally only matters when you create a batch of IDs at once, in that case you don't need to lock anything: generate ULID, keep incrementing sequence in that batch either by doing on the same thread, or by moving it from thread to thread. Kinda like creating an iterator and zip'ing it with iterator of thing you need IDs for…
> You only need to lock sequence if you care about IDs being ordered within a millisecond
Yes, and that's when sequences are only used. I guess that's to avoid hogging the CPU or emptying the OS entropy pool during high loads.
However, that "optimization" is a failure mode if you're not aware how ULID internals work. It's easy to shoot yourself in the foot by blindly trusting ULID will always generate a unique ID across threads without blocking your thread. That's a sneaky footgun.
> That generally only matters when you create a batch of IDs at once
No, any web service instance can receive requests at arbitrary times, and sometimes in the same millisecond zone. The probability is proportional to the number of concurrent users and requests.
> If your goal is to have global order intact, then neither ULID nor UUIDv7 is going to work for you.
Agreed.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#29Whenever ULID comes up, I need to remind that it has a sequential ID generation mode in its spec which is prone to conflicts on multi-threads, processes or hosts which kills the purpose of a "universal" identifier. If you need a sequential ID, just use an integer, preferably one that's autoincremented by the database. It's best to stick to UUIDv7 because of such quirks of ULID.
ULID's initial segment is timestamp generated, with a random suffix at the end. This kind of collision you're concerned about is not an issue at all, across multi-threads, processes or hosts.
Not if the same ULID generator instance is used across threads.
Re: ULID: Universally Unique Lexicographically Sortable Identifier
#30Whenever ULID comes up, I need to remind that it has a sequential ID generation mode in its spec which is prone to conflicts on multi-threads, processes or hosts which kills the purpose of a "universal" identifier. If you need a sequential ID, just use an integer, preferably one that's autoincremented by the database. It's best to stick to UUIDv7 because of such quirks of ULID.
> If you need a sequential ID, just use an integer Are monotonic/sequential ULIDs as easily enumerated as integers? It's the ease of enumerability that keeps a lot of folks away from using sequential integers as IDs
You mean someone who wants to attack your system might be discouraged by Base32 encoding?