Live data from Hacker News

Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

github.com

81–90 of 236 posts

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#81
post #34

Earlier quoted context omitted.

Not every bit of UUID is required to be random. The goals are smallness, uniqueness, monotonicity, resistance to enumeration attacks, etc. Not randomness for randomness sake. My UUIDv7+ can be consumed as a standard UUIDv7. It is not intended to be v8. A program can treat the last 16 bits as random noise if it wants.

Well UUIDv7 can be consumed as a UUIDv4 in the same way, its just 16 bytes. The point of the standard is to define how the particular bytes are chosen.

The latest standard for v7 does not meaningfully describe how to interpret the last segment.

It says they could be pseudorandom and non-monotonic. Or it could be monotonic and non-random. These are completely disjoint cases! "X or not X" is tautological. And there is no way to determine which (e.g. there could be a flag that indicates this mode, but there is not).

To be clear, the standard should be amended to resolve this ambiguity. Say the last bits MAY be monotonic or MAY be pseudorandom. Or add a flag that indicates which.

As there is currently no standard way to interpret these bits, I feel perfectly justified in using the a few of the least significant ones to encode additional information.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#82

Earlier quoted context omitted.

If I and O are already excluded and you also exclude U that removes a lot of potential rude looking three letter combinations like *** and *** and *** and also the four letter ones like **** and **** and the dreaded ****. Of course because you have A then **** is still a possibility but very very unlikely

Wow I didn't know HN even had obscenity filters, and I've been here for many years. Guess that's a credit to the general civility of the community. EDIT: It appears that other people in this thread are freely using profanity, so either your comment was targeted by automation due to the unusual density of banned words, or it's a joke that went over my head :)

[deleted]

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#83
post #75

Earlier quoted context omitted.

That if a hundred servers are generating (uuid, timestamp) tuples that are subsequently merged on a single machine, and sorted by uuid, it would have almost the same order as if sorted by timestamp. This property is useful for RDBMS writes, when the UUID is used as a primary key and this locality ensures that fewer slotted pages need to be modified to write the same amount of data.

> it would have almost the same order as if sorted by timestamp. Is there documentation covering the scenarios on how the order can become out of sync and what the odds are? There's a big difference between "almost" and "always" if we're talking about using this as a database PK.

> There's a big difference between "almost" and "always"

Not in the context of an RDBMS, which use b+/b*-tree variants (or LSM sstables). Sequentially generated UUID's will end up near each other when sorted lexicographically, regardless of the fact that the sort order doesn't perfectly match the timestamp order.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#84
post #10

Naive Question: The type safe part is just appending a string at the beginning? What if I do that with UUIDv4? is user_49b9cd12-9964-4b9c-8512-742f0a2c9be4 type safe now?

Yep. The whole point is that you /never/ assign ids that begin with "user" to types that are not users. Because of that, you can be sure nobody can accidentally copy an id that begins with "user" when meaning to address a different type and get back a result other than "not found".

Example:

I have userId=4 and userId=2. Suppose a user can have multiple bank accounts and userId=4 has accountId=5 and accountId=6 and a defaultAccound accountId=5. userId=2 has an account, accountId=7; I want to send userId=4 some money so I use the function `sendUserMoneyFromAccount(to: int, from: int)`. This is a bad interface but these things exist in the wild a lot. I could accidentally assume that because I want to send userId=4 the money to their default account that I would call it using `sendUserMoneyFromAccount(4, 7)` and that would work, but if under the hood it wants 2 accountIds, I've just sent accountId=4 money rather than userId=4's defaultAccount, accountId=5.

With prefixed ids that indicate type, a function that assumes type differently from the one supplied will not accidentally succeed.

In addition, humans who copy ids will be less likely to mistake them. This is just an ergonomic/human centric typing.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#88
post #4

Unrelated, but this links to "Crockford's alphabet", https://www.crockford.com/base32.html , which is a base-32 system that includes all alphanumeric characters except I and L (which are confusable with 1), O (which is confusable with 0), and U (????). The page says the reason for excluding U is "accidental obscenity'. What the heck is it talking about?

The F word has a U in it. Sure you could just say FVCK

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#89
post #34

Earlier quoted context omitted.

Not every bit of UUID is required to be random. The goals are smallness, uniqueness, monotonicity, resistance to enumeration attacks, etc. Not randomness for randomness sake. My UUIDv7+ can be consumed as a standard UUIDv7. It is not intended to be v8. A program can treat the last 16 bits as random noise if it wants.

I am aware, just saying per spec, its supposed to be random bit data, thats all I was saying. I am familiar with a spec since I maintain a UUID library that has 6,7, and a custom 8 implemented. It can have extra monotonicity data instead, per section 6.2 but ideally its random. Again, Not saying you can't do what you are doing, I just know per the conversations while the draft was gathering feedback, your type of cha…

> per spec, its supposed to be random bit data

> It can have extra monotonicity data instead

Well, which is it? These are incompatible requirements.

If I give you a standard UUIDv7 sample, it is impossible for you to interpret the last 62 bits. You cannot determine how they were generated. If I give you two samples with the same timestamp, you cannot say which was generated first. These bits are de facto uninterpretable, unlike e.g. the 48 MSB, which have clearly defined semantics.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#90
post #13

I have some complaints about UUIDs. Why not just combining time + random number without the ceremony of UUID versioning. And for when locality doesn't matter, just use a 128bit random number directly. And in my experience most people somehow think a UUID must be stored into the human friendly hex representation, dashes included. Wasting so much space in database, network, memory.

While this isn't the worst area I see this in, there does seem to be a tendency in the UUID space to speak as if one use case stands for all and therefore there is a best UUID format.

The reality is that it is just like any other engineering situation. Sit down, write down your requirements, and see what, if anything, solves it.

Reading about the advantages of various formats is very helpful in helping you skip learning about certain things the hard way and use somebody else's experience of learning them the hard way instead. From that point of view I recommend at least glancing through them all. Sortability and time-based locality is one that you may not naturally think about, and if you need it, you will appreciate not learning that the hard way four years into a project after you threw that data away and then realizing you needed it. And some UUID formats actually managed to introduce small security issues into themselves (thinking MAC address leak from UUID v1 here), nice to avoid those too.

If you have a use case where there's an existing solution then, hey, great, go ahead and use it. Maybe if anyone ever needs that but in another language they can pull a library there too.

But if not, don't sweat it. The biggest use of UUIDs I personally have I specified as "just send me a unique string, use a UUID library of your choice if it makes you feel better". I think I've got a unique format per source of data in this system and it's fine. I don't have volume problems, it's tens of thousands of things per day. I don't have any need to sort on the UUID, they're not really the "identifier", they're just a unique token generated for a particular message by the originator of the message so we can detect duplicate arrivals downstream in a heterogenous system where I can't just defer that task to the queue itself since we have multiple. I don't even need them to be globally unique, I just need them unique within a rather small shard, and in principle I wouldn't even mind if they get repeated after a certain amount of time (though I left the system enforcing across all time anyhow for simplicity). In this particular case, I do indeed generate my own UUIDs for the stuff I'm originating by just grabbing some stuff from /dev/urandom and encoding it base64, with a size selected such that base64 doesn't end the encoding with ==. Even that's just for aesthetic's sake rather than any actual problem it would cause.

Post reply on HN