Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

github.com

Post reply on HN
Print view View original

111–120 of 236 posts

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#111

Post by mtlmtlmtlmtl »

pluijzer wrote: post #99
kibwen wrote: post #64

Earlier quoted context omitted.

> The page says the reason for excluding U is "accidental obscenity'. Crockford is being cheeky. To make a nice base32 alphabet out of non-confusable alphanumeric characters you only need to exclude O, I, and L. This leaves you with 33 characters still, so you need to remove one more, and it doesn't matter which one you remove, so you might as well pick an arbitrary reason for the last character that gets removed (an…

You could argue that U can be confused with V.

A vaguely related historical tangent is that V and U used to be just two ways of writing the same letter in Early Modern English. Which I imagine is why W is named as "double U" in speaking.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#112

Post by littlestymaar »

oleganza wrote: post #71
pavlov wrote: post #12

Earlier quoted context omitted.

True Latinists find the letter U vulgar to the point of obscenity because it didn’t exist in Cicero’s time.

Trve Latinists wovld appreciate yovr point.

Gotcha, there was no “W” in the Latin alphabet either ;)

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#113

Post by zrail »

aartav wrote: post #95

I've been doing this kind of thing for years with two notable differences: 1. I don't believe people actually hand type-in these values, so I'm not really concerned about the 'l' vs '1' issue. I do base 32 without `eiou` (vowels) to reduce the likelihood of words (profanity) sneaking in. 2. I add two base-32 characters as a checksum (salted of course). This is prevents having to go look at the datastore when the valu…

I implemented number two as part of an encoding scheme a few months ago. I'm not sure how much it's saved in terms of database lookups but it's aesthetically pleasing to know it won't hit a more inscrutable error while trying to decode.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#114

Post by michaelt »

nickjj wrote: post #75
tomnipotent wrote: post #53

Earlier quoted context omitted.

That if a hundred servers are generating (uuid, timestamp) tuples that are subsequently merged on a single machine, and sorted by uuid, it would have almost the same order as if sorted by timestamp. This property is useful for RDBMS writes, when the UUID is used as a primary key and this locality ensures that fewer slotted pages need to be modified to write the same amount of data.

> it would have almost the same order as if sorted by timestamp. Is there documentation covering the scenarios on how the order can become out of sync and what the odds are? There's a big difference between "almost" and "always" if we're talking about using this as a database PK.

The key seems to be based on UUIDv7, starting with a timestamp in milliseconds.

So the order can become out of sync if multiple events happen in the same millisecond; or if your servers' clock error is greater than a millisecond (i.e. if you're an NTP user)

More than sufficient for things like ordering tweets. If you're ordering bank account transactions, well, you'd probably be using transactions in an ACID-compliant relational database.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#115

Post by lll-o-lll »

K-Sortable is a great concept; having weakly sorted keys solves a bunch of use-cases. I really like the idea of a typed, condensed string representation. However I wonder if an unintended side affect of UUID V7 is going to be a bunch of security problems.

People aren’t meant to use uuids as tokens, and they aren’t supposed to use PKs from a DB for this either - but they do. Because UUID v4 is basically crypto random, I think we’ve been getting away with a bunch of security weaknesses that would otherwise be exploited.

With UUID v7 we are back to 32bits of actually random data. It’s going to require some good educating to teach devs that uuids are guessable.

[edit] Looks like I am off base with the guess-ability of the V7 UUID, as the draft recommends CSPRNG for the random bits, and the amount of entropy is at least 74 bits and it is specifically designed to be “unguessable”. It does say “UUID v4” for anything security related, but perhaps that is simply in regard to the time stamp?

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#116

Post by netcraft »

nzgrover wrote: post #58
tomnipotent wrote: post #53

Earlier quoted context omitted.

That if a hundred servers are generating (uuid, timestamp) tuples that are subsequently merged on a single machine, and sorted by uuid, it would have almost the same order as if sorted by timestamp. This property is useful for RDBMS writes, when the UUID is used as a primary key and this locality ensures that fewer slotted pages need to be modified to write the same amount of data.

Is that what they mean by "used as the primary key in a database while ensuring good locality"/"database locality"? That read/write access will hit fewer disk pages?

yes, exactly

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#117

Post by Eduard »

vikeri wrote: post #19

Another, less known, useful thing about these IDs is that you can double click on them and the full id will always be selected

Also, they are safe to use within filenames and directory names (Filesystem paths) without conversion (at least in today's Filesystem not limited to e.g. 8.3 characters) .

Compare that with otherwise nice ISO 8601 datetime format (e.g. 2023-06-28T21:47:59+00:00): it requires conversion for file systems that don't allow colons and plus signs.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#118

Post by dloreto »

aartav wrote: post #95

I've been doing this kind of thing for years with two notable differences: 1. I don't believe people actually hand type-in these values, so I'm not really concerned about the 'l' vs '1' issue. I do base 32 without `eiou` (vowels) to reduce the likelihood of words (profanity) sneaking in. 2. I add two base-32 characters as a checksum (salted of course). This is prevents having to go look at the datastore when the valu…

The checksum idea is interesting. I'm considering whether it makes sense to add it as part of the TypeID spec.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#119

Post by snuxoll »

Xeoncross wrote: post #98
tasn wrote: post #63

Earlier quoted context omitted.

We maintain a couple of popular ksuid libraries[1][2] and use it, so we definitely like ksuid. Though one big issue with ksuid is that being 160bit means that it doesn't fit into native uuid types in databases (e.g. postgres), which means that they come with a performance penalty. 1: https://github.com/svix/rust-ksuid 2: https://github.com/svix/python-ksuid

I'm curious, why do you not store these as binary data or do you and you're saying that the UUID operations are better optimized than sorts on binary data?

I can compare a 128bit UUID in a single instruction, a 160-bit ksuid is a little weirder to work with at the hardware level.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#120

Post by klabb3 »

A couple of suggestions:

Lock down the prefix string now before it’s too late and document it. I see in Go that it’s lowercase ascii, which seems fine except for compound types (like “article-comment”). May be worth looking at allowing a single separator given that many complex projects (and ORMs) can’t avoid them.

The Go implementation has no tests. This is very unit-testable. Add tests goddammit!

For Go, I’d align with Googles UUID implementation, with proper parse functions and an internal byte array instead of strings. Strings are for rendering (and in your case, the prefix). Right now, it looks like the parsing is too permissive, and goes into generation mode if the suffix is empty. And the SplitN+index thing will panic if no underscores, no? Anyway, tests will tell.

As for the actual design decisions, I tried to poke holes but I fold! I think this strikes the sweet spot between the different tradeoffs. Well done!

Post reply on HN