Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

github.com

Post reply on HN
Print view View original

91–100 of 236 posts

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#91

Post by Zamicol »

ajkjk wrote: post #4

Unrelated, but this links to "Crockford's alphabet", https://www.crockford.com/base32.html , which is a base-32 system that includes all alphanumeric characters except I and L (which are confusable with 1), O (which is confusable with 0), and U (????). The page says the reason for excluding U is "accidental obscenity'. What the heck is it talking about?

There's more!

- base 58 - Satoshi's/Bitcoin's https://en.wikipedia.org/wiki/Binary-to-text_encoding#Base58

- "base62" - Keybase's saltpack https://github.com/keybase/saltpack

- The famous "Adobe 85" - https://en.wikipedia.org/wiki/Ascii85

- basE91 - https://base91.sourceforge.net

At work we defined several new "bases" for QR code. IMHO, it is an under applied area of computer science.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#92

Post by taberiand »

titanomachy wrote: post #70
codeulike wrote: post #48

Earlier quoted context omitted.

If I and O are already excluded and you also exclude U that removes a lot of potential rude looking three letter combinations like *** and *** and *** and also the four letter ones like **** and **** and the dreaded ****. Of course because you have A then **** is still a possibility but very very unlikely

Wow I didn't know HN even had obscenity filters, and I've been here for many years. Guess that's a credit to the general civility of the community. EDIT: It appears that other people in this thread are freely using profanity, so either your comment was targeted by automation due to the unusual density of banned words, or it's a joke that went over my head :)

No obscenity filters, but there is a pretty good password filter I hear. For example, my password 'hunter2' will be all **** to you

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#93

Post by crdrost »

swyx wrote: post #57

for those researching this topic, I keep a list of these UUID/GUID implementations! https://github.com/swyxio/brain/blob/master/R%20-%20Dev%20No...

Thanks for this!

I have one idea which is perhaps nerdy enough to make the list but I've never fully fleshed it out, it's that one can encode the nonnegative integers {0, 1, 2, ...} into the finite bitstrings {0, 1}* in a way which preserves ordering.

So if we use hexits for the encoding the idea would be that 0=0, 1=1, ... E=14, then

    F00 = 15
    F01 = 16
    ...
    F0F = 30
    F100 = 31
    F101 = 32
    ...
    F1FF = 286
    F2000 =
so the format is F, which is the overflow sigil, followed by a recursive representation of the length of the coming string, followed by a string of hexits that long.

What if you need 16 hexits? That's where the recursion comes in,

    F F00 0123456789ABCDEF
     \   \    \
      \   \    \----- 16 hexits
       \   \
        \   \-- the number 15, "there are 15+1 digits to follow"
         \       (consisting of overflow, 0+1 digits to follow, and hex 0)
          \ 
           \--- overflow sigil
Kind of goofy but would allow a bunch of things like "timestamp * 1024 + 10-bit machine ID" etc without worrying about the size of the numbers involved

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#94

Post by AceJohnny2 »

codeulike wrote: post #48
ajkjk wrote: post #4

Unrelated, but this links to "Crockford's alphabet", https://www.crockford.com/base32.html , which is a base-32 system that includes all alphanumeric characters except I and L (which are confusable with 1), O (which is confusable with 0), and U (????). The page says the reason for excluding U is "accidental obscenity'. What the heck is it talking about?

If I and O are already excluded and you also exclude U that removes a lot of potential rude looking three letter combinations like *** and *** and *** and also the four letter ones like **** and **** and the dreaded ****. Of course because you have A then **** is still a possibility but very very unlikely

you accidentally the whole thing

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#95

Post by aartav »

I've been doing this kind of thing for years with two notable differences:

1. I don't believe people actually hand type-in these values, so I'm not really concerned about the 'l' vs '1' issue. I do base 32 without `eiou` (vowels) to reduce the likelihood of words (profanity) sneaking in.

2. I add two base-32 characters as a checksum (salted of course). This is prevents having to go look at the datastore when the value is bogus either by accident or malice. I'm unsure why other implementations don't do this.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#96

Post by 9dev »

taberiand wrote: post #92
titanomachy wrote: post #70

Earlier quoted context omitted.

Wow I didn't know HN even had obscenity filters, and I've been here for many years. Guess that's a credit to the general civility of the community. EDIT: It appears that other people in this thread are freely using profanity, so either your comment was targeted by automation due to the unusual density of banned words, or it's a joke that went over my head :)

No obscenity filters, but there is a pretty good password filter I hear. For example, my password 'hunter2' will be all **** to you

Isn’t it nice how some traditions do stick around. It’s been a while, Cthon98!

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#97

Post by jtmarmon »

This looks great! Is there a reason one couldn't use this with v4 UUIDs? A quick test shows that they encode/decode just fine. Wondering if I could use the encoded form as a way to niceify our URLs without having to change how the IDs (currently v4 uuids) are stored

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#98

Post by Xeoncross »

tasn wrote: post #63
Xeoncross wrote: post #9

Assuming you don't need to use UUIDv7 (or any UUID's) then https://github.com/segmentio/ksuid provides a much bigger keyspace. You could just append a string prefix if you wanted to namespace, but the chance of collisions of a ksuid is many times smaller than a UUID of any version. ksuid is the best general purpose id generator with sort-able timestamps I've found and has libraries in most languages. UUID v1-7 are wa…

We maintain a couple of popular ksuid libraries[1][2] and use it, so we definitely like ksuid. Though one big issue with ksuid is that being 160bit means that it doesn't fit into native uuid types in databases (e.g. postgres), which means that they come with a performance penalty. 1: https://github.com/svix/rust-ksuid 2: https://github.com/svix/python-ksuid

I'm curious, why do you not store these as binary data or do you and you're saying that the UUID operations are better optimized than sorts on binary data?

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#99

Post by pluijzer »

kibwen wrote: post #64
ajkjk wrote: post #4

Unrelated, but this links to "Crockford's alphabet", https://www.crockford.com/base32.html , which is a base-32 system that includes all alphanumeric characters except I and L (which are confusable with 1), O (which is confusable with 0), and U (????). The page says the reason for excluding U is "accidental obscenity'. What the heck is it talking about?

> The page says the reason for excluding U is "accidental obscenity'. Crockford is being cheeky. To make a nice base32 alphabet out of non-confusable alphanumeric characters you only need to exclude O, I, and L. This leaves you with 33 characters still, so you need to remove one more, and it doesn't matter which one you remove, so you might as well pick an arbitrary reason for the last character that gets removed (an…

You could argue that U can be confused with V.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#100

Post by sokoloff »

aartav wrote: post #95

I've been doing this kind of thing for years with two notable differences: 1. I don't believe people actually hand type-in these values, so I'm not really concerned about the 'l' vs '1' issue. I do base 32 without `eiou` (vowels) to reduce the likelihood of words (profanity) sneaking in. 2. I add two base-32 characters as a checksum (salted of course). This is prevents having to go look at the datastore when the valu…

> base 32 without `eiou` (vowels) to reduce the likelihood of words (profanity) sneaking in.

We had “analrita” as an autogenerated password that resulted in a complaint many years ago. Might consider adding ‘a’ as an excluded letter.

Post reply on HN