Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

github.com

Post reply on HN
Print view View original

171–180 of 236 posts

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#171

Post by unshavedyak »

aftbit wrote: post #143
klabb3 wrote: post #141

Earlier quoted context omitted.

> We have tests for the base32 encoding which is the most complicated part of the implementation I didn’t look into it much but it seems like a great encoding even outside of this project. Predictable length, reasonable density, “double clickable” etc. I’ve been annoyed with both hex and base64 for a while so it’s pretty cool just by itself. > Re: prefix, is the concern that I haven't defined the allowed character se…

My personal favorite encoding is base58 aka Bitcoin address encoding. It uses characters [A-Za-z0-9] except for [0OIl]. It is almost as dense as base64, "double clickable", but not (as) predictable in length as base32. It was chosen to avoid a number of the most annoying ambiguous letter shapes for hand-entry of long address strings. https://en.bitcoin.it/wiki/Base58Check_encoding

Is there a good reason why Base63 (nopad) doesn't exist? Ie Base64 minus the `-`, so that you almost get the density of base64 (nopad) but the double click friendly feature.

I was reviewing encodings recently and didn't want to drop all the way down to base32, but for some reason the library i was using didn't allow anything beyond base32 and bas64 variants, despite having a feature where you can define your own base.

I thought maybe it was performance oriented. An odd prefix length like base63 would mean .. i think, a slightly more computationally demanding set of encoding instructions?

Either way i basically want base58 but i don't care about legibility, i just wanted double click and url friendly characters.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#172

Post by Ironchefpython »

crdrost wrote: post #93
swyx wrote: post #57

for those researching this topic, I keep a list of these UUID/GUID implementations! https://github.com/swyxio/brain/blob/master/R%20-%20Dev%20No...

Thanks for this! I have one idea which is perhaps nerdy enough to make the list but I've never fully fleshed it out, it's that one can encode the nonnegative integers {0, 1, 2, ...} into the finite bitstrings {0, 1}* in a way which preserves ordering. So if we use hexits for the encoding the idea would be that 0=0, 1=1, ... E=14, then F00 = 15 F01 = 16 ... F0F = 30 F100 = 31 F101 = 32 ... F1FF = 286 F2000 = so the fo…

https://datatracker.ietf.org/doc/html/rfc2550

Note that this RFC also supports lexicographic sorting for negative numbers using the 10k complement notation.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#173

Post by dloreto »

inopinatus wrote: post #159

I'm not wild about the Crockford encoding. In practice I've found it to be a flat-out mistake when you come to provide technical support or analysis for values encoded this way. The Crockford alphabet is based on design goals that are rarely encountered in practice, such as pronouncing identifiers over the phone. It introduces ambiguity, which is a disaster for grepping logs or any other circumstances where you might…

I hear you ... and I debated using either base58 or base64url. I do like the more compact encoding they provide.

Ultimately I ended up leaning towards a base32 encoding, because I didn't want to pre-suppose case sensitivity. For example, you might want to use the id as a filename, and you might be in an environment where you're stuck with a case insensitive filesystem.

Note that TypeID is using the Crockford alphabet and always in lowercase – *not* the full rules of Crockford's encoding. There's no hyphens allowed in TypeIDs, nor multiple encodings of the same ID with different variations of the ambiguous characters.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#174

Post by dap »

It's weird to me that people talk about UUIDv4 as bad for databases because of locality. This behavior can be good or bad, depending on what you're doing. Sure, UUIDv4 is worse if you're using it to locate a bunch of related objects that you want to be on the same database node. (If you're doing that, though, you can often attach some kind of group identifier -- e.g., "user" -- and index that.) I generally prefer UUIDv4 in a lot of distributed database applications because as a sharding key it's likely to distribute data well across all available nodes.

I debugged a system that used the equivalent of time-ordered UUIDs where even though the database was horizontally scaled, the performance was limited by the capacity of one server. It was exactly because the uuids being generated started with similar prefixes, so at any given time, all the current data was going to whichever database node was responsible for that range of the keyspace.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#175

Post by bumbledraven »

veec_cas_tant wrote: post #154
dloreto wrote: post #118

Earlier quoted context omitted.

The checksum idea is interesting. I'm considering whether it makes sense to add it as part of the TypeID spec.

What value does the checksum provide? I think I'm missing something because I really don't see a benefit.

Checksums facilitate error detection. For typed UUIDs, checksums help detect errors introduced by changing the prefix/type or changing a “digit”.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#177

Post by autoexec »

darajava wrote: post #161

In the readme, Crockford’s alphabet is referenced [0]. In this specification, “U” is excluded because it is an “accidental obscenity”. Does anyone have any idea what that means? Is it a joke? 0. https://www.crockford.com/base32.html

I'm guessing that it's because U just happens to appear in some popular four letter words, which might make things awkward when you're reading letters out to another person over the phone. It might also come before a lot of other rude letter combinations making them seem more personal.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#178

Post by TedDoesntTalk »

unshavedyak wrote: post #171
aftbit wrote: post #143

Earlier quoted context omitted.

My personal favorite encoding is base58 aka Bitcoin address encoding. It uses characters [A-Za-z0-9] except for [0OIl]. It is almost as dense as base64, "double clickable", but not (as) predictable in length as base32. It was chosen to avoid a number of the most annoying ambiguous letter shapes for hand-entry of long address strings. https://en.bitcoin.it/wiki/Base58Check_encoding

Is there a good reason why Base63 (nopad) doesn't exist? Ie Base64 minus the `-`, so that you almost get the density of base64 (nopad) but the double click friendly feature. I was reviewing encodings recently and didn't want to drop all the way down to base32, but for some reason the library i was using didn't allow anything beyond base32 and bas64 variants, despite having a feature where you can define your own base…

It’s not too difficult to write your own encoding. Probably 10 lines of code or less if you hard-code your encoding alphabet.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#179

Post by Racing0461 »

taosx wrote: post #22
stronglikedan wrote: post #16

Earlier quoted context omitted.

You can prevent any obscenity, O and 0 confusion, and I and L confusion, just by excluding vowels. If someone interprets "f4g" in an offensive way, then they have bigger issues than can be dealt with in software.

Why we care about obscenity in pseudo-random ids and url?

i don't, the people that can start a **storm on twitter/tumblr causing your stock to drop 10% does.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#180

Post by TedDoesntTalk »

pizzapill wrote: post #39
theptip wrote: post #20

Earlier quoted context omitted.

The Scunthorpe problem? https://en.m.wikipedia.org/wiki/Scunthorpe_problem

E-Mail accounts seem the worst. Just lets write letters again, if you need a pencil I recommend penisland.net

Penis mean “tail” in Latin.
Post reply on HN