Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

github.com

Post reply on HN
Print view View original

191–200 of 236 posts

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#191

Post by fauigerzigerk »

unshavedyak wrote: post #171
aftbit wrote: post #143

Earlier quoted context omitted.

My personal favorite encoding is base58 aka Bitcoin address encoding. It uses characters [A-Za-z0-9] except for [0OIl]. It is almost as dense as base64, "double clickable", but not (as) predictable in length as base32. It was chosen to avoid a number of the most annoying ambiguous letter shapes for hand-entry of long address strings. https://en.bitcoin.it/wiki/Base58Check_encoding

Is there a good reason why Base63 (nopad) doesn't exist? Ie Base64 minus the `-`, so that you almost get the density of base64 (nopad) but the double click friendly feature. I was reviewing encodings recently and didn't want to drop all the way down to base32, but for some reason the library i was using didn't allow anything beyond base32 and bas64 variants, despite having a feature where you can define your own base…

>Is there a good reason why Base63 (nopad) doesn't exist? Ie Base64 minus the `-`, so that you almost get the density of base64 (nopad) but the double click friendly feature.

Yes, the reason is that you need 64 characters if you want each character to encode 6 bits as log2(64) == 6. If you only have 63 characters in your alphabet then one of your 6-bit combinations has no character to represent it.

Base32 can represent 5 bits per character because log2(32) == 5. Anything in between 32 and 64 doesn't buy you anything because there is no integer between 5 and 6.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#192

Post by atonse »

programmarchy wrote: post #6
ajkjk wrote: post #4

Unrelated, but this links to "Crockford's alphabet", https://www.crockford.com/base32.html , which is a base-32 system that includes all alphanumeric characters except I and L (which are confusable with 1), O (which is confusable with 0), and U (????). The page says the reason for excluding U is "accidental obscenity'. What the heck is it talking about?

Yeah, wtf?

Obviously by “wtf” you must mean “why the face?” Right? Right?? :-)

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#193

Post by atonse »

lll-o-lll wrote: post #115

K-Sortable is a great concept; having weakly sorted keys solves a bunch of use-cases. I really like the idea of a typed, condensed string representation. However I wonder if an unintended side affect of UUID V7 is going to be a bunch of security problems. People aren’t meant to use uuids as tokens, and they aren’t supposed to use PKs from a DB for this either - but they do. Because UUID v4 is basically crypto random,…

This brings up an interesting ergonomics problem.

By naming them UUIDv4 and UUIDv7, is it going to be this never ending confusion for people to have to remember which one is good for databases and which one good for one time tokens?

Not sure what the backwards compatible solution here is either.

In elixir the function is UUID.uuid4() to generate a v4 UUID.

So we could theoretically scan code for its use I suppose. But all this increases chances of errors.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#194

Post by mtlmtlmtlmtl »

jabbany wrote: post #126
mtlmtlmtlmtl wrote: post #111

Earlier quoted context omitted.

A vaguely related historical tangent is that V and U used to be just two ways of writing the same letter in Early Modern English. Which I imagine is why W is named as "double U" in speaking.

This is also interesting since in French (and I think Spanish?) W is (correctly) called "double V"

Same thing in Norwegian. But W is also not really a part of Norwegian orthography, it's just kind of there in the alphabet anyway. Only useful for names and maybe a couple loanwords.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#195

Post by inopinatus »

ash wrote: post #181
inopinatus wrote: post #159

I'm not wild about the Crockford encoding. In practice I've found it to be a flat-out mistake when you come to provide technical support or analysis for values encoded this way. The Crockford alphabet is based on design goals that are rarely encountered in practice, such as pronouncing identifiers over the phone. It introduces ambiguity, which is a disaster for grepping logs or any other circumstances where you might…

I agree that pronouncing identifiers over the phone is rare. But I’m occasionally typing identifiers from: 1. a screenshot or a screen share that contains an identifier 2. another device where I can’t easily take an identifier

That's fair. From experience I think the most common problem with screenshots is [0O] and [Il] ambiguity. As a point of comparison I'm willing to suggest that both base58 and crockford32 handle the matter reasonably, albeit differently, through their omitted-characters and decoding tables.

One feature I do like from crockford32, that base58 lacks, and which also assists transcription from noisy sources, is the check symbol. So much that it is quite unfortunate that this check symbol is optional. In 2023 it's hard to fight the urge to specify a mandatory emoji to encode a check value (caveat engineer: this is not actually a good idea :))

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#196

Post by vore »

jandrewrogers wrote: post #45
ceejayoz wrote: post #33

Earlier quoted context omitted.

No identifier can guarantee that. We just get close enough to be acceptable. Per Wikipedia, the probability to find a duplicate within 103 trillion version-4 UUIDs is one in a billion. so-youre-saying-theres-a-chance.gif

I have single datasets with trillions of UUID. Collision probability becomes a thing. That aside, UUIDv4 is banned in many orgs because there have been several instances in the wild where the “random” number wasn’t nearly as random as advertised from some sources for a variety of reasons, leading to collisions. It is relatively easy to screw this up so many orgs don’t risk it.

A 1 in a billion collision probability?

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#197

Post by jhoechtl »

klabb3 wrote: post #120

A couple of suggestions: Lock down the prefix string now before it’s too late and document it. I see in Go that it’s lowercase ascii, which seems fine except for compound types (like “article-comment”). May be worth looking at allowing a single separator given that many complex projects (and ORMs) can’t avoid them. The Go implementation has no tests. This is very unit-testable. Add tests goddammit! For Go, I’d align…

I think your misconception is that prefix is sthg. fixed? You decided on the prefix depending on the usage domain.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#199

Post by dolmen »

dloreto wrote: post #134
klabb3 wrote: post #120

A couple of suggestions: Lock down the prefix string now before it’s too late and document it. I see in Go that it’s lowercase ascii, which seems fine except for compound types (like “article-comment”). May be worth looking at allowing a single separator given that many complex projects (and ORMs) can’t avoid them. The Go implementation has no tests. This is very unit-testable. Add tests goddammit! For Go, I’d align…

Thanks for the feedback! We have tests for the base32 encoding which is the most complicated part of the implementation ( https://github.com/jetpack-io/typeid-go/blob/main/base32/bas... ) but your point stands. We'll add a more rigorous test suite (particularly as the number of implementations across different languages grows, and we want to make sure all the implementations are compatible with each other) Re: prefix…

There is no tests.

There is just a single test. Which only tests the decoding of a single known value. No encoding test.

Go has infrastructure for benchmarking and fuzzing. Use it!

Also, you took code from https://github.com/oklog/ulid/blob/main/ulid.go which has "Copyright 2016 The Oklog Authors" but this is not mentionned in your base32.go.

Re: Type-safe, K-sortable, globally unique identifier inspired by Stripe IDs

#200

Post by dolmen »

kibwen wrote: post #64
ajkjk wrote: post #4

Unrelated, but this links to "Crockford's alphabet", https://www.crockford.com/base32.html , which is a base-32 system that includes all alphanumeric characters except I and L (which are confusable with 1), O (which is confusable with 0), and U (????). The page says the reason for excluding U is "accidental obscenity'. What the heck is it talking about?

> The page says the reason for excluding U is "accidental obscenity'. Crockford is being cheeky. To make a nice base32 alphabet out of non-confusable alphanumeric characters you only need to exclude O, I, and L. This leaves you with 33 characters still, so you need to remove one more, and it doesn't matter which one you remove, so you might as well pick an arbitrary reason for the last character that gets removed (an…

This assumes that english is the only relevant language regarding curse words. Which is quite biased.
Post reply on HN