Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UUIDs are popular, but bad for performance (2019)

percona.com

Post reply on HN
Print view View original

211–220 of 246 posts

Re: UUIDs are popular, but bad for performance (2019)

#211

Post by Zamicol »

carfacts wrote: post #139
mhoad wrote: post #133

I recently read a book by Google’s head guy on API design that was specifically about designing APIs and it had a big section on what makes a good identifier and why people reach for UUIDs and why specifically it is a problem on multiple levels. The thing that he ended up recommending however was super interesting in that I had never seen it mentioned before but it was basically to use this instead http://www.crockfo…

This format was discussed in a HN first page post just this week: > This alphabet, 0123456789ABCDEFGHJKMNPQRSTVWXYZ, is Douglas Crockford's Base32, chosen for human readability and being able to call it out over a phone if required. https://news.ycombinator.com/item?id=29794186

Looking for that quote I can't find it on that page.

Re: UUIDs are popular, but bad for performance (2019)

#212

Post by anamexis »

rco8786 wrote: post #196
inetknght wrote: post #194

Earlier quoted context omitted.

I never understood why people would use sequential UUIDs. That rather defeats the purpose of a UUID. If you need something sequential then just use a much more simple number

Because simple integers are not universally unique, a major feature of UUIDs…

UUIDs are just 128-bit simple integers.

Re: UUIDs are popular, but bad for performance (2019)

#213

Post by lelandbatey »

mhoad wrote: post #133

I recently read a book by Google’s head guy on API design that was specifically about designing APIs and it had a big section on what makes a good identifier and why people reach for UUIDs and why specifically it is a problem on multiple levels. The thing that he ended up recommending however was super interesting in that I had never seen it mentioned before but it was basically to use this instead http://www.crockfo…

I'm a big fan of Crockford-flavored Base32, as in $dayjob the folks who did some of the fundamental engineering work for the product I work with decided 10+ years ago to use Crockfor-flavored Base32 as the way to expose most user-visible IDs. Things I love about it are all mentioned in Crockford's spec, but I'll restate them here:

- It's case-insensitive allowing data encoded with it to survive going through most random line-of-business applications which may have errant UPPER() or lower() calls somewhere in their depths, as well as making it easy for humans to talk about out-loud (no need to specify case makes it easy)

- Making all the "pillar" shaped characters (I, i, 1, l) equivalent and all the "donut" shaped characters (o, O, 0) equivalent means big swaths of human typing mistakes are avoided. After so much exposure to Crockford Base32, I now loath having to decipher "is it a capitol 'I'? Or a lowercase 'l'?"

Overall, it's a great way to encode any user-facing ID; just make sure you understand that just using this encoding won't stop users from realizing that Crockford Base32 encoded IDs are sequential. If you want sequential IDs to appear non-obviously sequential after encoding you'll need to use additional techniques. Assuming Crockford Base32 obscures sequential IDs is the one case where I saw someone do something they shouldn't as it directly relates to Crockford Base32.

Re: UUIDs are popular, but bad for performance (2019)

#214

Post by staticassertion »

dpark wrote: post #199
staticassertion wrote: post #167

Earlier quoted context omitted.

The linked article discusses the issues - the storage size when encoded as char, as 16byte values, as well as the impact on read/write locality.

Where does it discuss any of this? It compares base 10/16/32/64. There’s no mention at all of UUIDs or GUIDs. (Or do you mean the original article, rather than the linked Crockford article?)

I mean the original.

Re: UUIDs are popular, but bad for performance (2019)

#215

Post by withinboredom »

isoprophlex wrote: post #114
kyrra wrote: post #108

Earlier quoted context omitted.

Your other option is to hash your monotonically increasing numbers. You don't want to use something like SHA, as it does not give good distribution. You use something like murmur hash ( https://en.m.wikipedia.org/wiki/MurmurHash ). I've used it before for indexes at Google for values that may hotspot. See Google's impl here: https://github.com/google/zetasketch/blob/master/java/com/go...

Can you elaborate on the claim about SHA? If I'm not mistaken, all these cryptographic hashes have a 50% chance of flipping any one output bit upon changing a single input bit -- is there some hidden gotcha I'm unaware of?

It’s about random distribution. If you output an array of the key space as a bitmap, it should look like white noise. SHA doesn’t look like white noise. There are certain parts of the space it rarely goes with certain classes of inputs (numeric, usernames, etc) while murmur and md5 give that white noise look.

Re: UUIDs are popular, but bad for performance (2019)

#216

Post by withinboredom »

datavirtue wrote: post #155
JimDabell wrote: post #95

Earlier quoted context omitted.

> You can't just expect a relational database to magically become a distributed system just by using UUIDs. Nobody thinks this, do they?

Somebody thought RabbitMQ did load balancing because it had three redundant nodes. Pretty much ruined the company. Definitely caused all of us to lose our equity.

I’d love to hear that story.

Re: UUIDs are popular, but bad for performance (2019)

#217

Post by relueeuler »

wbl wrote: post #152
mhoad wrote: post #133

I recently read a book by Google’s head guy on API design that was specifically about designing APIs and it had a big section on what makes a good identifier and why people reach for UUIDs and why specifically it is a problem on multiple levels. The thing that he ended up recommending however was super interesting in that I had never seen it mentioned before but it was basically to use this instead http://www.crockfo…

Would you happen to have a link to the book?

https://www.amazon.com/API-Design-Patterns-JJ-Geewax/dp/1617...

Re: UUIDs are popular, but bad for performance (2019)

#218

Post by kune »

People should have a look as k-sortable unique identifiers (KSUID). Binary they are represented by 20 bytes and their string representation has 27 characters, which is shorter than UUIDs since the use a base62 encoding. They are sortable since the 20 bytes start with a 32 bit UNIX timestamp followed by random 128 bits. They should be very efficient for clustered indexes / B+-Trees.

Note also that as long as you have a single central database you don't need UUIDs. They are only needed if you have several processes creating objects without coordination.

Re: UUIDs are popular, but bad for performance (2019)

#219

Post by stickfigure »

luhn wrote: post #188
stickfigure wrote: post #156

Earlier quoted context omitted.

Firestore has exactly the opposite problem - indexed sequential values put pressure on the "last tablet" and require the same tablet to repeatedly split. This becomes the limiting factor on insert volume. Random generated keys are better, because they spread inserts across multiple tablets (ie servers). I don't know for certain, but I suspect DynamoDB and most other databases that can trace their origins to the bigta…

DynamoDB runs the partition key through a hash function, so sequential values end up being evenly distributed across partitions.

If so, how does it iterate in key order?

I should clarify - the problem with Firestore is the index tablets. The indexes inherently need to be inorder or you can't perform ordered queries.

Re: UUIDs are popular, but bad for performance (2019)

#220

Post by oldsecondhand »

cperciva wrote: post #204
et-al wrote: post #179

Earlier quoted context omitted.

If you're worried about clarity over the phone, you should look into the NATO phonetic alphabet: https://en.wikipedia.org/wiki/NATO_phonetic_alphabet

I prefer to use Aeon, Bdellium, Czar, Djinn, Eye, etc.

> Djinn

Fun fact: dzs counts as a single letter in Hungarian (e.g. in alphabetical ordering).

Post reply on HN