Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New UUID Formats

ietf.org

Post reply on HN
Print view View original

81–90 of 172 posts

Re: New UUID Formats

#81

Post by doliveira »

lewisl9029 wrote: post #75

I've been using ULID for a while now, which analogous to UUID v7 but with a different (better IMHO) string representation. They've been awesome for using as sort keys in dynamo for instance, since they're lexicographically sortable as strings. But one thing I'm still wary about is exposing these IDs with millisecond-precision time components to end users, since I've seen multiple discussions here on HN about the pote…

I came up with a scheme in which the random part of the ULID is a slice of the hash of a UUID, which seems to work fine because I'm generating it all server side, I guess? It works very well for insertion into NoSQL databases, for instance.

So I'd also like to know the threat model for these timing attacks.

Re: New UUID Formats

#82

Post by jxcole »

zbuf wrote: post #39
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

"Unique IDs" _can_ be super really easy to work with if they're not so baffling complicated. A random string generated using quality randomness can be adjusted to length to suit the quantity of data (negligible probability of a collision) which in most cases is very short. It's easy to increase the length as you get more data. They are visually very different for each item of data. They're evenly spread which means t…

We used almost this exact scheme for app id indices and the curious problem we had to design against was inadvertent profanity. At some point we decided to just never use vowels to avoid ever having a complaint about 12f*ck if in the URL

Re: New UUID Formats

#83

Post by amluto »

firebird84 wrote: post #45
amluto wrote: post #19

Fortunately this is a bit less relevant today as Windows loses market share in database and server applications, but: UUIDs have historically massively screwed up endian handling. While this new draft discusses sorting UUIDs as strings of octets (bytes) and the text of RFC4122 is fairly explicit about most significant bytes coming first, the C UUID structure in RFC 4122 appendix A is entirely misguided: typedef struc…

It's actually worse than that. The first 3 groupings (textually) of the uuid might be little endian while the other 2 are big endian. Learning this cost me more time than I care to admit. https://en.wikipedia.org/wiki/Universally_unique_identifier#...

This is consistent with the misguided structure in the RFC. The first three fields (the time fields) are multibyte integers. The remainder is just bytes. The dashes in the textual representation are just there to confuse you.

Re: New UUID Formats

#84

Post by RobertRoberts »

jeremyjh wrote: post #48
RobertRoberts wrote: post #41

Earlier quoted context omitted.

Yep, I want UUID v7, because right now I am using ULID and it's fantastic, but I'd like more official and wide support as well.

I like the textual representation of ULIDs as well though; I wish they’d just adopted ULID as v7. At least it is binary compatible with existing UUID types.

Is there not an equivalent text representation for UUIDv7?

Re: New UUID Formats

#85

Post by jasonwatkinspdx »

jandrewrogers wrote: post #73
jasonwatkinspdx wrote: post #55

Earlier quoted context omitted.

Using purely random ids in your database destroys locality. They mention this in the introduction: > Non-time-ordered UUID versions such as UUIDv4 have poor database index locality. Meaning new values created in succession are not close to each other in the index and thus require inserts to be performed at random locations. The negative performance effects of which on common structures used for this (B-tree and its v…

100 billion UUIDs per millisecond is the 50% collision probability threshold. Achieving an acceptable collision probability for most applications would limit the UUID generation rate to more like thousands of UUIDs per millisecond. Even if one was not generating millions of UUIDs per second on average, the risk of spiky temporal distributions when generating UUIDs would still need to be considered.

I'm aware of what the bound I quoted is, and am just too lazy to type the refinement into Wolfram for a discussion like this. But just knowing the value off the top of my head for 64 bits is 200k for 1e-9 probability of collision, I'm pretty happy with 74 bits.

And although this standard obviously wants to stick within the existing UUID footprint, if you were say doing some IoT software that would run on billions of nodes simultaneously, just add another 32/64/whatever bits of random data and deal with the minor annoyance of longer ids and lack of UUID RFC compatibility. But even then, you can truncate and reformat these these to v4 UUIDs trivially without meaningfully impacting the collision resistance, for unsorted external ids in systems that need the compatability.

The actual big risk with this sort of scheme is vm initialization. You need to be sure the CSPRNG you're using is initialized, which can be slightly tricky in cloud environments with configuration/control layer stuff that's racy. Mess this up and two nodes hydrated from the same snapshot may overlap in sequence as they start generating ids, and of course the time component cannot be trusted to save you in this instance.

Re: New UUID Formats

#86

Post by nixpulvis »

okr wrote: post #78
nixpulvis wrote: post #66

Maybe I'm being slow right now, but can somewhat help me understand why Max UUID is ever specifically useful?

My guess is, that it is just a constant, ready to be used for bitwise operations.

Yep... I was just being slow. I thought the specialized variant was a new kind of UUID (being inverse of RFC4122), but it's just the inverse of RFC4122's Nil UUID; a single value.

Sorry for the silly comment. This value is just a bunch of binary 1s.

Re: New UUID Formats

#87

Post by blueflow »

dtech wrote: post #71
blueflow wrote: post #24

Earlier quoted context omitted.

This mishap lives forth in the UUID stored in a machines DMI data, as well in GPT partition tables, which are required when EFI is used. It would be really cool if we had some replacement for EFI that would not harbour these kind of painful legacies.

On the other hand, this is an easy to implement conversion, while changing such a fundamental thing from EFI sounds pretty hard, making it not worth it.

You cannot fix this with an conversion because you do not know if your UUID is correct or needs conversion. DMI data for example has inconsistent endian-ness depending on the vendor. So if you have a UUID sticker on a new server, you still have two options which UUID the machine will send during PXE, either the printed UUID in big-endian encoding or in the microsoft mixed-endian encoding.

Use BIOS boot instead of EFI, it has less legacy to implement: PE executables, FAT file system, Win64 ABI

Re: New UUID Formats

#89

Post by jasonwatkinspdx »

zbuf wrote: post #67
jasonwatkinspdx wrote: post #55

Earlier quoted context omitted.

Using purely random ids in your database destroys locality. They mention this in the introduction: > Non-time-ordered UUID versions such as UUIDv4 have poor database index locality. Meaning new values created in succession are not close to each other in the index and thus require inserts to be performed at random locations. The negative performance effects of which on common structures used for this (B-tree and its v…

Thanks for drawing my attention to that, it's the useful answer I was looking for; my use cases haven't been bound by write performance in this manner. However, I'd still be considering carefully before making use of these UUID schemes.

If you want something like this, and need a "I just want it to work, require no central coordination, and to have vanishingly small probability of collision" then just use the same concepts but wider than the 128 bit footprint limit of this scheme. This limit makes sense for the RFC in the post, as there backwards compatibility is an explicit goal. But if you used say a 64 bit nanosecond counter (to preserve best case precision on a single machine) along with 128+ bits of random data and you'll need to worry more about gamma ray bursts than collisions.

Re: New UUID Formats

#90

Post by jasonwatkinspdx »

jxcole wrote: post #82
zbuf wrote: post #39

Earlier quoted context omitted.

"Unique IDs" _can_ be super really easy to work with if they're not so baffling complicated. A random string generated using quality randomness can be adjusted to length to suit the quantity of data (negligible probability of a collision) which in most cases is very short. It's easy to increase the length as you get more data. They are visually very different for each item of data. They're evenly spread which means t…

We used almost this exact scheme for app id indices and the curious problem we had to design against was inadvertent profanity. At some point we decided to just never use vowels to avoid ever having a complaint about 12f*ck if in the URL

Another approach is to use something like EFF's dice words lists. One of the smaller lists in particular is interesting as it's 6^4 words, filtered for profanity, and where all words have both a unique 3 letter prefix and an edit distance of 3. That makes them robust for the use case of someone reading out the phrase to someone typing or such.

Never using vowells is a smart idea I wish I'd used in the past. Previously when I've needed something like this I've used other dictionary lists vs EFF's, and those were not curated sufficiently to avoid some really unfortunate combinations.

Post reply on HN