Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New UUID Formats

ietf.org

Post reply on HN
Print view View original

111–120 of 172 posts

Re: New UUID Formats

#111

Post by tehlike »

evil-olive wrote: post #100
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

my personal favorite UUID replacement, when not concerned with external compatibility or standards-compliance: a 96-bit random value, base64-urlsafe encoded to 16 ASCII characters >>> import secrets >>> secrets.token_urlsafe(12) 'uUBpBk2eENDslHyw' with UUID, you can store it as a compact 16 bytes in a database, but it needs 36 characters if you want to embed it in a URL or a JSON payload. and there's a temptation to…

uuid is just what you said - 16bytes. you could presumably base64 that too...

Re: New UUID Formats

#112

Post by jandrewrogers »

Merad wrote: post #95
cameronh90 wrote: post #35

Earlier quoted context omitted.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

> UUIDs have a few distinct advantages: you'll never run out I'm curious what kind of applications are limited by the range of bigint values? I have no doubt that such applications exist somewhere, but most software engineers won't ever come close to encountering those limits. Even if you have a table that is consistently consuming a billion (with a B) bigint id values _every second_ (is that even feasible with curre…

Some famous RDBMS bugs had their root cause in this kind of reasoning, because the model embeds assumptions on how those IDs will be generated and used which may not hold true in the future for reasons that are difficult to anticipate.

For example, Oracle had a 48-bit ID rollover bug many years ago that by all calculations should never occur in real systems. This calculation was made under the assumption that the IDs were mostly actually used. However, many features added later necessitated generating or reserving vast numbers of IDs in bulk, a low-cost optimization, the vast majority of which were ultimately discarded. It got to the point where very large systems started running out of these IDs due to the fact that such a low percentage were used in the way the designers had anticipated.

Extremely large systems do not run into the limitations of bigint because at that scale the identifiers are naturally segmented, often implicitly.

Re: New UUID Formats

#113

Post by smallnamespace »

blueflow wrote: post #87
dtech wrote: post #71

Earlier quoted context omitted.

On the other hand, this is an easy to implement conversion, while changing such a fundamental thing from EFI sounds pretty hard, making it not worth it.

You cannot fix this with an conversion because you do not know if your UUID is correct or needs conversion. DMI data for example has inconsistent endian-ness depending on the vendor. So if you have a UUID sticker on a new server, you still have two options which UUID the machine will send during PXE, either the printed UUID in big-endian encoding or in the microsoft mixed-endian encoding. Use BIOS boot instead of EFI…

Isn’t endianness marked by the variant field? 1 is IETF, 2 is Microsoft, and this can be inferred from the sticker.

Re: New UUID Formats

#114

Post by jamesrom »

Zamicol wrote: post #25

Why isn't there an option with a strong cryptographic hash like SHA-256?

You could utilize UUIDv8 for this purpose.

Of course v5 isn't strong in the cryptographic sense, but probably suits most purposes where you need a unique ID per an object: https://datatracker.ietf.org/doc/html/rfc4122#section-4.3

The idea of using UUIDs for cryptographic purposes is probably misguided anyway.

Re: New UUID Formats

#115

Post by feoren »

cameronh90 wrote: post #35
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

> if you ever need to combine multiple data sources together in migration and recovery type scenarios

This insane idea that combining data sources is a rare event in some unusual "migration and recovery" scenarios is one of the most poisonous and yet pervasive ideas in all of database design. You are always combining multiple data sources, all the time. Users submitting data from a form is a data source. Test, staging, and production deployments, with multiple of each. External APIs. Multiple clients. Eventual consistency. Replication. Microservices with distributed systems. Or even sharing any common data at all between different systems, like unit conversions, chemical data, country names, engineering constants, etc.

Anyone who even considers using a single authoritative source for all entity identity either better be making a system in an underground bunker that will never talk to any other system. Otherwise they are making a serious and extremely avoidable mistake. Never use auto-incrementing IDs.

It's even wrong in a monolith! Why does everyone abandon this idea of "separation of concerns" and "single responsibility principle" and "bounded contexts" and proper abstraction and limited communication between system parts and literally every design principle they've ever been taught when they go to design a database? It all just goes out the window! Why do you guys bother reading books about system design if you ignore them when you build a database? "Multiple systems communicating with each other" should apply recursively all the way from deployment and external integration down to individual functions. That means database, too. Auto-incrementing IDs are anathema to that.

Re: New UUID Formats

#116

Post by selcuka »

inopinatus wrote: post #110
selcuka wrote: post #96

Earlier quoted context omitted.

You may consider encoding the uuid in base58 as it is shorter, safe to use in URLs, and easier to debug as it doesn't use characters that can be confused such as 0, 0, I, and l.

I'll add a voice for base58 representation of UUIDs on the wire, it's my preferred solution as well. Not only for debugging: by being compact, unambiguous, and URL-safe, the helpdesk is also spared a cringeworthy source of PEBCAK incidents since misquoting the identifiers is simply harder. Caveat programmer, though, there is a hazard, occurring when someone is glib about the usage and relies on randomly generated fix…

If I'm not mistaken base64 should not cause an integer overflow (i.e. it will always be a 128-bit integer) if you limit it to 22 characters (and append two = characters to pad it when you actually decode it).

It may not be as URL-safe as base58 though.

Re: New UUID Formats

#117

Post by Daegalus »

I maintain the Dart UUID library. For anyone using dart, or wants to see one of many implementations of v6,v7 and a custom v8 UUID, feel free to look at my in-progress branch linked below, I plan to merge it in once they add different string representations in a future draft (I've been involved in the conversations).

https://github.com/Daegalus/dart-uuid/tree/uuidv6

Re: New UUID Formats

#118

Post by lewisl9029 »

djbusby wrote: post #103
lewisl9029 wrote: post #75

I've been using ULID for a while now, which analogous to UUID v7 but with a different (better IMHO) string representation. They've been awesome for using as sort keys in dynamo for instance, since they're lexicographically sortable as strings. But one thing I'm still wary about is exposing these IDs with millisecond-precision time components to end users, since I've seen multiple discussions here on HN about the pote…

ULID has like 80 bits of random per millisecond. I found some calculator online that showed the probability of collision on 80bits was very, very low even across time much longer than millisecond. I think knowing when an ID was created is harmless - and if it isn't there is a design problem that is larger than ID choice. There is another lib called hashid which can be used if masking that ID algo is important.

> I think knowing when an ID was created is harmless - and if it isn't there is a design problem that is larger than ID choice.

I hope you're right, since I've been cautiously operating under that assumption so far. Cautiously, because of warnings, from people better versed in cryptography than I, in discussions like this one: https://news.ycombinator.com/item?id=29805433. I still haven't quite been able to internalize when this should vs shouldn't be something I should be concerned about, hence the comment.

Re: New UUID Formats

#119

Post by inopinatus »

selcuka wrote: post #116
inopinatus wrote: post #110

Earlier quoted context omitted.

I'll add a voice for base58 representation of UUIDs on the wire, it's my preferred solution as well. Not only for debugging: by being compact, unambiguous, and URL-safe, the helpdesk is also spared a cringeworthy source of PEBCAK incidents since misquoting the identifiers is simply harder. Caveat programmer, though, there is a hazard, occurring when someone is glib about the usage and relies on randomly generated fix…

If I'm not mistaken base64 should not cause an integer overflow (i.e. it will always be a 128-bit integer) if you limit it to 22 characters (and append two = characters to pad it when you actually decode it). It may not be as URL-safe as base58 though.

Yes. I think of base64 as an unconventional encoding, in that regard, since it's more than just an alphabet for powers of the base but also communicates properties of the output octet length in the serialized form. To do that it's relying on an elegant alignment of bit values to that base58 doesn't have. Perhaps there could be a means to inject the same confluence of form and function into base58, albeit I can't see it after a couple minutes of staring at the algorithm¹.

And also yes: the equals sign is potentially hazardous in a query string, and the use of non-alphanumeric symbols in base64 also creates line-break and double-click/double-tap traps when passed around in an ad-hoc fashion, even in the url-safe variant.

[1] https://datatracker.ietf.org/doc/html/draft-msporny-base58

Re: New UUID Formats

#120

Post by oofbey »

I really don’t understand why we need standards for UUIDs. I get that with cryptography it’s super easy to make subtle mistakes. But UUID4 (the most commonly used variant these days) is just a long random number. Except for a few bits which aren’t random because the standard says so. As long as you’re using a goodsource of randomness (most are these days) it’s pretty hard to screw up. Why do we need a standard to tell us how to combine time stamps and random numbers?
Post reply on HN