Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New UUID Formats

ietf.org

Post reply on HN
Print view View original

61–70 of 172 posts

Re: New UUID Formats

#61

Post by kevincox »

Someone wrote: post #44
amluto wrote: post #19

Fortunately this is a bit less relevant today as Windows loses market share in database and server applications, but: UUIDs have historically massively screwed up endian handling. While this new draft discusses sorting UUIDs as strings of octets (bytes) and the text of RFC4122 is fairly explicit about most significant bytes coming first, the C UUID structure in RFC 4122 appendix A is entirely misguided: typedef struc…

> Hint: do not use integer types in C code for portable data structures. ntohl, etc are a mess. Just use arrays of bytes. I don’t see how that helps much. If a developer forgets to call ntohl on multi-byte integer fields, I don’t trust them to correctly convert said integers to arrays of bytes, either.

If you write it the naive way it works.

    uint8_t bytes[4];
    uint32_t = bytes[0]
The endianness is whatever you write in the indexing and will be the same across architectures.

Re: New UUID Formats

#62

Post by kortex »

GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

> It's much easier to remember/recognize an integer based PK when troubleshooting a data problem.

How often are you actually relying on memory of an ID to troubleshoot a problem? I mean sure, if you are scanning visually, it's good to recognize the same ID over and over again, but my ability to do so caps around 4-6 characters. So I just look at the last 4 chars regardless when fast scanning.

I use copy-paste for any time I need to transport IDs between contexts (that isn't just scripted, which is best). Having a copy-paste stack (Alfred, Raycast and others have this feature) is a huge game changer here.

Re: New UUID Formats

#63

Post by jwilk »

staticassertion wrote: post #28
Zamicol wrote: post #25

Why isn't there an option with a strong cryptographic hash like SHA-256?

It would be slow and doesn't really serve a purpose since you either have uuids that are totally random or uuids that need to preserve their structure.

Or UUIDs derived from other identifiers:

https://datatracker.ietf.org/doc/html/rfc4122#section-4.3

Re: New UUID Formats

#64

Post by [deleted] »

quotemstr wrote: post #51
amluto wrote: post #19

Fortunately this is a bit less relevant today as Windows loses market share in database and server applications, but: UUIDs have historically massively screwed up endian handling. While this new draft discusses sorting UUIDs as strings of octets (bytes) and the text of RFC4122 is fairly explicit about most significant bytes coming first, the C UUID structure in RFC 4122 appendix A is entirely misguided: typedef struc…

Little endian won. I don't see the point of maintaining theoretical compatibility with big endian systems that are at best esoteric right now and soon to be extinct. Likewise, every reasonable platform aligns data fields on natural alignment these days. It's just a waste of effort to make software portable to evolutionary dead ends.

[deleted]

Re: New UUID Formats

#65

Post by munawwar »

CharlesW wrote: post #46
cameronh90 wrote: post #35

Earlier quoted context omitted.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

> There are a few compact representations you can use in URLs which make it a bit less ugly… Any thoughts on where to find best-practices guidance? I need to create an external ID scheme for several million items. hashids (hashids.org) seems interesting, but I have anxiety about choosing a solution with weaknesses that I can't identify given my current level of experience in regards to this.

I took a shot at the math behind this at https://www.codepasta.com/databases/2020/09/10/shorter-uniqu...

Using the equation listed in the article I couldn't generate a collision so far. Yet, I still check (in code) for id collision, and pick new id, just to be 100% sure.

Re: New UUID Formats

#67

Post by zbuf »

jasonwatkinspdx wrote: post #55
zbuf wrote: post #39

Earlier quoted context omitted.

"Unique IDs" _can_ be super really easy to work with if they're not so baffling complicated. A random string generated using quality randomness can be adjusted to length to suit the quantity of data (negligible probability of a collision) which in most cases is very short. It's easy to increase the length as you get more data. They are visually very different for each item of data. They're evenly spread which means t…

Using purely random ids in your database destroys locality. They mention this in the introduction: > Non-time-ordered UUID versions such as UUIDv4 have poor database index locality. Meaning new values created in succession are not close to each other in the index and thus require inserts to be performed at random locations. The negative performance effects of which on common structures used for this (B-tree and its v…

Thanks for drawing my attention to that, it's the useful answer I was looking for; my use cases haven't been bound by write performance in this manner. However, I'd still be considering carefully before making use of these UUID schemes.

Re: New UUID Formats

#68

Post by ntauthority »

mort96 wrote: post #57
quotemstr wrote: post #51

Earlier quoted context omitted.

Little endian won. I don't see the point of maintaining theoretical compatibility with big endian systems that are at best esoteric right now and soon to be extinct. Likewise, every reasonable platform aligns data fields on natural alignment these days. It's just a waste of effort to make software portable to evolutionary dead ends.

This isn't about compatibility with big-endian machines. This is about compatibility between different uuid libraries, potentially on different operating systems, all on little endian CPU architectures.

Every existing library follows the standard which specifies host byte order, which usually means little endian. The Rust library cited a few levels up this chain ignored that, somehow assuming big endian, and then had to correct for this mistake.

Re: New UUID Formats

#69

Post by [deleted] »

quotemstr wrote: post #51
amluto wrote: post #19

Fortunately this is a bit less relevant today as Windows loses market share in database and server applications, but: UUIDs have historically massively screwed up endian handling. While this new draft discusses sorting UUIDs as strings of octets (bytes) and the text of RFC4122 is fairly explicit about most significant bytes coming first, the C UUID structure in RFC 4122 appendix A is entirely misguided: typedef struc…

Little endian won. I don't see the point of maintaining theoretical compatibility with big endian systems that are at best esoteric right now and soon to be extinct. Likewise, every reasonable platform aligns data fields on natural alignment these days. It's just a waste of effort to make software portable to evolutionary dead ends.

[deleted]

Re: New UUID Formats

#70

Post by mappu »

chrismorgan wrote: post #33
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

I’ve been working on a robust scheme for encrypted sequential IDs, which is done, including library implementations in Rust, JavaScript and Python, pending just a smidgeon more writing about it and reviewing a decision on naming. You store an integer in the database, then encrypt it with a real block cipher, and stringify with Base58. I have three modes: one for 32-bit IDs, using Speck32/64 and producing 4–6 characte…

I've done something similar to obfuscate private DB IDs in a large existing application - just ensure they're all Skip32-encoded in all query parameters with an app-wide secret.

It works well but you have to be very disciplined to catch every case individually. Using GUID PKs from the start just removes this entire category of problem.

Post reply on HN