Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New UUID Formats

ietf.org

Post reply on HN
Print view View original

101–110 of 172 posts

Re: New UUID Formats

#101

Post by RhodesianHunter »

cameronh90 wrote: post #35
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

Something like Snowflake ids has many of these advantages while being integers.

Re: New UUID Formats

#102

Post by iroddis »

chrismorgan wrote: post #33
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

I’ve been working on a robust scheme for encrypted sequential IDs, which is done, including library implementations in Rust, JavaScript and Python, pending just a smidgeon more writing about it and reviewing a decision on naming. You store an integer in the database, then encrypt it with a real block cipher, and stringify with Base58. I have three modes: one for 32-bit IDs, using Speck32/64 and producing 4–6 characte…

> UUIDs are crazy overkill in any situation where you can have centralised ID allocation.

Except that’s specifically the use case of UUIDs: to have a decentralized method to generate unique IDs with minimal chance of collisions. If you have centralized control, of course there will be options with more attractive properties: they aren’t dealing with the same constraints.

Re: New UUID Formats

#103

Post by djbusby »

lewisl9029 wrote: post #75

I've been using ULID for a while now, which analogous to UUID v7 but with a different (better IMHO) string representation. They've been awesome for using as sort keys in dynamo for instance, since they're lexicographically sortable as strings. But one thing I'm still wary about is exposing these IDs with millisecond-precision time components to end users, since I've seen multiple discussions here on HN about the pote…

ULID has like 80 bits of random per millisecond. I found some calculator online that showed the probability of collision on 80bits was very, very low even across time much longer than millisecond. I think knowing when an ID was created is harmless - and if it isn't there is a design problem that is larger than ID choice.

There is another lib called hashid which can be used if masking that ID algo is important.

Re: New UUID Formats

#104

Post by rossriley »

tqkxzugoaupvwqr wrote: post #98

Is there a way to verify a UUID was created by me? I want to avoid database lookups of UUIDs from malicious/manipulated URLs.

You could use the Luhn algorithm or something similar to add a check digit to the end, so for instance, generate a standard uuid, remove the last character, pass the remaining characters through the Luhn function and then use the result as the final replacement character for the UUID.

It's obviously not cryptographically secure, as in someone else could use the same algorithm to generate UUIDs that your system recognises, but could be a quick way of doing a verification before you hit the DB.

Re: New UUID Formats

#105

Post by [deleted] »

selcuka wrote: post #96
CharlesW wrote: post #46

Earlier quoted context omitted.

> There are a few compact representations you can use in URLs which make it a bit less ugly… Any thoughts on where to find best-practices guidance? I need to create an external ID scheme for several million items. hashids (hashids.org) seems interesting, but I have anxiety about choosing a solution with weaknesses that I can't identify given my current level of experience in regards to this.

You may consider encoding the uuid in base58 as it is shorter, safe to use in URLs, and easier to debug as it doesn't use characters that can be confused such as 0, 0, I, and l.

[deleted]

Re: New UUID Formats

#106

Post by joshspankit »

When I read the headline I thought “why do we need another UUID format?” but my respect was quickly earned.

A lot of excellent thought put in to this, and thank you to the authors for casting aside the abomination called leap seconds.

Re: New UUID Formats

#107

Post by Gigachad »

Merad wrote: post #95
cameronh90 wrote: post #35

Earlier quoted context omitted.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

> UUIDs have a few distinct advantages: you'll never run out I'm curious what kind of applications are limited by the range of bigint values? I have no doubt that such applications exist somewhere, but most software engineers won't ever come close to encountering those limits. Even if you have a table that is consistently consuming a billion (with a B) bigint id values _every second_ (is that even feasible with curre…

For some reason a lot of things used to default to 32 bit primary keys and you absolutely did run out of those and its been a major issue for a lot of apps for the last few years. But yeah, you'll never run out of bigints SQL servers couldn't cope with the amount of data that would require.

Re: New UUID Formats

#108

Post by kjmph »

canadiantim wrote: post #50

How would one go about trying to use UUID 7 in a Postgres database / python codebase now?

I created a PL/pgSQL function for this purpose. As Starlevel001 mentioned, the UUID format will accept opaque bytes. Perhaps you could find this useful?

https://gist.github.com/kjmph/5bd772b2c2df145aa645b837da7eca...

Re: New UUID Formats

#109

Post by thayne »

cameronh90 wrote: post #35
GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

Also, using an incrementing id can lead to to information disclosure if they are visible in user facing APIs. For example, if I know an id for my user, document, cart, etc. Then I know all ids lower than that are probably also valid.

Re: New UUID Formats

#110

Post by inopinatus »

selcuka wrote: post #96
CharlesW wrote: post #46

Earlier quoted context omitted.

> There are a few compact representations you can use in URLs which make it a bit less ugly… Any thoughts on where to find best-practices guidance? I need to create an external ID scheme for several million items. hashids (hashids.org) seems interesting, but I have anxiety about choosing a solution with weaknesses that I can't identify given my current level of experience in regards to this.

You may consider encoding the uuid in base58 as it is shorter, safe to use in URLs, and easier to debug as it doesn't use characters that can be confused such as 0, 0, I, and l.

I'll add a voice for base58 representation of UUIDs on the wire, it's my preferred solution as well.

Not only for debugging: by being compact, unambiguous, and URL-safe, the helpdesk is also spared a cringeworthy source of PEBCAK incidents since misquoting the identifiers is simply harder.

Caveat programmer, though, there is a hazard, occurring when someone is glib about the usage and relies on randomly generated fixed-length base58 values directly. It happens readily because some popular frameworks include such generation as a utility function. However, 58^22 > 2^128 > 58^21, so a 22-character base58 representation is expected for UUIDs but carelessly random 22-character base58 values may exceed the capacity of UUID's familiar hexadecimal serialization, effectively an integer overflow. We never generate base58 identifiers as a PK, for example, for this reason (they would of course not be UUIDs either). Alas, there is no conventional base encoding more compact than hexadecimal that is robust to the general problem. And I mention it because this issue was observed in the wild.

Post reply on HN