Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

New UUID Formats

ietf.org

Post reply on HN
Print view View original

121–130 of 172 posts

Re: New UUID Formats

#121

Post by chrismorgan »

iroddis wrote: post #102
chrismorgan wrote: post #33

Earlier quoted context omitted.

I’ve been working on a robust scheme for encrypted sequential IDs, which is done, including library implementations in Rust, JavaScript and Python, pending just a smidgeon more writing about it and reviewing a decision on naming. You store an integer in the database, then encrypt it with a real block cipher, and stringify with Base58. I have three modes: one for 32-bit IDs, using Speck32/64 and producing 4–6 characte…

> UUIDs are crazy overkill in any situation where you can have centralised ID allocation. Except that’s specifically the use case of UUIDs: to have a decentralized method to generate unique IDs with minimal chance of collisions. If you have centralized control, of course there will be options with more attractive properties: they aren’t dealing with the same constraints.

Sure, decentralised allocation is the intent of UUIDs, but in practice UUIDv4 is used extremely widely where it’s completely unnecessary, because it’s a convenient way of generating non-sequential IDs with good tooling support (UUID libraries, UUID data types in some databases, &c.).

Re: New UUID Formats

#122

Post by chrismorgan »

mappu wrote: post #70
chrismorgan wrote: post #33

Earlier quoted context omitted.

I’ve been working on a robust scheme for encrypted sequential IDs, which is done, including library implementations in Rust, JavaScript and Python, pending just a smidgeon more writing about it and reviewing a decision on naming. You store an integer in the database, then encrypt it with a real block cipher, and stringify with Base58. I have three modes: one for 32-bit IDs, using Speck32/64 and producing 4–6 characte…

I've done something similar to obfuscate private DB IDs in a large existing application - just ensure they're all Skip32-encoded in all query parameters with an app-wide secret. It works well but you have to be very disciplined to catch every case individually. Using GUID PKs from the start just removes this entire category of problem.

If it requires discipline, you’re doing it wrong. For best results, it should be threaded through your ORM or equivalent so that you can’t do it wrong, that at your app layer you never get raw IDs.

Re: New UUID Formats

#123

Post by staticassertion »

jandrewrogers wrote: post #112
Merad wrote: post #95

Earlier quoted context omitted.

> UUIDs have a few distinct advantages: you'll never run out I'm curious what kind of applications are limited by the range of bigint values? I have no doubt that such applications exist somewhere, but most software engineers won't ever come close to encountering those limits. Even if you have a table that is consistently consuming a billion (with a B) bigint id values _every second_ (is that even feasible with curre…

Some famous RDBMS bugs had their root cause in this kind of reasoning, because the model embeds assumptions on how those IDs will be generated and used which may not hold true in the future for reasons that are difficult to anticipate. For example, Oracle had a 48-bit ID rollover bug many years ago that by all calculations should never occur in real systems. This calculation was made under the assumption that the IDs…

> Extremely large systems do not run into the limitations of bigint because at that scale the identifiers are naturally segmented, often implicitly.

Agreed. That's the real key imo. You could have a 2^64 random number as a primary key if that key is also namespaced per customer - maybe in aggregate your customers generate > 2^32 events but a given customer may not. And there's other ways to limit it further.

This is the approach we take, basically, although we use a counter and not a random number.

Re: New UUID Formats

#124

Post by unixhero »

GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

Useful to have a unique UIID too, but integers are better as PK I agree.

Re: New UUID Formats

#125

Post by swyx »

oofbey wrote: post #120

I really don’t understand why we need standards for UUIDs. I get that with cryptography it’s super easy to make subtle mistakes. But UUID4 (the most commonly used variant these days) is just a long random number. Except for a few bits which aren’t random because the standard says so. As long as you’re using a goodsource of randomness (most are these days) it’s pretty hard to screw up. Why do we need a standard to tel…

thats what the authors of uuid4 thought. but if you read the RFC again you’ll see they listed out the flaws and pain points. eg the impact of true randomness on database writes at scale.

having standards encodes lessons from a decade worth of pain. don’t dismiss it so casually.

Re: New UUID Formats

#126

Post by bruce511 »

feoren wrote: post #115
cameronh90 wrote: post #35

Earlier quoted context omitted.

UUIDs have a few distinct advantages: you'll never run out, you don't need a roundtrip to find out what they are after saving them, they often make a good partitioning key and it makes things easier if you ever need to combine multiple data sources together in migration and recovery type scenarios. I also quite like how they're unique across all data sources and tables, so if you just encounter a random contextless U…

> if you ever need to combine multiple data sources together in migration and recovery type scenarios This insane idea that combining data sources is a rare event in some unusual "migration and recovery" scenarios is one of the most poisonous and yet pervasive ideas in all of database design. You are always combining multiple data sources, all the time . Users submitting data from a form is a data source. Test, stagi…

While your tone might be a tad hyperbolic, I agree with your basic premise.

If I could go back and tell my 30 year ago self one tip, it would be to use uuids over auto-increments. And this is back when that was expensive - in disk space and database time.

Instead I'm stuck with my design, and as time has passed the real cost of auto-Inc has slowly revealed itself.

What's interesting to me though is that this view is not universal. I get a lot of push-back when promoting uuids, but I can really only speak to my experience.

Re: New UUID Formats

#127

Post by acmecorps »

bruce511 wrote: post #126
feoren wrote: post #115

Earlier quoted context omitted.

> if you ever need to combine multiple data sources together in migration and recovery type scenarios This insane idea that combining data sources is a rare event in some unusual "migration and recovery" scenarios is one of the most poisonous and yet pervasive ideas in all of database design. You are always combining multiple data sources, all the time . Users submitting data from a form is a data source. Test, stagi…

While your tone might be a tad hyperbolic, I agree with your basic premise. If I could go back and tell my 30 year ago self one tip, it would be to use uuids over auto-increments. And this is back when that was expensive - in disk space and database time. Instead I'm stuck with my design, and as time has passed the real cost of auto-Inc has slowly revealed itself. What's interesting to me though is that this view is…

You’re not alone. I’ve been migrating my tables to use uuid instead of integers and have been using uuid whenever I have new tables, unless I have very good reason not to. Experience was my teacher.

Re: New UUID Formats

#128

Post by vlovich123 »

acmecorps wrote: post #127
bruce511 wrote: post #126

Earlier quoted context omitted.

While your tone might be a tad hyperbolic, I agree with your basic premise. If I could go back and tell my 30 year ago self one tip, it would be to use uuids over auto-increments. And this is back when that was expensive - in disk space and database time. Instead I'm stuck with my design, and as time has passed the real cost of auto-Inc has slowly revealed itself. What's interesting to me though is that this view is…

You’re not alone. I’ve been migrating my tables to use uuid instead of integers and have been using uuid whenever I have new tables, unless I have very good reason not to. Experience was my teacher.

Don’t UUIDs as primary keys totally destroy the performance because UUIDs aren’t sortable and thus wreak havoc with the index for the primary key?

Re: New UUID Formats

#129

Post by citrin_ru »

GiorgioG wrote: post #22

I used to be a big proponent of using UUIDs for database PKs but I've found them inherently difficult to work with. It's much easier to remember/recognize an integer based PK when troubleshooting a data problem. This isn't to say you shouldn't use UUIDs at all, but I much prefer to use an "ExternalId" column of UUID type if you don't want to expose your integer based PKs externally.

Being impossible to remember to me is a minor problem compare to how much RAM and HDD space wasted on UUIDs across all systems being built nowadays. In addition to being large than say 64bit integer ID they are almost not compressible. DB compression will not reduce used by UUID space in DB, logs with UUIDs are bigger and compression not a great help here either.

Re: New UUID Formats

#130

Post by Akronymus »

vlovich123 wrote: post #128
acmecorps wrote: post #127

Earlier quoted context omitted.

You’re not alone. I’ve been migrating my tables to use uuid instead of integers and have been using uuid whenever I have new tables, unless I have very good reason not to. Experience was my teacher.

Don’t UUIDs as primary keys totally destroy the performance because UUIDs aren’t sortable and thus wreak havoc with the index for the primary key?

In my experience, thats a big NO. While there is some performance impact, in other ways it can actually be a benefit (spreading writes out to multiple pages, for example) so as with anything in regards to performance: benchmark, benchmark, benchmark.

PS.: UUID's also work quite nicely to deal with certain system hiccups, specifically temporal ones. Could be thought of as something similar to the Erlang supervisor trees.

Post reply on HN