Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding UUIDs, ULIDs and string representations

sudhir.io

Post reply on HN
Print view View original

71–80 of 104 posts

Re: Understanding UUIDs, ULIDs and string representations

#71

Post by mdekkers »

hderms wrote: post #61
traceroute66 wrote: post #51

Earlier quoted context omitted.

Beyond the obvious and important security implications of incremental numbers, there is one other major problem with them. They make life hell for database clustering, merges and migrations. In addition, on a more minor level, in a client-centric (apps, browser JS etc) world, the use of incremental numbers is an un-necessary pain point. If you use UUIDs, the client can generate its own without the need to a call back…

Is it really true that concerns around UUIDs as primary keys are wholly irrelevant? Maybe I'm working off outdated information but in high scale environments there are a lot of downsides primarily related to the random write patterns into B-trees causing page splitting and things like that.

I speak from bitter experience - UUID for PKs is not a good idea out of the box. Both writing as well as reading took very significant penalties. I did not design that particular system, but I had to figure out why relatively simple queries took minutes to return.

Re: Understanding UUIDs, ULIDs and string representations

#73

Post by RobertRoberts »

geenat wrote: post #32

Cockroach DB supports ULID since v21.2.3 using gen_random_ulid().

Support (native) in DBs and in libraries is currently the only disadvantage I have found using ULIDs for a while now. But adding a single library to generate ULIDs (front & back end) seems to be good enough for now.

Re: Understanding UUIDs, ULIDs and string representations

#74

Post by mrpotato »

> This entire idea of using random IDs assumes that your computers can generate random numbers that are random and unpredictable — this isn't an easy problem, and there's a lot of research being done in the field.

Can anyone here provide some hints as to how you can verify that the randomness is actually random?

If I was to create some blackbox and I claimed it generated 100% random numbers, is it possible to disprove my claims?

Re: Understanding UUIDs, ULIDs and string representations

#75

Post by oauea »

mrpotato wrote: post #74

> This entire idea of using random IDs assumes that your computers can generate random numbers that are random and unpredictable — this isn't an easy problem, and there's a lot of research being done in the field. Can anyone here provide some hints as to how you can verify that the randomness is actually random? If I was to create some blackbox and I claimed it generated 100% random numbers, is it possible to disprov…

Sure, nothing is 100% random so that is not possible. At best it's difficult to predict.

Re: Understanding UUIDs, ULIDs and string representations

#76

Post by jrochkind1 »

Sjoerd wrote: post #69
jrochkind1 wrote: post #65

Earlier quoted context omitted.

The OP proposes using `ULID`s, which are the same number of bytes as UUIDs, but have an initial timestamp component (ms since epoch), plus a subsequent random component. While these are sequential (not exactly "incremental"), so give two of them you can know which came first -- they aren't really "guessable", as you'd need to guess not only an exact timestamp (not infeasible if more of a challenge than with increment…

They aren't guessable, except for ULIDs generated by the same process in the same millisecond. To keep chronological order even within the same timestamp, ULIDs within generated within the same millsecond become incremental. This can become relevant for example when an attacker requests a password reset for himself and the victim simultaneously.

Interesting, I learned about ULIDs for the first time from this article, which says: "The remaining 80 bits [in a ULID] are available for randomness", which I read as saying those last 80 (non-timestamp) bytes were random, not incremental. But this was misleading/I got the wrong idea?

Going to the spec [1]... Yeah, that's weird. The spec calls those 80 bytes "randomness", and apparently you are meant to generate a random number for the first use within a particular ms... but on second and subsequent uses you need to increment that random number instead of generating another random number?

Very odd. I don't entirely understand the design constraints that led to a non-random "randomness" section still called "randomn" in the spec even though they are not.

[1]: https://github.com/ulid/spec

Re: Understanding UUIDs, ULIDs and string representations

#77

Post by lantern_oil »

I haven't used SQL Server directly in a while, but IIRC, we can use the identity() function, which does 2 things: 1. tells the system to start counting at a certain number, and 2. tells it to increment the next number by a certain multiple (i.e. x2)

I do agree that serially incrementing numbers may sometime be relegated to the history books, although this is such a baked in function that, lacking any other prebuilt functions, old-time SQL developers simply reach for it.

Re: Understanding UUIDs, ULIDs and string representations

#78

Post by traceroute66 »

hderms wrote: post #61
traceroute66 wrote: post #51

Earlier quoted context omitted.

Beyond the obvious and important security implications of incremental numbers, there is one other major problem with them. They make life hell for database clustering, merges and migrations. In addition, on a more minor level, in a client-centric (apps, browser JS etc) world, the use of incremental numbers is an un-necessary pain point. If you use UUIDs, the client can generate its own without the need to a call back…

Is it really true that concerns around UUIDs as primary keys are wholly irrelevant? Maybe I'm working off outdated information but in high scale environments there are a lot of downsides primarily related to the random write patterns into B-trees causing page splitting and things like that.

> Is it really true that concerns around UUIDs as primary keys are wholly irrelevant?

I would say yes, with the options we have today with modern compute.

We live in a world where compute is powerful enough to enable Let's Encrypt to issue SSL certificates for 235 million websites every 90 days off the back of a single MySQL server[1].

For high scale environments there are also other options such as async queues and Redis middleware.

Database technology itself is also evolving, and the degree of measurable downside is less than it might have been 10 years ago.

I would still argue that for the vast majority of people, UUIDs are the way to go. I would certainly urge caution against premature optimisation involved with the "but high scale" argument. Sure things MIGHT be noticeable at high scale, but I think its fair to say most people are not doing anywhere enough high scale to do so and should probably just use UUIDs and cross the "high scale" bridge if/when they ever come to it.

Finally, its also worth pointing out that all the hyperscalers use UUIDs or other unique identifiers widely in their infrastructure and APIs, all of which must inevitably be tied into a database backend.

[1]https://letsencrypt.org/2021/01/21/next-gen-database-servers...

Re: Understanding UUIDs, ULIDs and string representations

#79

Post by asimpletune »

heavenlyblue wrote: post #67
gorgoiler wrote: post #20

All this stuff about collisions and avoiding them, even though they will never happen, feels like a PHB compliance issue. “Great work Geoff! One question: what’s the probability of two transactions having the same ID?” “It is very low” “Hmmm. But it’s not zero?” “It’s so low that it practically is zero.” “But it’s not technically zero? This company wasn’t built on taking chances, son! Come back when your product comp…

As Dostoevsky once said "All functional people are the same, all dysfunctional people are dysfunctional in their own way". Don't get me wrong, but you can always invent an idiot who will go out of their way to make something unbearable, but it doesn't mean we should always base our judgment on that.

Didn't Tolstoy also say "All happy families are the same, and all unhappy families unhappy in their own way"? (I'm paraphrasing)

Re: Understanding UUIDs, ULIDs and string representations

#80

Post by traceroute66 »

mdekkers wrote: post #71
hderms wrote: post #61

Earlier quoted context omitted.

Is it really true that concerns around UUIDs as primary keys are wholly irrelevant? Maybe I'm working off outdated information but in high scale environments there are a lot of downsides primarily related to the random write patterns into B-trees causing page splitting and things like that.

I speak from bitter experience - UUID for PKs is not a good idea out of the box. Both writing as well as reading took very significant penalties. I did not design that particular system, but I had to figure out why relatively simple queries took minutes to return.

> I did not design that particular system ... why relatively simple queries took minutes to return

The road of databases is paved with many such bodies.

Whether it is developers treating databases like some blackbox dumping ground, or designing generic "portable" schemas, or people who don't know SQL writing weird long convoluted queries.

Many people are quick to blame "the database", but 99% of the time its the fault of those who designed the schema and/or the queries that run on it.

I think your statement "UUID for PKs is not a good idea out of the box" is unfair and too broad brush. Without knowing the exact details of every bit of your environment (from database hardware upwards), its not possible to accept such a generic statement to be read as a fact.

Post reply on HN