Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye integers, hello UUIDv7

buildkite.com

Post reply on HN
Print view View original

51–60 of 376 posts

Re: Goodbye integers, hello UUIDv7

#51

Post by andrewmatte »

jonhohle wrote: post #35

This is great for internal distributed systems where having ordered keys is useful, however, it should probably be noted that these probably shouldn't be used as public identifiers (even though this will probably be the defacto standard and used publicly without thought). Having any information, specifically time information, leaking from your systems may or may not have unanticipated security or business implication…

jonhohle, thanks. Do you know of examples of when milliseconds are part of the session tokens or accounts being created has been exploited?

Re: Goodbye integers, hello UUIDv7

#52

Post by kijin »

stepanhruda wrote: post #23
chacham15 wrote: post #9

Earlier quoted context omitted.

They can be bad for performance. It all depends on your access patterns. A common caching pattern is called "temporal locality" which means that theres a high likelihood that data created at the same time will be accessed at the same time. Therefore, if these pieces of information are on the same machine, they can be queried / returned much faster than if they were both on separate machines. This is doubly true if th…

Yes but if that machine with sequential data receives 100x the traffic of other machines, it can be worse than splitting this traffic evenly across all available machines.

If your database simply shards keys sequentially, it's going to get hotspots in a lot of use cases, like plain old integer keys and timestamps, not just UUIDv7. In that case it would be fair to say that your database is doing it wrong.

Fortunately, there's no rule that says you should shard your keys using the sequential part up front.

One of the rules for generating randomness from environmental sources is to throw away the high bits and only use the low bits. Distributed databases should do the same if they want a good distribution.

Re: Goodbye integers, hello UUIDv7

#54

Post by gabereiser »

perfmode wrote: post #21
pyrolistical wrote: post #2

And you can use it today with Postgres uuid type. Postgres doesn’t care what you store in it as long as it has the correct length. So you can generate a uuidv7 and store it natively

What are the benefits of using the Postgres uuid type (versus using TEXT or VARCHAR)?

Stored in binary format, validation, more efficient due to non-cast, faster access due to non char*, being able to split the high-low, indexing and uniqueness at the byte level.

Re: Goodbye integers, hello UUIDv7

#55

Post by oconnore »

jonhohle wrote: post #35

This is great for internal distributed systems where having ordered keys is useful, however, it should probably be noted that these probably shouldn't be used as public identifiers (even though this will probably be the defacto standard and used publicly without thought). Having any information, specifically time information, leaking from your systems may or may not have unanticipated security or business implication…

Given that a UUID identifier fits in a single cipher block, and the whole point is that these are unique by construction (no IV needed so long as that holds true), it seems like a single round of ECB-mode AES-128 would enable quickly converting between internal/external identifiers.

128 bits -> 128 bits

Re: Goodbye integers, hello UUIDv7

#57

Post by gabereiser »

chewbacha wrote: post #29
declan_roberts wrote: post #26

It’s 2023. Why aren’t we using more characters from the utf-8 keyspace to make things like UUIDs use less characters?

UUIDs are 128-bits, not characters. The string representation is just for humans.

This is the way. Look not at the characters but at the hex.

Re: Goodbye integers, hello UUIDv7

#58

Post by LAC-Tech »

erik_seaberg wrote: post #43

> first component (prefix) of the identifier is a sortable timestamp > values generated are practically sequential These statements aren’t strict enough to be relied on. Maybe you have engineered the hell out of your distributed clock scheme, and your IDs actually are completely monotonic, which is great. But you probably haven’t done that, which means conflicts will surely happen and you must handle them gracefully.

I feel like it's well beyond the scope of UUIDs to get into "are your clocks really monotonic?". They give you 48 bits for a timestamp, what that timestamp signifies (transactional time, valid time...) and how that's generated is up to you.

Out of curiosity, are you into hybrid logical clocks?

Re: Goodbye integers, hello UUIDv7

#59

Post by erik_seaberg »

__MatrixMan__ wrote: post #46
erik_seaberg wrote: post #43

> first component (prefix) of the identifier is a sortable timestamp > values generated are practically sequential These statements aren’t strict enough to be relied on. Maybe you have engineered the hell out of your distributed clock scheme, and your IDs actually are completely monotonic, which is great. But you probably haven’t done that, which means conflicts will surely happen and you must handle them gracefully.

When I'm using integer surrogate keys to manipulate subsets of a table, they usually correspond to some kind of out-of-band predicate. Like this is the data that appeared today , or this is everything after the bug happened , etc. Maybe there are applications where the monotonicity matters, but in my experience reasoning by surrogate key is rather coarse grained and you manually scrutinize the boundaries, so unless y…

I’ve seen too many people write a query like

  where ts between txn_start and txn_end
  order by ts
and not even realize that what they’re seeing is incomplete and misleading. Clock skew is very common, and we shouldn’t sweep that under the rug to promote time ordering, because people want to believe this works the way they think.

Re: Goodbye integers, hello UUIDv7

#60

Post by ianbutler »

erik_seaberg wrote: post #43

> first component (prefix) of the identifier is a sortable timestamp > values generated are practically sequential These statements aren’t strict enough to be relied on. Maybe you have engineered the hell out of your distributed clock scheme, and your IDs actually are completely monotonic, which is great. But you probably haven’t done that, which means conflicts will surely happen and you must handle them gracefully.

This was my first reaction as well. The keys use a unix timestamp, which are clearly not going to be synchronized by default so for event ordering purposes across a distributed system this is dodgy.

For providing better query locality it probably doesn't matter significantly though which seems to be the main benefit here while preserving the other benefits UUIDs provide.

Post reply on HN