Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye integers, hello UUIDv7

buildkite.com

Post reply on HN
Print view View original

251–260 of 376 posts

Re: Goodbye integers, hello UUIDv7

#251

Post by sgarland »

hinkley wrote: post #111
dheera wrote: post #16

Earlier quoted context omitted.

It's bad for performance if you frequently access large consecutive sets of records.

UUIDs are good for data where I want either lots of different users being able to insert without collision, or lots of users who I want to keep their peepers off of other user's metadata (eg, how many X they add to the system per day). In both cases I'm melding highly disjointed data into a single schema. There are no large consecutive sets of records. If you're using UUIDs, there's probably a reason. And that reason…

> If you're using UUIDs, there's probably a reason.

Not really; think of how many architectural decisions are made purely based on imaginary scaling problems, or what the latest blog said.

I would wager that if you polled 100 backend devs, very few of them could correctly articulate the pros and cons of a randomized primary key.

Re: Goodbye integers, hello UUIDv7

#252

Post by sgarland »

jhealy wrote: post #204
erik_seaberg wrote: post #43

> first component (prefix) of the identifier is a sortable timestamp > values generated are practically sequential These statements aren’t strict enough to be relied on. Maybe you have engineered the hell out of your distributed clock scheme, and your IDs actually are completely monotonic, which is great. But you probably haven’t done that, which means conflicts will surely happen and you must handle them gracefully.

I'm not the author but I work at the same company. None of our systems require perfect ordering of IDs generated across our distributed system. Most of the system was built with random UUIDv4 identifiers so no code assumes the ID ordering is significant. However, in much of our system recent data is frequently accessed while old data is rarely accessed. In that world, just having the IDs *approximately* clustered in…

> In that world, just having the IDs approximately clustered in creation order has been a huge performance boost for many queries, and we've seen significant reduction in postgres Write Ahead Log rates, because writes to UID indexes happen in a smaller number of pages.

Thank you. I’m so tired of seeing the same groupthink on UUIDv4 trotted out - “it only matters if you have a clustered index, Postgres is immune!” The hell it is.

Re: Goodbye integers, hello UUIDv7

#253

Post by insanitybit »

> The nature of Buildkite's products mean recent data is accessed more frequently than old data. With non-sequential identifiers, the most recent data will be randomly dispersed within an index and lack clustering

I would assume that `serial` would solve this problem too.

Re: Goodbye integers, hello UUIDv7

#254

Post by gabereiser »

dbbk wrote: post #250
gabereiser wrote: post #54

Earlier quoted context omitted.

Stored in binary format, validation, more efficient due to non-cast, faster access due to non char*, being able to split the high-low, indexing and uniqueness at the byte level.

What validation? "Postgres doesn’t care what you store in it as long as it has the correct length."

I mean, if(len(O) == 32) technically is a validation but you’re right, by default it just fits for size.

Re: Goodbye integers, hello UUIDv7

#255

Post by School-Cotton »

sgarland wrote: post #243
dalore wrote: post #203

Earlier quoted context omitted.

Having sequential ID's is more than just a security risk, it's an information risk. Competitors can use them to estimate the size of your business, the number of customers you have, and all sorts of stuff. This was used in the war to estimate the number of German tanks based on the sequential IDs https://en.wikipedia.org/wiki/German_tank_problem So just for business intelligence you don't want to leak your IDs.

I’ve heard this argument many times, but I’ve never seen anyone actually post a reference to it happening (as in, a company finding and using this information; not the German tank problem). To me, it reeks of solving imaginary problems while causing new ones.

I vaguely remember somebody figuring out from photoshop activation IDs how many sales adobe was making and trading options around their earnings report with that information.

I don’t remember the details, so maybe it was something else and not photoshop/adobe.

Re: Goodbye integers, hello UUIDv7

#256

Post by onion2k »

matharmin wrote: post #158
BiteCode_dev wrote: post #100

Earlier quoted context omitted.

What benefice over uuid4 ?

Reading their docs: No real benefits, just misconceptions. 1. Collision resistance / "weak" PRGNs used to generate UUIDv4. Firstly, these are properties of the implementation , not the spec. Secondly, the source for calling the browser `Crypto.getRandomValues()` insecure is an issue that has been fixed back in 2016. I would not trust the developers of this implementation to do a better job than current browsers. 2. "…

No argument given for why having a slower algorithm to generate random ids is more secure.

If the algorithm is too fast it means you can detect when some other part of the system is having a significant impact on how the key is returned. Eg checking a database to see if a user exists and returning their key versus getting null back and generating a new key. That difference can be used to determine if a user exists. You want your key gen process to be slow enough that it's a significant part of the process, which makes timing attacks hard.

Re: Goodbye integers, hello UUIDv7

#257

Post by mort96 »

ilyt wrote: post #245
mort96 wrote: post #233

Earlier quoted context omitted.

If you were using something like UUIDv4, you wouldn't be exposing that information at all though, neither in cleartext or ciphertext. It seems weird to say, "the user ID contains secret information so we encrypt it with an eternal fixed pre-shared key then share the ciphertext with the world", when you could've just said "the user ID contains no secret information". It feels like the right solution here is to pick be…

You could just randomize the timestamp. adding +/- month or two to the UUIDv7 won't break the advantages all that much.

That doesn't fix the info leak though, you're still leaking approximately when the account was created. Knowing whether an account was created in 2003 or 2023 is a pretty significant amount of information even if you just know that it was created some time between June 2003 and August 2003.

I mean it's certainly an improvement over telling everyone the millisecond the account was created, but if account creation times are to be considered non-public info, I would probably just not include any version of it in public facing user IDs. And if you do consider approximate account creation times to be public (such as HN, where anyone can see that my account was created July 1, 2015), then adding some fuzz to the timestamp seems to be a good way to avoid certain cryptographic issues.

Re: Goodbye integers, hello UUIDv7

#258

Post by phkahler »

Is there some reason new versions of UUID keep appearing? It seems like the desired properties are never quite achieved so new ones appear later. Is there a table with UUID version across the top and characteristics down the side, so I can see the differences and pick one that fits my needs? That might also help to explain why there are so many variants.

Re: Goodbye integers, hello UUIDv7

#259

Post by tgv »

CSDude wrote: post #105
tgv wrote: post #70

Earlier quoted context omitted.

The German tank production capacity was estimated by serial numbers of captured tanks. There are ways to read all kinds of information by observing energy usage. High resolution time and sequence data undoubtedly reveal more than you’d like.

Most of our lives as boring SaaS etc. software developer will not be near as exciting as this, but of course you may never know. I parsed the EV chargers APIs where I live (using Frida in Android) and one of the fields returned the daily revenue and profit.

Sure, for many places it doesn't really matter, but if your URLs or user ids can be seen/scraped by others, you might expose some commercially interesting information to competitors.

And the indexing argument isn't really compelling, is it? You lose very little by sticking to fully random UUIDs.

Re: Goodbye integers, hello UUIDv7

#260

Post by bhouston »

phkahler wrote: post #258

Is there some reason new versions of UUID keep appearing? It seems like the desired properties are never quite achieved so new ones appear later. Is there a table with UUID version across the top and characteristics down the side, so I can see the differences and pick one that fits my needs? That might also help to explain why there are so many variants.

https://en.wikipedia.org/wiki/Universally_unique_identifier#...
Post reply on HN