Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Goodbye integers, hello UUIDv7

buildkite.com

Post reply on HN
Print view View original

351–360 of 376 posts

Re: Goodbye integers, hello UUIDv7

#351

Post by chacham15 »

stepanhruda wrote: post #261
paulddraper wrote: post #81

Earlier quoted context omitted.

It depends if you have a request covers a lot of sequential data, or if you have a lot of requests of sequential data.

Correct, it speeds up latency in best case scenario, and falls over in worst case scenario. Randomly sharded keys give a more consistent performance.

You're painting with way too broad of a brush. It is not always better and not always worse. E.g. "give me a list of users who made two posts where both posts were created within 1 second of each other" This query would likely blow up on a system which has all the data completely randomly sharded (because you'd have to aggregate all the data centrally, unless you had a complicated shuffle setup (which most dbs dont)) whereas would work fine on a system which has posts sharded by time.

Re: Goodbye integers, hello UUIDv7

#352

Post by madeofpalk »

pnpnp wrote: post #347
madeofpalk wrote: post #219

Earlier quoted context omitted.

Not disagreeing with the general concept - these IDs leak information - but these are sequential IDs, not auto-incrementing IDs. The leak is the time the ID was generated, not the volume of IDs generated.

They’re not even strongly sequential (is there a term for this?). The gaps between them can be arbitrarily large.

They are sequential, where they are in a sequence where one is clearly before or after another.

They're not monotonic.

Re: Goodbye integers, hello UUIDv7

#353

Post by ricardobeat »

Xeoncross wrote: post #280
kozak wrote: post #272

Earlier quoted context omitted.

As someone who only cares about v4, I periodically wonder why don't I just use fully random 128-bit identifiers instead (without the version information).

UUID v4 isn't large enough to prevent collisions, that is why segment.io created https://github.com/segmentio/ksuid which is 160bit vs the 128bit of a UUIDv4.

Looks like you got things a bit mixed up. You’ll not see a v4 collision in your lifetime, that is not why they built ksuid. At the time, UUIDv1 was the only standard alternative that includes a time component, but in way less bits, not sortable, and the fixed mac address takes a lot of space from the random bits.

ksuid is similar to Twitter Snowflake, the main goal is distributed generation of collision-free sortable IDs. The UUIDv7 proposal is meant to address the same use case. You don’t need to worry about collisions as much here, as the timestamp is monotonically increasing, there is a 42-bit counter for every millisecond + the random 32 bits at the end. You’d have to be generating trillions of IDs per second to have the chance of a collision.

Re: Goodbye integers, hello UUIDv7

#354

Post by travisjungroth »

kiitos wrote: post #342
travisjungroth wrote: post #96

Earlier quoted context omitted.

How so? It seems like the only real use case for these timestamps is to get data from around the same time together. A second is fine for that. It's not about concurrency or avoiding collisions. A second can't handle that, but neither can a millisecond.

> It seems like the only real use case for these timestamps is to get data from around the same time together. Yep. > A second is fine for that. Not when you're doing O(1k-1M) operations per second, it isn't!

I’d think that the locality would only matter at the scale of your query. I’m sure someone has queries with a window less than a second and so much traffic, but it seems niche enough to not optimize the standard for it.

I could definitely be off. I work at a company that gets those levels of traffic but don’t deal with it directly.

Re: Goodbye integers, hello UUIDv7

#355

Post by hannasanarion »

jacobgorm wrote: post #321
hannasanarion wrote: post #303

Earlier quoted context omitted.

In the same millisecond, in the same database system, having rolled the same 74 bit (~22 digit) number?

If it is the same database system, you may as well use a SERIAL or similar synchronized datatype. The point of using UUIDs is that any participant in a distributed system should be able to generate them, without having to rely on a centralized authority. If the use is purely local then UUIDs would be pointless. If you have 74 bits of entropy, the birthday paradox says that after 2^37 keys you will hit a 50% risk of a…

But the uuid collision chance only matters if there is even a possibility of colliding writes and lookups.

So what if your microblogging platform's tweet uuid happens to collide with my cocktail recipe generator's ingredient uuid? The likelihood that any one device will ever be running our apps at the same time and trying to read data from one into the other is even smaller than the 1-in-380000000000000000000000000000000000000 (uuidv4) probability of having a collision in the first place.

Re: Goodbye integers, hello UUIDv7

#356

Post by rtsil »

pnpnp wrote: post #345
rtsil wrote: post #302

Earlier quoted context omitted.

Don't create them each time a record is created, create a batch in advance in sufficient number, and do the same every time the previous batch has ran out. UUIDv7 is 128 bits, you can store a large number of them without major penalty.

They’re also incredibly cheap to create & don’t need knowledge of each other. I mostly see batching of IDs like this when a lock is involved to prevent collisions & maintain performance. With UUIDv7, you are reasonably sure that there won’t be collisions (check your use-case first), and can just generate them wherever on-demand (no locks required). I’d argue batching IDs is actually more complicated than UUIDv7 for m…

This was a solution to the UUIDv7 problem of being time-dependend and therefore may leak information. Create the UUIDv7 in advance by batch of 10,000, use these randomly, and you fix that problem.

Re: Goodbye integers, hello UUIDv7

#357

Post by rtsil »

Macha wrote: post #349
rtsil wrote: post #340

Earlier quoted context omitted.

You don't store the UUIDs in the database as incomplete records. You can put them in a unused_uuids table and store some of the values in memory to minimize the round-trip. You can even store them in a simple file, and remove each used UUID from that file. When the file is empty, you create a million more of them.

The incomplete objects refers to when someone clicks "new" in your UI. Until it's saved back to the server, that "new" object has no ID, since you need to communicate with the server somehow to get that UUID in this approach. So now the client creates objects without IDs, so now all your models need to assume IDs are optional, and you can't create your object references on unsaved objects.

Then we can get a random sample of UUIDs in sufficient number from the batch, send it to the client at the beginning of the session along with the other data, lock that batch until the client releases the session to avoid duplicate use, and have the client use these UUIDs until they run out, at which point it can request a new batch.

Re: Goodbye integers, hello UUIDv7

#358

Post by [deleted] »

pnpnp wrote: post #346
lightbendover wrote: post #263

Earlier quoted context omitted.

Years ago I wrote a library that would exaggerate sequential IDs to make our SaaS platform appear more popular than it actually was to anyone trying to pay attention. Not sure if I’m proud of the hack or embarrassed. But of both I suppose.

but UUIDv7 isn’t sequential (unless I’m getting it mixed up). There’s just a time-based component which can make sorting really nice & some random bits at the end. If you don’t let an attacker iterate your data, all they can tell is when the ID was created.

[deleted]

Re: Goodbye integers, hello UUIDv7

#359

Post by jolux »

pnpnp wrote: post #346
lightbendover wrote: post #263

Earlier quoted context omitted.

Years ago I wrote a library that would exaggerate sequential IDs to make our SaaS platform appear more popular than it actually was to anyone trying to pay attention. Not sure if I’m proud of the hack or embarrassed. But of both I suppose.

but UUIDv7 isn’t sequential (unless I’m getting it mixed up). There’s just a time-based component which can make sorting really nice & some random bits at the end. If you don’t let an attacker iterate your data, all they can tell is when the ID was created.

> If you don’t let an attacker iterate your data, all they can tell is when the ID was created.

The ordering means that you can reconstruct the sequence if you have enough of them, though.

Re: Goodbye integers, hello UUIDv7

#360

Post by Jenda_ »

hannasanarion wrote: post #313
Xeoncross wrote: post #280

Earlier quoted context omitted.

UUID v4 isn't large enough to prevent collisions, that is why segment.io created https://github.com/segmentio/ksuid which is 160bit vs the 128bit of a UUIDv4.

I think you vastly overestimate the likelihood of collisions. If all of the 2 billion computers in the world produced a new uuidv4 every millisecond, we still wouldn't expect a collision for the next 5 quintillion years. Or if the same number of bits are used in a more structured manner, like uuidv1 which combines a 48 bit MAC address, 60 bit 100-nanosecond timestamp, and a 14 bit uniquifier with an effective resolut…

> If all of the 2 billion computers in the world produced a new uuidv4 every millisecond, we still wouldn't expect a collision for the next 5 quintillion years.

This would generate 2^127.8 UUIDs.

First, no, the collision would be expected in less than one year, approximately after exhausting square root of the available space (2^64 generated UUIDs): https://en.wikipedia.org/wiki/Birthday_problem

Second, no, the UUIDv4 has 122 random bits, not 128 as you thought: https://en.wikipedia.org/wiki/Universally_unique_identifier#...

Post reply on HN