> On my system, the minimum increment was 32 ns.
Then you aren’t using a nanosecond clock.
If you want actual nanosecond precision then you probably want rdtsc.
Nanosecond timestamp collisions are common
41–50 of 291 posts
Re: Nanosecond timestamp collisions are common
#42Earlier quoted context omitted.
I will take my chance :-) More seriously, If you can use them, good old increments are probably best. They are fast and cheap. Especially in a database. They can have privacy/security issues (you could guess things by the values of ids of stuff). UUIDs are better in those case or when you deal with a distributed system.
> They can have privacy/security issues (you could guess things by the values of ids of stuff). Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).
If you're going to do that then you might as well just use UUID, since you effectively reintroduce the negative aspects of that (infinitesimally miniscule chance of collisions, computation involved in the calculation, etc.)
Re: Nanosecond timestamp collisions are common
#43Earlier quoted context omitted.
> They can have privacy/security issues (you could guess things by the values of ids of stuff). Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).
If you're going to do that then you might as well just use UUID, since you effectively reintroduce the negative aspects of that (infinitesimally miniscule chance of collisions, computation involved in the calculation, etc.)
The difference is that you can still use sequential IDs internally, while exposing hashed IDs to the outside. This protects your database from collisions under all circumstances, while in the absolute worst case, a single user might experience bugs because two external IDs collide.
Re: Nanosecond timestamp collisions are common
#44This is why you should use ids that combine both a time component and a sequence. Eg UUIDv7 has a milliseconds time component and then a field that increments for each event in the same millisecond, and then enough random bits to make collisions between ids generated on different machines astronomically unlikely. Of course there are only so many bits so you might generate too many events in the same time slice so the…
Isn’t it simpler to use sequence keys then?
Ah but you see, UUIDs are web scale.
And by web scale, I mean they're too big to exchange by any offline channel.
Re: Nanosecond timestamp collisions are common
#45A lot of mention of UUDv7 in this thread which is good. But I also wonder what the collision rate for Ulids are.
I frankly don't understand how it's good. UUID originally was intended as something you use very sparingly, to name, say, a product SKU maybe, an organization, something like that. Not literally content that collides commonly at the same nanosecond, in the same application, in the same platform/org. At some point we have to question the sanity of using one single flat address space for everything from the tiniest ide…
> We can have registrars for global ids, and we can nest local ids inside them, we can have a hierarchy, and we can have local compact ids, 32, 64 or 128 bit, which will never collide, and be locally cohesive.
We have this. It's called OID (Object Identifier) and is used in X.509, LDAP, SNMP, and many other technologies. A global registry exists (I have an entry) and it's a giant tree.
> So why aren't we doing this? Is it ignorance?
The problem you are solving here is for durable, long-lasting keys. There is also a need to generate large batches of short-lived keys that need to never collide for security/identification purposes. A centralized registry will not work for that and it requires a wholly different technique.
Re: Nanosecond timestamp collisions are common
#46Earlier quoted context omitted.
If you're going to do that then you might as well just use UUID, since you effectively reintroduce the negative aspects of that (infinitesimally miniscule chance of collisions, computation involved in the calculation, etc.)
The difference is that you can still use sequential IDs internally, while exposing hashed IDs to the outside. This protects your database from collisions under all circumstances, while in the absolute worst case, a single user might experience bugs because two external IDs collide.
Yes, I tend to like this philosophy in database design, of internal sequential ids which are used for joins between tables etc. and an exposed "external reference". But I typically would use a UUID for my external reference rather than a hash of the internal id.
Re: Nanosecond timestamp collisions are common
#47Re: Nanosecond timestamp collisions are common
#48This is why you should use ids that combine both a time component and a sequence. Eg UUIDv7 has a milliseconds time component and then a field that increments for each event in the same millisecond, and then enough random bits to make collisions between ids generated on different machines astronomically unlikely. Of course there are only so many bits so you might generate too many events in the same time slice so the…
Why do you need the time component anyway? It's just eating up bits in your UUID without contributing much entropy.
Re: Nanosecond timestamp collisions are common
#49Timestamps should probably never be used as a "unique" id.
The problem is achieving locality in cohesion/meaning, which usually involves locality in time, which provokes using timestamps as part of the id at the very least. But it's a chain of very lazy thinking IMHO and it's like a peek at the house of cards some large systems are built like.
Why do you want this kind of locality anyway?
Re: Nanosecond timestamp collisions are common
#50Earlier quoted context omitted.
The difference is that you can still use sequential IDs internally, while exposing hashed IDs to the outside. This protects your database from collisions under all circumstances, while in the absolute worst case, a single user might experience bugs because two external IDs collide.
Yes, I tend to like this philosophy in database design, of internal sequential ids which are used for joins between tables etc. and an exposed "external reference". But I typically would use a UUID for my external reference rather than a hash of the internal id.
Doesn't that just add a whole lot of unnecessary complexity? If elements have multiple IDs, one of which should not be leaked to the outside, that's just asking for trouble in my opinion.
Is generating UUIDv4 or UUIDv7 really too much effort? I'd assume that writing the row to the database takes longer than generating the UUID.