181–190 of 291 posts
Comment out CLOCK_MONOTONIC_RAW because that's not available on FreeBSD, and it looks OK to me? My understanding is that we should see some timestamps repeating if there's a collision, correct? I can't get any collisions... > ./clock_gettime_demo/clock_gettime_demo clock_getres(CLOCK_REALTIME, ...)=1 ns clock_getres(CLOCK_MONOTONIC, ...)=1 ns CLOCK_REALTIME 30 samples: 1689957434662039455 1689957434662039526 (diff=71…
Earlier quoted context omitted.
I feel like v7 is almost a strictly better v4. Assuming you can generate a v7 (you have a time source), what are the disadvantages?
Entropy. You loose bits to a known source, which reduce entropy of the UUID.
Earlier quoted context omitted.
You need it to make database indices perform better. If you don't need that, but just need a random UUID, UUIDv4 is better.
I dont know why people use relational databases other than they were first and “that’s the way it’s always been done”. Why not use a graph database? O(1) lookups instead of O(N). Why need indices if you can just point to the data. Why use JOINs when map-reduce querying is far more flexible?
The reason why you want indices is because some query patterns don't have a key to perform a look up on.
Comment out CLOCK_MONOTONIC_RAW because that's not available on FreeBSD, and it looks OK to me? My understanding is that we should see some timestamps repeating if there's a collision, correct? I can't get any collisions... > ./clock_gettime_demo/clock_gettime_demo clock_getres(CLOCK_REALTIME, ...)=1 ns clock_getres(CLOCK_MONOTONIC, ...)=1 ns CLOCK_REALTIME 30 samples: 1689957434662039455 1689957434662039526 (diff=71…
Do you run it on 4 cores simultaneously as the author did?
Earlier quoted context omitted.
But a hash() destroys information. When you are trying to reduce collisions, why would you use a function that is known to introduce collisions?
Because the raw value from the timestamp will be very low entropy and have the short scale variation concentrated in just a few bits. A hash not just destroys information, it also creates entropy by mixing that information over all the bits that come out of it. And using 64 bit hash replacing a 64 bit nanosecond counter that has short term entropy of less than 16 bits, you're in fact reducing the likelihood of a coll…
It's a nitpick, but it concentrates the entropy. It doesn't create any.
I do think it will make the answer more clear, as the hash concentrates the less than 64 bits of entropy on that 128 bits of original data into a usable 64 bits package.
Comment out CLOCK_MONOTONIC_RAW because that's not available on FreeBSD, and it looks OK to me? My understanding is that we should see some timestamps repeating if there's a collision, correct? I can't get any collisions... > ./clock_gettime_demo/clock_gettime_demo clock_getres(CLOCK_REALTIME, ...)=1 ns clock_getres(CLOCK_MONOTONIC, ...)=1 ns CLOCK_REALTIME 30 samples: 1689957434662039455 1689957434662039526 (diff=71…
Do you run it on 4 cores simultaneously as the author did?
running longer zeros test ...
sampled 10000000 pairs; 0 time diff zeros = 0.000000%; 0 nano diff zeros = 0.000000%
starting parallel test 24 goroutines x 10000000 samples ...
10000000 samples from a thread; 0 collisions inside the thread; 0 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 945466 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 1982688 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 2739919 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 3334361 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 4109772 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 4489881 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 5157178 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 5596508 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 5854763 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 5937583 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 6434076 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 6521917 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 6932626 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7104428 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7285076 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7514904 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7833317 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7737265 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7723545 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7730441 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 8311161 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 7965117 collisions with other threads
10000000 samples from a thread; 0 collisions inside the thread; 8460173 collisions with other threads
102297835 final samples; 137702165 total collisions = 57.375902%; possible duplicate collisions? 0Earlier quoted context omitted.
I will take my chance :-) More seriously, If you can use them, good old increments are probably best. They are fast and cheap. Especially in a database. They can have privacy/security issues (you could guess things by the values of ids of stuff). UUIDs are better in those case or when you deal with a distributed system.
> They can have privacy/security issues (you could guess things by the values of ids of stuff). Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).
If you want a secure identifier, make a random 64 or 128 bits number (a UUID type 4). And do not use this number as an internal identifier, because identifiers performance is all about predictability and low entropy.
Earlier quoted context omitted.
> replace the low bits with the hash of them Doesn't this hash-step only increase the probability of collisions? What is this step intended to solve?
> replace the low bits with the hash of them, concatenated with the value of the CPU cycle counter (`RDTSC` on x86) you're concatenating two values and then taking the hash of the combination, ie: hash(low bits + CPU cycle counter)
low bits + CPU cycle counter
Earlier quoted context omitted.
> They can have privacy/security issues (you could guess things by the values of ids of stuff). Push them through a secure hash function, and that problem is solved too (assuming you can keep the base counter private).
Hum, no. You can easily hash numbers from 1 up to the value you see and guess the next value. If you want a secure identifier, make a random 64 or 128 bits number (a UUID type 4). And do not use this number as an internal identifier, because identifiers performance is all about predictability and low entropy.
Earlier quoted context omitted.
Because the raw value from the timestamp will be very low entropy and have the short scale variation concentrated in just a few bits. A hash not just destroys information, it also creates entropy by mixing that information over all the bits that come out of it. And using 64 bit hash replacing a 64 bit nanosecond counter that has short term entropy of less than 16 bits, you're in fact reducing the likelihood of a coll…
> it also creates entropy It's a nitpick, but it concentrates the entropy. It doesn't create any. I do think it will make the answer more clear, as the hash concentrates the less than 64 bits of entropy on that 128 bits of original data into a usable 64 bits package.