Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nanosecond timestamp collisions are common

evanjones.ca

Post reply on HN
Print view View original

111–120 of 291 posts

Re: Nanosecond timestamp collisions are common

#111

Post by danbruc »

EGreg wrote: post #97
danbruc wrote: post #91

Earlier quoted context omitted.

More importantly if you have an index on purely random IDs, then each insert will go to some random position into the tree whereas having IDs that increase with time will make all new IDs end up at the end of the tree which reduces index fragmentation.

Or, you could use a graph database and stop having frustrating relational impedance mismatch, nonlocality etc. You can have O(1) lookups instead of O(log N) for almost everything

That will depend on which graph database you use as a graph database might just store the graph in an underlying relational database. And it will also depend on what kind of data you have and what kind of queries you want to perform. For a graph database it might be faster to navigate along links in the graph but I would guess you will have to pay a big performance penalty if you have to operate orthogonally to your links, like aggregate across all instances of some entity.

Re: Nanosecond timestamp collisions are common

#112

Post by e12e »

madsbuch wrote: post #67
mcv wrote: post #64

Earlier quoted context omitted.

But shouldn't that be a separate field then?

I am curios about this, and might be misunderstanding what you mean. Can you layout a demo architecture where you use multiple keys like you propose?

Any pure relational database design will eschew surrogate keys - most real-world systems will (should) add them back - because a surprising number of good natural keys end up changing (names change, phone numbers change, emails change, twitter handles become irrelevant/disappear, location of birth may change subject to geographic regions changing size...).

And on top of all that, there are efficiency concerns.

That said, at least for join tables AFAIK - there aren't often a need for row IDs beyond the involved foreign keys - unless you need to add meta data from other relations/tables...

Re: Nanosecond timestamp collisions are common

#113

Post by pravus »

3cats-in-a-coat wrote: post #93
pravus wrote: post #83

Earlier quoted context omitted.

I'll also have the need to make namespaces dynamic and centrally coordinate their lifecycle along with the rest of my infrastructure as I stand up anything that needs some sort of coordinated ID. So I've just moved this issue to an even larger scope with even more complexity. What do I gain for this? UUIDs solve all of this because the chance of creating a duplicate is so low it can effectively be ignored. I can name…

When you get used to this basic principle of when you create something, the creator slaps a name on it, it stops being considered a hassle or confusing. More specifically: 1. When you create something, the creator names it. 2. When you bring something you created outside your scope, you prepend your name to it. It's kind of what we do with (actual) children if you think, slightly less structured, but the idea was alw…

I generate children faster than the birth registry can issue certificates and Cronus eats them before it could be issued anyway. Nesting namespaces doesn't solve the problem of scale within a single namespace.

Re: Nanosecond timestamp collisions are common

#114

Post by EGreg »

e12e wrote: post #109
EGreg wrote: post #97

Earlier quoted context omitted.

Or, you could use a graph database and stop having frustrating relational impedance mismatch, nonlocality etc. You can have O(1) lookups instead of O(log N) for almost everything

How big is the 1 though?

Look, if you have N items related to X, at insert time, you store them in an array and have X point to that array, instead of foreign keys.

For example, when a user has 7 articles. Do you want to just point to where the articles are stored? Or do you want to do O(log n) lookup for each article?

And if you have many-to-many, do you want to join an Intermediate Table for even more processing, or just follow a pointer to a range of an intermediate node directly and traverse?

Re: Nanosecond timestamp collisions are common

#118

Post by hardwaresofton »

wood_spirit wrote: post #5

This is why you should use ids that combine both a time component and a sequence. Eg UUIDv7 has a milliseconds time component and then a field that increments for each event in the same millisecond, and then enough random bits to make collisions between ids generated on different machines astronomically unlikely. Of course there are only so many bits so you might generate too many events in the same time slice so the…

And they play nicely with sort ordering in popular databases like postgres!

They’re not in tree yet but there are a bunch of awesome pg extensions that provide access to uuidv7 (outside of just using it at the application level)

Re: Nanosecond timestamp collisions are common

#119

Post by Thiez »

beefield wrote: post #95
p-e-w wrote: post #21

If you want unique identifiers, use version 4 (random) UUIDs. Problem solved. The probability of a collision is roughly the same as the probability of a fully grown dinosaur spontaneously manifesting in your bedroom due to quantum fluctuations.

Knowing nothing about UUID v4 generation, I have likely a stupid question. What makes you so confident that all implementations and their entropy sources are flawless enough to make actual collision probability close enough to theory?

What makes us so confident that our database implements ACID correctly, the RAM stores bins correctly, and the disk drivers store the data correctly?

In the end we have to make some assumptions about the correctness of (some of the) components.

Re: Nanosecond timestamp collisions are common

#120

Post by ndriscoll »

EGreg wrote: post #114
e12e wrote: post #109

Earlier quoted context omitted.

How big is the 1 though?

Look, if you have N items related to X, at insert time, you store them in an array and have X point to that array, instead of foreign keys. For example, when a user has 7 articles. Do you want to just point to where the articles are stored? Or do you want to do O(log n) lookup for each article? And if you have many-to-many, do you want to join an Intermediate Table for even more processing, or just follow a pointer t…

How is that different from a clustered index?
Post reply on HN