For me #1 would be to add a version to any data format or communication protocol. If you want to know how hard not doing so can bite, don't look further than Git and it's tourcherous migration from sha1.
What happened with git? Could somebody elaborate?
Things you should do now (2011)
91–100 of 115 posts
Re: Things you should do now (2011)
#92Earlier quoted context omitted.
Actually, afaik uuids are pseudo-random and can thus absolutely collide in theory. Collision is solved by every db backend out there.
UUID collision is not possible in practice. You can absolutely code in the absolute knowledge that your application will not generate 2 identical UUID's ever. The odds are astronomical. You have bigger concerns. And yes, every DB backend has solved collision. But as I said, it's useful to be able to generate ID's in code sometimes. This is possible with UUID's and not possible with integers.
> UUID collision is not possible in practice.
UUID is a loosely defined concept. There are many implementations. Some of them have no chance of collision at all, some have an astronomical chance, and some have very real odds that you'll receive a call at 3AM during the new year's celebration.
Re: Things you should do now (2011)
#93Earlier quoted context omitted.
This is really an ancillary benefit and not a real solution to enumeration vulnerabilities. The concern is usually over sharding the data, unique ID generation guarantees, and performance. You shouldn't really be deciding UUID vs. Int based on the possibility for ID enumeration. It's almost certainly easier to come up with a solution to slow/prevent enumeration than move from Ints to UUID.
There's also leaking financial, business data. E.g you create an order and you see your order id being 3000 the first day and 4000 the next, so then you'll be able to guess how many orders this company has etc.
This is called the German Tank Problem[0].
Re: Things you should do now (2011)
#94Earlier quoted context omitted.
What's wrong with integer IDs?
The main issue in my experience are so called "user enumeration attacks", especially with sequential ids that are normally used. This is where an attacker is able to leak information from your system just by guessing ids. If you used a sequential id then you can cycle from 1 to X and probably easily find which are valid users. You can then likely see how many valid users/ids there are and potentially pull their data…
Obscure user IDs could be a defense in depth measure but really you need to be authorizing the data you release against the authenticated session cookie. A view meant for the user's own consumption shouldn't take a user ID at all, just pull it from the session.
Re: Things you should do now (2011)
#95Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
Integers are by far the best primary key material possible. Only for lack of discipline do they seem inferior to alternatives. See a prior book I wrote on HN regarding this: https://news.ycombinator.com/item?id=25309248 The current domain model I am working with utilizes a global integer sequence to key all entities. This implicitly eliminates the class of bugs where the same keys of different types overlap and would…
The creation of integers doesn't scale for large data volumes. Plus you need a place to create these integers, and a failover location, which adds complexity. Multiple machines can each be creating their own guids in a very simple manner.
And the odds of a guid collision is extremely low, and for most applications is acceptable. Having worked with petabytes of data guid performance isn't really an issue as there are more important factors to worry about.
Re: Things you should do now (2011)
#96Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
ULID. 128bit, human "readable", sorted (like int!), nearly impossible to guess/collide and has timestamp embedded. I love this format.
How does it compare to cuid [1]? If you know the differences, can you please explain them for those like me who do not know both?
Re: Things you should do now (2011)
#97Earlier quoted context omitted.
Integers are by far the best primary key material possible. Only for lack of discipline do they seem inferior to alternatives. See a prior book I wrote on HN regarding this: https://news.ycombinator.com/item?id=25309248 The current domain model I am working with utilizes a global integer sequence to key all entities. This implicitly eliminates the class of bugs where the same keys of different types overlap and would…
The creation of integers doesn't scale for large data volumes. Plus you need a place to create these integers, and a failover location, which adds complexity. Multiple machines can each be creating their own guids in a very simple manner. And the odds of a guid collision is extremely low, and for most applications is acceptable. Having worked with petabytes of data guid performance isn't really an issue as there are…
> The creation of integers doesn't scale for large data volumes.
At which specific integer does the scaling start to slow down?
Re: Things you should do now (2011)
#98Earlier quoted context omitted.
The creation of integers doesn't scale for large data volumes. Plus you need a place to create these integers, and a failover location, which adds complexity. Multiple machines can each be creating their own guids in a very simple manner. And the odds of a guid collision is extremely low, and for most applications is acceptable. Having worked with petabytes of data guid performance isn't really an issue as there are…
> The creation of integers doesn't scale for large data volumes. At which specific integer does the scaling start to slow down?
One concurrent insertion, or one network partition.
UUIDs can be generated on many machines with no awareness of each other and merged later.
Re: Things you should do now (2011)
#99Earlier quoted context omitted.
> The creation of integers doesn't scale for large data volumes. At which specific integer does the scaling start to slow down?
One concurrent insertion, or one network partition. UUIDs can be generated on many machines with no awareness of each other and merged later.
I would recommend reviewing my prior comments on this, as I address the concerns of multiple nodes needing to be able to independently produce identities without collisions or coordination.
If you know beforehand the maximum number of participants in your system, you can divide the keyspace across that quantity. If you are using BigInteger or equivalent, you have an infinite number of these things to work with, so it doesnt really matter if you wind up skipping trillions of identities at first. The original article even advocates for this as its first point, but without as much practical justification.
Re: Things you should do now (2011)
#100Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
Tell that to your database which is highly optimized for using integers as IDs. Using a UUID everywhere when you could be using an integer is an invitation to low-performance-city.
This sounds like a case of "premature optimization is the root of all evil." Do you have any performance numbers that show UUIDs being any noticeably slower?
A UUID is only the size of two 64-bit integers anyway. And either identifier likely will be a tiny fraction of the data in a particular row. So I'd doubt this is any real performance problem in the vast majority of applications.
UUIDs look big and scary in hex notation, but underneath it's a compact and fast binary format, just a 128-bit integer.