81–90 of 115 posts
Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
ULID. 128bit, human "readable", sorted (like int!), nearly impossible to guess/collide and has timestamp embedded. I love this format.
Earlier quoted context omitted.
its not blacklist and whitelist because "black=bad, white=good", black is the abscense of light, hence the light is blocked, white would obviously be the opposite. So its black because you are blocking things, and white because you aren't.
It was never about this. The meta-game is to mess with people and teach them compliance and self-censorship and hyper-attention to anything even remotely related to race. Race must be always the 1st thing you think of. Don't forget race.
Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
An ID should be:
1.) Actually an identifier, i.e. it needs to identify objects uniquely and must be immutable throughout the lifetime of the object.
2.) Assigned at object-creation time; you can't have an object without an ID (in persistent storage), lest you'll have no way to refer to that object later. This presents particular problems for a lot of workflows where you want to introduce the concept of a user early, before soliciting personal data. Think of storing browsing histories for guest users, or persisting shopping carts before checkout.
3.) Integral to the storage system. IDs will usually be the ways that you lookup and join objects; the performance characteristics of your database can influence the type of data you choose for an ID.
4.) Because of #2 & #3, oftentimes a significant concurrency bottleneck for object creation. This is the downfall of many auto-increment integer schemes.
5.) The foreign key for other objects. This has space implications that you often have to trade-off against latency implications. If you store useful information within the ID, you can use that info without needing to make a separate query or join against the DB. However, you need to carefully consider whether that'll run afoul with #1, and the more information you put in the ID, the bigger the size bloat for other objects that reference that ID.
6.) Oftentimes a security & PII risk. Because IDs are the primary means for lookup and are guaranteed to be unique, it's awfully tempting to put them in your public APIs (like HTTP URLs). But then anyone who has a URL has all the information included in the ID. If you use sequential IDs, they also have the ability to scan your entire database; this was the downfall of Parler.
Integer keys do really well for #1, #2, #3, and #5, but are very problematic for #4 and #6. But then, there are fairly easy workarounds for those: #4 is often solved by hashing a unique natural key of the object (also solving #6), while #6 can be solved by never exposing internal IDs to the outside world and instead using a lookup table on some friendly URL scheme, which is better for UX anyway (at the cost of #3). GUIDs do well for #1, #2, #4, and #6, but perform worse for #3 and #5.
Natural keys (where you make the identifier some combination of the actual data) are also frequently underrated. The most obvious use for these are relation tables, where each row just indicates that two entities have the relationship that the table describes; you wouldn't normally put an auto-increment ID on that. But think also of something like a search refinement: the most natural key for that is [query, language] => [list of suggestions], and that uniquely identifies each set of refinements, and it has other useful properties where the rest of your search engine doesn't need to know that refinements exist (it already has the query), and you don't need a lookup call from query to some search_refinement_id or vice versa, and you can easily enumerate the set of languages that a given query has data for, and you can follow a chain of refinements without any intermediate lookups. If you were to suggest either integers or UUIDs for this problem I'd say that you're overengineering. And if the problem domain changes such that the key-set changes (for example, you want to include past search context, or you want to personalize refinements to each user) then I would recommend you start a different system from scratch and eventually replace or merge in the current one, because those problems have sufficiently different requirements that your whole data pipelines are going to be different. (In particular, personalized search requires PII handling, a lot of care in logging, encryption of the data when at rest, knowledge of a lot more entities in the system, etc.)
Earlier quoted context omitted.
No you should not. UUIDs are not always a good solution. If you order things by id (often useful for pagination, since incrementing ids are usually ordered by creation date) uuid are of no help. You would need to sort by created_at and this would require an additional index. No to speak of pagination by id ranges or whatever is used sometimes. UUIDs are mpossible to remember and thus somewhat cumbersome to use too. T…
Integer ID's are always a bad thing. Yes, you should be sorting things by created_at. Having another index is not a bad thing. > The weird id mixed up with index problem from the article is something which never happened to me ever in over 10 years of web development. How do you know? It's a remarkably difficult bug to detect.
All of this points to using serial numeric IDs as both a unique key and the clustering key. UUIDs do nothing to help with any of those requirements, and generally hurt all of them. They do that because of certain characteristics they have that are specifically there in order to solve one of the few problems we didn't have to worry about.
Tangentially, in software systems design, I've long since realized that the word "always", when left to roam around freely, unchaperoned by any qualifiers to limit its universality, is an indicator of limited breath of experience. So, when you encounter it, it's useful to mentally insert "in my experience" as a stand-in qualifier. Having done so, the next conundrum is that "in my experience" advice is only actionable to the extent that you know what experience the advice giver has to draw on.
Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
integer ids are still often used internally for database primary keys with UUIDs being the done thing for external interfaces. Personally I've never experienced the "whole class of bugs" that starting with a big integer is supposed to solve. I'm not using PHP so maybe that's why?
Assuming you can stomach the latency hit, the best solution is usually a lookup table where you take all the friendly URL keys and map them to internal identifiers. So if you're making a multiplayer game site and want to create a page where folks can find their friends, then you might support yourgame.net/user/username, yourgame.net/character/charactername, yourgame.net/steam/steamlogin, yourgame.net/xbox/xboxgamertag, etc. Internally you have an inverted index that maps [type, string] to the internal ID for the player, then proceed normally.
Earlier quoted context omitted.
What's wrong with integer IDs?
The main issue in my experience are so called "user enumeration attacks", especially with sequential ids that are normally used. This is where an attacker is able to leak information from your system just by guessing ids. If you used a sequential id then you can cycle from 1 to X and probably easily find which are valid users. You can then likely see how many valid users/ids there are and potentially pull their data…
Earlier quoted context omitted.
integer ids are still often used internally for database primary keys with UUIDs being the done thing for external interfaces. Personally I've never experienced the "whole class of bugs" that starting with a big integer is supposed to solve. I'm not using PHP so maybe that's why?
I've run into one, but it was pretty dumb. Multitenant application using the account ID as the first element in the URL on a Rails app. when it got to account 404, my address.com/404 went to the static 404 page rather than account 404. That user was pretty confused for awhile.
For me #1 would be to add a version to any data format or communication protocol. If you want to know how hard not doing so can bite, don't look further than Git and it's tourcherous migration from sha1.
Things you should never do: use integers as ID's. This is literally a solved problem, and the solution is UUIDs, which were invented for exactly this job.
One of the best pieces of programming advice I got at Google was consider IDs carefully . This is not a solved problem, and there isn't a one-sized-fits-all solution. Rather, there are a bunch of guide rules, and then you need to understand your problem domain very well to choose good identifiers. An ID should be: 1.) Actually an identifier , i.e. it needs to identify objects uniquely and must be immutable throughout…
For me #1 would be to add a version to any data format or communication protocol. If you want to know how hard not doing so can bite, don't look further than Git and it's tourcherous migration from sha1.