Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You'll regret using natural keys

blog.ploeh.dk

Post reply on HN
Print view View original

421–430 of 568 posts

Re: You'll regret using natural keys

#421

Post by what »

kstenerud wrote: post #316
biztos wrote: post #231

Earlier quoted context omitted.

I think that 0 and 1 are likely to cause problems when customers end up reading their "user ID" back to your employees in Customer Support Country over the phone. "It's one-three-oh-dee-ee-el. Yes, I'm sure, EL as in elephant."

I developed safe32 for this reason. https://github.com/kstenerud/safe-encoding/blob/master/safe3... Notably, confusable characters are interchangeable when being ingested (although a machine encoder MUST always produce canonical output). https://github.com/kstenerud/safe-encoding/blob/master/safe3... So a user can confuse 1 for l, 0 for o, I for l, u for v, uppercase, lowercase etc, or the agent can say any of those…

Isn’t this just crockford encoding?

Re: You'll regret using natural keys

#422

Post by jmcphers »

psychoslave wrote: post #401
strnisa wrote: post #151

I've become a fan of unique, relatively short and "human-readable" IDs, such at the ones used by Stripe, e.g. `cus_MJA953cFzEuO1z` for an ID of a customer. Here's a Stripe dev article on the topic: https://dev.to/stripe/designing-apis-for-humans-object-ids-3... If you use JavaScript/TypeScript, you can make them like this: function makeSlug(length: number): string { const validChars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcde…

To my mind, it always felt so saddening that adoption of a truly straightforwardly readable notation for numbers never took of. I mean it’s so easy to do. You can start for example with a single syllable per digit, and for example only target CV syllables. From this there is many possibilities, but for example, let’s consider only a base ten. Starting with vowels order o, i, e, a, u with mnemonic o, i graphically clo…

There is a old system for making numbers pronounceable as words using a mapping from each number to a consonant value. It's typically used to help memorize numbers:

https://en.wikipedia.org/wiki/Mnemonic_major_system

Re: You'll regret using natural keys

#423

Post by JackSlateur »

He is right: natural keys are difficult to handle

I'd just say this is not so right: As it turned out, though, whoever made that piece of software knew what they were doing, because the mechanic just changed the chassis number, and that was that.

Because, yes, everybody can just reprogram your car and change its VIN and stuff.

The manufacturer software have protection against that. But nothing prevents you from using another software. At the end, you can directly program whatever you want into the car : change the VIN, change the odometer etc.

Re: You'll regret using natural keys

#424

Post by shkkmo »

nostrademons wrote: post #419
phkahler wrote: post #376

Earlier quoted context omitted.

>> If a person's CPR number changes because they've changed their gender, you will want a separate table recording a.) the date of the change. The new CPR number is not valid before that time b.) the new gender c.) probably the reason for the CPR number change, since if the policy now is that they can change because of a gender change, there's a decent chance they'll be some other policy in the future that results in…

You need this complexity anyway - if you had surrogate keys, the primary key would be an opaque identifier, and to do anything involving the CPR, you'd need to join against your CPR table (which, again, needs to be 1:many because CPRs themselves are not a 1:1 relationship). The first CPR in this case becomes identical to your surrogate key - it's an opaque ID that you use to reference other tables in the DB - but wit…

> but with the added benefit that for the common case, you don't need any additional lookups. You only need to lookup CPR changes if you don't find the CPR that the user gave you

Doing the CPR lookup ONLY when you don't find a record is really, really stupid. All you need is one bad cache, one database update that go rolled back and suddenly you've lost all recent updates the the user and will only see the bad record with no indication you've failed.

Trying to use force changeable data into being a 'natural key' means that the number of edge cases you have to predict, write around and test for is going to rise significantly for no real benefit.

Re: You'll regret using natural keys

#425

Post by cogman10 »

jordanthoms wrote: post #411
cogman10 wrote: post #298

Earlier quoted context omitted.

A problem with this approach is it's not monotonical. Especially if you want to use this thing as an index in a database, you'll run into problems where you try doing middle insertions frequently, which causes fragmentation. The solution to this problem is making the higher order characters time sorted [1]. You don't need to go all out like uuid, you can have a pretty low resolution. It's more important that new inse…

This is dependent on the database you are using - if it's a key-sharded distributed database, you want to have insertions evenly spread across the key space in order to avoid having all the inserts go into a single shard (which could overload it)

This is the great thing about using random bits for the lower bits. Because you are unlikely to use more than say 2^64 database nodes, any sharding algorithm will have to figure out how to spread a key with 64 bits (or however many bits are in your key) across n nodes.

Because of the random portion of the key, that means you'll get good distribution so long as the distribution algorithm isn't something stupid like relying solely on the highest order bits.

Re: You'll regret using natural keys

#426

Post by duped »

JackSlateur wrote: post #423

He is right: natural keys are difficult to handle I'd just say this is not so right: As it turned out, though, whoever made that piece of software knew what they were doing, because the mechanic just changed the chassis number, and that was that. Because, yes, everybody can just reprogram your car and change its VIN and stuff. The manufacturer software have protection against that. But nothing prevents you from using…

> At the end, you can directly program whatever you want into the car : change the VIN, change the odometer etc.

And you can wind up with an undriveable car this way. Particularly the odometer if you roll it back and try and sell it, your state's DMV will probably reject the title transfer.

Re: You'll regret using natural keys

#427

Post by IggleSniggle »

nostrademons wrote: post #340

There's a better solution for many of the exceptional cases that the author describes: aliases & audit logs. Take for example the Danish CPR number. That's perfectly fine as a natural key; its definition is the first CPR number assigned. If a person's CPR number changes because they've changed their gender, you will want a separate table recording a.) the date of the change. The new CPR number is not valid before tha…

URLs are only really uniform not necessarily unique. The most obvious case is something like `http://localhost/file.txt`, where the resource being located is almost sure to be different on every single "localhost," but this is true of any server. The pointer is neither unique nor has any true guarantee that the resource being pointed to can or should be considered unique for any given context. It is merely unique within its calling context, which, when dealing with URLs, is often anywhere on the web where DNS responds.

Even if you presume that the resource being located is in fact a unique resource in the general case, the "unique resource" may not be unique in the way that you presume. Some URLs at a given location are intended to be idempontent and cache-able, others are not, and many are time-limited forwarders. And there's no guarantee or even expectation that two identical forwarding URL's will resolve to the same location; it may well be network-topography dependent.

Re: You'll regret using natural keys

#428

Post by ajuc »

Kon-Peki wrote: post #30

> how about a personal identification number? In Denmark we have the CPR number, and I understand that the US Social Security Number is vaguely analogous. The US SSN is not guaranteed to be unique, the SSN assigned to a person could change, there is no guarantee that a person with an SSN assigned to them is a US citizen, and there is no guarantee that a US citizen has an SSN - they must be requested, and you don’t ne…

Yeah, these things are almost never as simple as they are supposed to be.

Poland has PESEL numbers since 70s. It was supposed to be unique, only apply to Polish citizens, never change, and have a checksum digit. Every Polish citizen gets one at 18 when they get their national ID document, and you can request it earlier if you want to.

Turns out there are duplicated PESEL numbers. A LOT of non-Polish citizens have them assigned (mostly Ukrainian refuges but not only). The checksums are sometimes wrong. And some people have several PESEL numbers.

If you used PESEL as database key you're fucked.

The system works perfectly, but it interfaces with external world through computer-human-paper-human-computer interface. And at some point the mistake propagates so far that it becomes the truth assumptions be damned.

Re: You'll regret using natural keys

#429

Post by Karellen »

trevor-e wrote: post #391
nostrademons wrote: post #340

There's a better solution for many of the exceptional cases that the author describes: aliases & audit logs. Take for example the Danish CPR number. That's perfectly fine as a natural key; its definition is the first CPR number assigned. If a person's CPR number changes because they've changed their gender, you will want a separate table recording a.) the date of the change. The new CPR number is not valid before tha…

URLs can be tricky and have plenty of gotchas depending on what you're trying to do. For example, the order of query params is free to change but it's still the same URL. Nothing that can't be worked around with a little normalization.

> For example, the order of query params is free to change but it's still the same URL.

Are you sure that's guaranteed by any spec? I thought an end-point would be free to treat `?a=1&b=2` and `?b=2&a=1` differently. I mean, it would be a nightmarish implementation, but I don't think it would be non-conforming?

Re: You'll regret using natural keys

#430

Post by shkkmo »

mst wrote: post #333
jval43 wrote: post #304

Earlier quoted context omitted.

It's never a single database in the real world. As soon as you integrate something or have an API to something the keys are out there. Unless you add a translation layer, but then you could just as well use surrogate keys directly.

Using a surrogate UUID for communicating with the outside world is often very useful. This is true for an internal PK that's an auto-inc id as well as for natural keys, though. Using a natural PK -inside- your own database can still be a lot more pleasant to work with, even if you don't let it escape.

> Using a natural PK -inside- your own database can still be a lot more pleasant to work with

Until you need to do anything like described above. The advantage of artificial keys is that they have no semantic content. Anything with semantic content carries the risk that the role that semantic content plays in your system can change and cause problems. Having a non-semantic identifier protects you from that.

This is not to say that you should never use a semantic identifier as a key. However, you should always have a non-semantic artificial key as the identifier and use the semantic identifiers only when necessary.

Post reply on HN