Live data from Hacker News

Applying “make invalid states unrepresentable”

kevinmahoney.co.uk

101–110 of 193 posts

Re: Applying “make invalid states unrepresentable”

#101
>I think this happens because of atomistic, object-orientated thinking.

If you think storing a list of date tuples is "OOP thinking", you have no clue what OOP really is. Educate yourself by listening to people who invented it, not Java consultants or FP zealots.

OOP is about interacting with things via interfaces and messages, rather than data. An OOP solution to inconsistencies of this sort would be an interface that either automatically corrects inconsistencies or throws errors when you try to introduce them. The whole point of OOP approach is that you're not locked into a single data representation, so, for example, you can improve how you store data without re-engineering everything in your system that relies on that data.

Re: Applying “make invalid states unrepresentable”

#102
post #49

I like the concept but I’ve seen a fair few examples of where the developers and users clearly had differing opinions about which states are invalid! Dates are a rich vein of examples. Some users will happily consider “25th December” to be a date, without any year, because it might be the name of a folder in which they store their Christmas stuff. More seriously, genealogists or historians may want to record “25th De…

If you only have a day and month like 25th December you should represent it with a type that contains just that information. Java for example has the MonthDay class (https://docs.oracle.com/javase/8/docs/api/java/time/MonthDay...) which can be converted into a local date when the additional information is available. If your users want to refer to that as a date then that should be handled in the UI but should not lead to ambiguity in the internal representation.

Re: Applying “make invalid states unrepresentable”

#103
post #65

So Google's protocol buffers have this feature called "required" fields, which enforce schema in the type system. You should never use it. Never. It's one of those things that sound good until you're a few years into the project. Similar to how you should never be using meaningful IDs as primary keys for objects, always use meaningless fingerprint-like integers. Or how all integers should be signed unless you're dead…

> And how many integers should actually be strings, unless you're dead sure this is a number.

I phrase this as: If it doesn't make sense to do math on it, it's not a number. What does adding one to a customer account number mean? Absolutely nothing -- you get a completely different account number. So it's not a number, but a numeric string.

Re: Applying “make invalid states unrepresentable”

#104
post #65

So Google's protocol buffers have this feature called "required" fields, which enforce schema in the type system. You should never use it. Never. It's one of those things that sound good until you're a few years into the project. Similar to how you should never be using meaningful IDs as primary keys for objects, always use meaningless fingerprint-like integers. Or how all integers should be signed unless you're dead…

That feature's been dropped in proto3, all fields are optional.

Re: Applying “make invalid states unrepresentable”

#105
post #43
post #26

In general I agree that it's nice to make invalid states unrepresentable, but I'm not sure if I agree that this counts as a fundamental "invalid state". There is nothing about contracts which require that you can only have one active at the same time, or that that current one must be open ended. From a practical point of view it might be advantageous if you maintain only a single contract with a customer at all times…

Then the code will be changed. Making up business requirements is the number one reason for instant legacy code. Code is not set in concrete, you can add that flexibility later when it is needed, but making everything overly generic to make it easier to "implement new requirements" only leads to code that is hard to change in my experience. Also don't forget that this is only an example.

The requirements here aren't clear, but I'm guessing the requirement is to model the contracts the company actually has with customers.

The business also tells you that there are never two contracts running at the same time. But are you actually going to believe that? Is this condition really "impossible?"

A vital and necessary factor here is whether the system being designed has complete control of the creation of contracts. This is perhaps taken for granted by the author, but it's too important to leave implicit. You have three choices in a situation like this: make it impossible for contracts to overlap, model it, or don't model it and accept the consequences. Depending on the frequency and the consequences of the assumption being incorrect, maybe it's acceptable not to model it. Maybe not. My point is that you can't assume something is impossible unless you can actually prevent it from happening, and the author should not have sidestepped this part of the analysis (though possibly they meant for it to be understood that contract creation happens through this data model.)

> Also don't forget that this is only an example.

The problem is that this is an example meant to illustrate and justify a rule of thumb, but it's extremely, extremely simple. How often do you deal with requirements that are this simple, this mathematical? Is this really the kind of example you want to build a rule of thumb from?

Realistically, when I hear requirements like this, I assume they're wrong (very common at the beginning of a project) and I get together with the product manager and ideally a domain expert representing the customer (if the product manager isn't too territorial about that being their job) and figure out what the hell the actual requirements are. What if a customer has a contract to rent 10 units of space at $5 and in the middle of that contract needs more 5 more units but the price has gone up $10? Do you tell them they have to cancel the existing contract at $5 and pay $10 for all their units if they want to add some? Or give them the new units at the old price? Or is it okay to represent the same customer by distinct customer records?

I do like the principle of making invalid states unrepresentable, but I would like to add two supplementary principles:

1. Oftentimes what the business tells you about the data they produce is purely aspirational.

"There will never be overlapping contracts," often means, "We swear we're going to stop creating overlapping contracts, and this time we really mean it." You have to follow up with questions like, "How often have we had overlapping contracts in the past? When was the most recent occurrence?" You should even ask, "When do we anticipate signing the next one?" A logically-oriented software developer might expect someone to take offense if you respond to "we don't sign contracts like that" with "Do you have any currently in the pipeline?" but this is a totally normal kind of question to ask.

2. When users give you a rule in their business requirements, they often take it for granted that the software will handle exceptions to the rule gracefully.

They don't necessarily appreciate how bad things can go in software when something "impossible" happens. When they say, "Contracts will never overlap," you have to say, "What should happen when they do?" If you are talking to a mathematician or a programmer this might come off as questioning their competence, but most people will not find it unusual at all or at least will appreciate that the question is motivated by experience rather than disrespect. It's not like a math problem in school; it is legitimate to question the givens.

Re: Applying “make invalid states unrepresentable”

#106
post #92
post #68

This is very much like database normalization, in that it has the benefit of making invalid data impossible, but the drawback of often making queries into the data much more cumbersome and usually also inefficient. As with database normalization, it is a good idea to first do it as much as possible, and then denormalize again until it is fast enough.

I have always has the idea of a database that does the denormalizations you want automatically for you. Essentially, you keep the DB in a normalized state. You define views of the DB that you want. Then the DB keeps those views as tables for you, and the DB does all of the hard work of keeping those view tables consistent with the denormalized data. Essentially the DB does the atomicity, cache-invalidation, and cache…

Check out the Noria project/database from MIT, I think you'd like it :)

https://github.com/mit-pdos/noria

https://corecursive.com/030-rethinking-databases-with-jon-gj...

https://notamonadtutorial.com/interview-with-norias-creator-...

https://pdos.csail.mit.edu/papers/noria:osdi18.pdf

Re: Applying “make invalid states unrepresentable”

#107
post #4

The time period example seems to miss an obvious weakness in the described Time Period 'object' - it's implied that the end date should be >= the start, but if you are representing a time period with ( Date, Date ) then you are still allowing invalid states to be represented - yet this is what the writer is trying to avoid. Likewise, a timeline split into contiguous periods can still represent out-of-order Dates. a T…

I think the example's last visualization is confusing.

The timeline is {date1, date2, date3, date4}. Let's say you have 2 periods, date1 - date3, and date2 - date4. Period 1 can be represented as {date1, date2, date3}. Period 2 can be {date2, date3, date4}.

Am I understanding this correctly?

Re: Applying “make invalid states unrepresentable”

#108
post #105
post #43

Earlier quoted context omitted.

Then the code will be changed. Making up business requirements is the number one reason for instant legacy code. Code is not set in concrete, you can add that flexibility later when it is needed, but making everything overly generic to make it easier to "implement new requirements" only leads to code that is hard to change in my experience. Also don't forget that this is only an example.

The requirements here aren't clear, but I'm guessing the requirement is to model the contracts the company actually has with customers. The business also tells you that there are never two contracts running at the same time. But are you actually going to believe that? Is this condition really "impossible?" A vital and necessary factor here is whether the system being designed has complete control of the creation of c…

I once did weeks of work trying to unpack what my employer meant by the term "customer" - if your customers are large companies with hundreds of legal entities and hundreds of locations across the world things can get pretty complex pretty fast e.g. does "never two contracts running at the same time" mean that you can't have a contract with a subsidiary and another separate subsidiary of the same parent (which might make sense for credit checking purposes)? What about subsidiaries in different countries? What about partly owned subsidiaries...

Re: Applying “make invalid states unrepresentable”

#109
post #26

In general I agree that it's nice to make invalid states unrepresentable, but I'm not sure if I agree that this counts as a fundamental "invalid state". There is nothing about contracts which require that you can only have one active at the same time, or that that current one must be open ended. From a practical point of view it might be advantageous if you maintain only a single contract with a customer at all times…

I think the fundamental problem here is that the table/entity is incredibly badly named. What kind of contract has only three fields?! This isn't a case of YAGNI; no real-world "contract" is this simple.

It appears to actually be some sort of contract_period or contract_duration, and probably has a link to a real "contract" object somewhere that contains the real meat of the concept. But it's hard to tell what's literal and what's the author trying to simplify the example for us.

Re: Applying “make invalid states unrepresentable”

#110
post #99
post #90

This line raised a huge red flag for me: "If the customer doesn’t have a fixed contract, it is assumed they are on a default contract" No. Don't assume, specify. Explicitly. If this is part of your specification, have a DefaultContract entity of some kind somewhere. And don't call this table just "Contracts", make it clear that those exist in addition to or overlay a default contract. It might sound like overkill, bu…

> have a DefaultContract entity of some kind somewhere This is the OO mindset described at the bottom - the odd compulsion to have a reified entity for every concept. A lot of people have missed that the representation you persist doesn't have to match the representation you present. In this case, don't store default contracts, but present them e.g. via a database view.

But nothing of this has to do with OO. This is an argument at the relational level.

And yes, if there's something as fundamental a concept in the business model as a default contract that's in effect when no other contracts overrule it, then IMO it damn well should be represented explicitly in the persisted data model.

I didn't talk about the specific nature of the representation. The important part is that the intent of the data should be explicit - data lives longer than code.

I've seen far too many DB schemas leaning too hard on implicit assumptions and inferring information that lead to hard to understand data models, unnecessary complex (and hard to optimize) data access (no matter the paradigm), and well, lots of errors.

Post reply on HN