Applying “make invalid states unrepresentable” • Hacker News Board

Re: Applying “make invalid states unrepresentable”

#151

Post by masklinn » Mon, Oct 05, 2020, 6:10 PM UTC

maest wrote: post #86

etripe wrote: post #3

This is a good introduction on a conceptual level. I think a large contributor to the problem is story-oriented development, where all that matters in the sprint is "getting it done" and not looking at the broader context. To make unrepresentable states practical, Scott Wlaschin has an excellent write-up here (0). His book (plugged in that article) is also excellent. [0] https://fsharpforfunandprofit.com/posts/design…

Sum types are one of the main things I miss when working in Python. Is anyone aware of any good ways of adding sum types to Python?

The entire point of sum types is that they be statically checked. Without static typing, they don't seem really useful: Erlang doesn't have sum types, and it doesn't really have a use for them until it gets a type system which can leverage them. Instead it models "sum types" as tagged tuples e.g.

    {ok, Ok} | {err, Err}

however Erlang has very good pattern matching. Python… doesn't.

There's a PEP but I stopped following it because the discussion was a mess. And it's apparently now split into 3 different PEP, I don't know whether that's an improvement or not though.

Furthermore Python's dislike of HOFs means you can't really do "monadic" processing as you'd do in, say, smalltalk where your "variants" would really be subtypes with cool higher-order messages. So you're mostly just adding indirections.

Re: Applying “make invalid states unrepresentable”

#152

Post by layoutIfNeeded » Mon, Oct 05, 2020, 6:12 PM UTC

judofyr wrote: post #26

In general I agree that it's nice to make invalid states unrepresentable, but I'm not sure if I agree that this counts as a fundamental "invalid state". There is nothing about contracts which require that you can only have one active at the same time, or that that current one must be open ended. From a practical point of view it might be advantageous if you maintain only a single contract with a customer at all times…

This.

Basically this is what event sourcing tries to solve: it lets you change your state representation to reflect new requirements, because you can always rebuild everything from the event log.

Re: Applying “make invalid states unrepresentable”

#153

Post by bcrosby95 » Mon, Oct 05, 2020, 6:18 PM UTC

judofyr wrote: post #26

In general I agree that it's nice to make invalid states unrepresentable, but I'm not sure if I agree that this counts as a fundamental "invalid state". There is nothing about contracts which require that you can only have one active at the same time, or that that current one must be open ended. From a practical point of view it might be advantageous if you maintain only a single contract with a customer at all times…

Absent business requirements, I would love to see what you think fundamental invalid states for a contract would be. Every property of a contract I can come up with seems like a business requirement.

Re: Applying “make invalid states unrepresentable”

#154

Post by tsimionescu » Mon, Oct 05, 2020, 6:28 PM UTC

goto11 wrote: post #142

cle wrote: post #133

Earlier quoted context omitted.

I've seen this too. And I've also seen the converse--complexity that was added because engineers refused anything but the most myopic designs, using thought-terminating cliches like "YAGNI" or "that's hypothetical". "Never think ahead" is obviously not good advice. There's no silver bullet here--we have to think about how likely future scenarios are, and plan for them based on the business context and needs. Many of…

Under what circumstances did following YAGNI lead to added complexity?

A common pattern is writing code as a series of isolated cases, when taking some time to design the general case would greatly reduce the amount of code. You add a bool parameter to a function to modify one small bit of what it does, then another one, and you add some new return value, and before long, you've got a class with several getters and instance variables represented as code in a single function, with parameters controlling which actual method is run.

Re: Applying “make invalid states unrepresentable”

#155

Post by dodobirdlord » Mon, Oct 05, 2020, 6:50 PM UTC

srtjstjsj wrote: post #143

H8crilA wrote: post #65

So Google's protocol buffers have this feature called "required" fields, which enforce schema in the type system. You should never use it. Never. It's one of those things that sound good until you're a few years into the project. Similar to how you should never be using meaningful IDs as primary keys for objects, always use meaningless fingerprint-like integers. Or how all integers should be signed unless you're dead…

"required" is an example of making an invalid state representable and having it ruin your program.

Yea, a better example of making invalid state unrepresentable in Google’s protocol buffers is to use the “oneof” feature to mark that a set of fields are mutually exclusive. If A, B, and C are mutually exclusive you can put them in a oneof, which also saves space in the binary representation. If in future you discover that A and B but not C needs to be a valid state, you can add a 4th AB option inside of the oneof.

Re: Applying “make invalid states unrepresentable”

#156

Post by tzs » Mon, Oct 05, 2020, 6:51 PM UTC

A large fraction of the comments are about how if you do this and then someday your requirements change you might have to redo your underlying data structures or databases, with the implication being that you should therefore make those as general and flexible as possible.

That reminds me of an interesting point I saw in a book whose title and author escape me. He said one of the reason you encounter so many bad designs in Java programs is that many new Java programmers look at the design of Swing to learn good design.

He wasn't saying that Swing is badly designed--but Swing is a framework/library, not an application. What it takes to be a good framework/library is different than what it takes to be a good application.

If you are writing an inventory management application you can design your tables and data structures and interfaces around things inventory management applications need. If you are writing a medical billing application you can design around what medical billing needs.

If you are writing a framework or library that might be used by inventory management applications and medical billing applications and all the other nearly infinite kinds of applications people will write you need to keep it very general and flexible...but you also have to keep it fast and not too bloated. It's a much harder design problem, with different best practices for what is good design and what is not.

Re: Applying “make invalid states unrepresentable”

#157

Post by dodobirdlord » Mon, Oct 05, 2020, 6:56 PM UTC

srtjstjsj wrote: post #145

trevor-e wrote: post #117

Earlier quoted context omitted.

Can you elaborate more on the "required" fields point? We've been using a similar feature for several years now in APIs at my work and haven't run into any issues, though we do only use it very sparingly for fields that logically can never be missing. At some point a client has to make the call for what they consider essential, so pushing it in the schema makes this less ambiguous from what I've seen. Maybe it's fine…

https://capnproto.org/faq.html#how-do-i-make-a-field-require... Required now means requires forever because people can't migrate safely. But technically you can change a protocol descriptor from required to optional, which is invalid (usually, in a distributed non-transactional system (the common kin) but nothing stops you from doing it. So why not make required forever? Well, do you really want to commit to anything…

Protocol buffers already require you to commit to some things forever, like the type of a field, or whether two fields belong in a oneof together. I’m not saying that “required” was a great feature, but it’s not exactly unique.

Re: Applying “make invalid states unrepresentable”

#158

Post by Terr_ » Mon, Oct 05, 2020, 7:01 PM UTC

isbvhodnvemrwvn wrote: post #60

jl6 wrote: post #49

I like the concept but I’ve seen a fair few examples of where the developers and users clearly had differing opinions about which states are invalid! Dates are a rich vein of examples. Some users will happily consider “25th December” to be a date, without any year, because it might be the name of a folder in which they store their Christmas stuff. More seriously, genealogists or historians may want to record “25th De…

Also in genealogy: - estimated dates - calculated dates (e.g. someone was 30 in 1870, so he was born in "calculated 1840") - unreadable or unavailable months or days (typically recorded as 1980-00-13) - time ranges with all of the above as boundaries e.g. "after 1760-03-00 and before calculated 1800" - plainly incorrect dates, but that's what the document says (1865-02-30) - no dates (some software tries to enforce p…

All plausible and scary. It sounds like a recipe for a system where the different qualities of dates are their own entity, and anybody doing a date-search needs to provide some criteria on the degree of specificity or certainty they require.

For that matter, someone might want to do a text-based search on dates: "This damaged photo shows 196_-_1, which could be at least 20 different months..."

Re: Applying “make invalid states unrepresentable”

#159

Post by yongjik » Mon, Oct 05, 2020, 7:09 PM UTC

H8crilA wrote: post #65

So Google's protocol buffers have this feature called "required" fields, which enforce schema in the type system. You should never use it. Never. It's one of those things that sound good until you're a few years into the project. Similar to how you should never be using meaningful IDs as primary keys for objects, always use meaningless fingerprint-like integers. Or how all integers should be signed unless you're dead…

I may be in the minority, but after happily using protobuf for years, I believe that there's nothing inherently wrong with required fields - instead, what's "wrong" is the protocol buffer API.

Namely, when constructing a protobuf, theoretically, there might be two different ways: (A) first gather all the fields, and then construct the protobuf from these fields; (B) first construct an empty protobuf, and fill in the fields as necessary. The actual protobuf uses (B) - which is convenient in most cases, because when you start constructing a protobuf usually you don't have all the data ready yet.

However, with required fields, this means when you construct the protobuf it starts with all required fields missing - i.e., an invalid state!

I'm not sure what's the best way to fix it, because it would be infeasible to rewrite all the code to gather all the fields and then construct the protobuf - also it will be hugely inefficient in many cases. However, I feel the "no required fields" rule is essentially a null pointer (the "billion dollar mistake") in disguise - the actual problem is that the API doesn't enforce type safety.

Re: Applying “make invalid states unrepresentable”

#160

Post by jpindar » Mon, Oct 05, 2020, 7:14 PM UTC

cle wrote: post #133

philwelch wrote: post #123

Earlier quoted context omitted.

I like to call this, "speculative complexity". I've seen many cases where speculative complexity was added, and persisted for a long time, for reasons that fundamentally mispredicted the way the system would evolve and actually inhibited that evolution.

I've seen this too. And I've also seen the converse--complexity that was added because engineers refused anything but the most myopic designs, using thought-terminating cliches like "YAGNI" or "that's hypothetical". "Never think ahead" is obviously not good advice. There's no silver bullet here--we have to think about how likely future scenarios are, and plan for them based on the business context and needs. Many of…

But then you'd have to have domain knowledge and why would you learn domain knowledge when the ideal career is a new job in a new industry every few months.