Applying “make invalid states unrepresentable” • Hacker News Board

Re: Applying “make invalid states unrepresentable”

#141

Post by dnautics » Mon, Oct 05, 2020, 5:03 PM UTC

philwelch wrote: post #123

goto11 wrote: post #53

Earlier quoted context omitted.

> Multiple times I've designed systems where I've reduced the representable states to the minimum, and when some requirements change I realize I have to re-design the full system. Yes, if requirements change, you change the design and code to support the new requirements. Compromising the consistency and maintainability of the current design to accommodate a hypothetical future requirement change is a bad trade-off I…

I like to call this, "speculative complexity". I've seen many cases where speculative complexity was added, and persisted for a long time, for reasons that fundamentally mispredicted the way the system would evolve and actually inhibited that evolution.

Correctly judging this tradeoff is what makes the difference between a good architect and a great architect. There are definitely cases where I have put in a bit of effort (a week or so's worth of programming) to make things flexible because due to the requirements I knew they would be necessary in the 9-12 month timeframe (I've also been wrong about architectural decisions). Then when the time came around, it was painless to make the transition.

I suppose if you were cynical, you could claim that if it's painless no one sees how important you are. And then you wind up leaving the company, because they think everything is easy and don't provide you with the autonomy to achieve what you need to make their system work. And then they discover that it's actually hard.

Re: Applying “make invalid states unrepresentable”

#142

Post by goto11 » Mon, Oct 05, 2020, 5:08 PM UTC

cle wrote: post #133

philwelch wrote: post #123

Earlier quoted context omitted.

I like to call this, "speculative complexity". I've seen many cases where speculative complexity was added, and persisted for a long time, for reasons that fundamentally mispredicted the way the system would evolve and actually inhibited that evolution.

I've seen this too. And I've also seen the converse--complexity that was added because engineers refused anything but the most myopic designs, using thought-terminating cliches like "YAGNI" or "that's hypothetical". "Never think ahead" is obviously not good advice. There's no silver bullet here--we have to think about how likely future scenarios are, and plan for them based on the business context and needs. Many of…

Under what circumstances did following YAGNI lead to added complexity?

Re: Applying “make invalid states unrepresentable”

#143

Post by srtjstjsj » Mon, Oct 05, 2020, 5:16 PM UTC

H8crilA wrote: post #65

So Google's protocol buffers have this feature called "required" fields, which enforce schema in the type system. You should never use it. Never. It's one of those things that sound good until you're a few years into the project. Similar to how you should never be using meaningful IDs as primary keys for objects, always use meaningless fingerprint-like integers. Or how all integers should be signed unless you're dead…

"required" is an example of making an invalid state representable and having it ruin your program.

Re: Applying “make invalid states unrepresentable”

#144

Post by bluGill » Mon, Oct 05, 2020, 5:19 PM UTC

pc86 wrote: post #120

chii wrote: post #52

Earlier quoted context omitted.

The YAGNI (you aint gonna need it) principle overrides the DRY principle imo.

This is partly why I've found it helpful to wait until there are at least 3 identical (not nearly identical) implementations of something before trying to make a more generic/abstracted version of it.

Computers have been widespread for between 70 and 30 years (there is reason to debate, but whatever). Nearly everything has been done before: what you are doing isn't fundamentally new. There is lots of opportunity to add minor new features, reliability, or better user interfaces. But the fundamentals of what you are doing isn't new anymore. You can look at your past versions and what competitors do for guidance on what you will probably need next. If you have any broad knowledge of your problem domain you can make reasonable guesses as to what you will need and what you won't need. When replacing a subsystem I know if there will be 100 users of it in the future or if it is a leaf with 1 user - because I know what the old crufty subsystem has (the first case I spend days thinking about the interface, the second I design the interface when I integrate the one subsystem)

Re: Applying “make invalid states unrepresentable”

#145

Post by srtjstjsj » Mon, Oct 05, 2020, 5:23 PM UTC

trevor-e wrote: post #117

H8crilA wrote: post #65

So Google's protocol buffers have this feature called "required" fields, which enforce schema in the type system. You should never use it. Never. It's one of those things that sound good until you're a few years into the project. Similar to how you should never be using meaningful IDs as primary keys for objects, always use meaningless fingerprint-like integers. Or how all integers should be signed unless you're dead…

Can you elaborate more on the "required" fields point? We've been using a similar feature for several years now in APIs at my work and haven't run into any issues, though we do only use it very sparingly for fields that logically can never be missing. At some point a client has to make the call for what they consider essential, so pushing it in the schema makes this less ambiguous from what I've seen. Maybe it's fine…

https://capnproto.org/faq.html#how-do-i-make-a-field-require...

Required now means requires forever because people can't migrate safely. But technically you can change a protocol descriptor from required to optional, which is invalid (usually, in a distributed non-transactional system (the common kin) but nothing stops you from doing it. So why not make required forever? Well, do you really want to commit to anything forever?

Re: Applying “make invalid states unrepresentable”

#146

Post by sakoht » Mon, Oct 05, 2020, 5:27 PM UTC

Presuming if you are 100% sure what "invalid" is: It is possible that being _unable_ to represent a logic error might mean that a logic error is represented as a "valid but incorrect" value, which is even more dangerous.

Take the example of storing a continuous series of date ranges. If I only store the first date of each pair, I can never accidentally have an overlap or gap. But if my code has a logic error that incorrectly calculates a range, being able to represent it could throw an error. If that code error translates to an incorrect break-point instead, I haven't prevented a bug, I've hidden it.

Re: Applying “make invalid states unrepresentable”

#147

Post by jax_the_dog » Mon, Oct 05, 2020, 5:29 PM UTC

goto11 wrote: post #142

cle wrote: post #133

Earlier quoted context omitted.

I've seen this too. And I've also seen the converse--complexity that was added because engineers refused anything but the most myopic designs, using thought-terminating cliches like "YAGNI" or "that's hypothetical". "Never think ahead" is obviously not good advice. There's no silver bullet here--we have to think about how likely future scenarios are, and plan for them based on the business context and needs. Many of…

Under what circumstances did following YAGNI lead to added complexity?

Not OP, but it would add complexity because that method you "weren't going to need" turns out to actually be needed.

Now you have to work around your simplified design because you decided that you didn't need anything more.

Re: Applying “make invalid states unrepresentable”

#148

Post by refactor_master » Mon, Oct 05, 2020, 5:37 PM UTC

goto11 wrote: post #142

cle wrote: post #133

Earlier quoted context omitted.

I've seen this too. And I've also seen the converse--complexity that was added because engineers refused anything but the most myopic designs, using thought-terminating cliches like "YAGNI" or "that's hypothetical". "Never think ahead" is obviously not good advice. There's no silver bullet here--we have to think about how likely future scenarios are, and plan for them based on the business context and needs. Many of…

Under what circumstances did following YAGNI lead to added complexity?

I think if you misinterpret YAGNI as “you don’t need to change this later”. So your becomes rigidly hard coded, instead of having easily configurable variables and arguments. The over-engineered solution (the real YAGNI) was an interface, an object, methods and fields, only serving a very niche purpose with a lot of boilerplate.

Re: Applying “make invalid states unrepresentable”

#149

Post by codeulike » Mon, Oct 05, 2020, 5:41 PM UTC

alch- wrote: post #138

codeulike wrote: post #35

Earlier quoted context omitted.

Dude, this is HN - we're all either software engineers, or people pretending to be software engineers edit: or related hangers-on, like entrepreneurs or "thought leaders"

It was a joke, son.

But jokes aren't allowed here

Re: Applying “make invalid states unrepresentable”

#150

Post by skybrian » Mon, Oct 05, 2020, 5:47 PM UTC

judofyr wrote: post #74

goto11 wrote: post #53

Earlier quoted context omitted.

> Multiple times I've designed systems where I've reduced the representable states to the minimum, and when some requirements change I realize I have to re-design the full system. Yes, if requirements change, you change the design and code to support the new requirements. Compromising the consistency and maintainability of the current design to accommodate a hypothetical future requirement change is a bad trade-off I…

> Yes, if requirements change, you change the design and code to support the new requirements. Code and representation (i.e. schema) are vastly different. In my experience it's takes an order of magnitude longer to change a representation than to change code. Once there are multiple services/tools which works with a representation you typically have to support both the new and the old representation at the same time…

Getting good at data migrations (with tools and processes to do this) can pay off. It's a more general way of preparing for the future than attempting to anticipate specific changes.

On the other hand, some may say YAGNI.