Live data from Hacker News

Ask HN: How to avoid over-engineering software design for future use cases?

news.ycombinator.com

151–160 of 258 posts

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#151
post #136

Earlier quoted context omitted.

> In many companies (think FAANG) > The more complicated the design, the more 'foresight', the better. Certainly not in Amazon. Surely there can be exceptions but in general the company has a culture of simplifying stuff anywhere is possible.

You must not have spent long at Amazon. My original comment was 100% based on my experience as a developer at Amazon.

I did, across different roles in two well known teams. As I said, there are exceptions, and the hiring bar has been dropping a lot in the recent years.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#152
Imo it is an organizational thing. Big is good, powerful and important. The more subordinates the more powerful a manager is. Allocated resources (e.g. dev hours) must be spent on something and more often than not must deliver just a little bit short in order to allocate more the next time. Overengineering is rewarded, it creates more tickets to work on and keep the people busy. Then on the other end there are the resume driven folks, who are mutually interested in piling up complexity. Still, invoking Hanlon, it can not be ruled out that some people just do not know better. Sometimes I think KISS is directly at odds with enterprise development.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#153
post #134

Something I've recently realised after having listened to Kevlin Henney talk about software engineering is how much of the existing knowledge we ignore. The early software engineers in the '60s and '70s were discovering pattern after pattern of useful design activities to make software more reliable and modular. Some of this work is really rigorous and well-reasoned. This is knowledge most engineers I've met complete…

Agreed. But something I'd like to add: You can read all the papers you want. At the end of the day, you actually need to practise writing code! Every piece of software involves a different domain, different requirements, etc. I feel like the majority of developers these days just copy and paste stuff from the internet and glue some npm/nuget/maven packages together, brush their hands together and feel like gods. That…

Part of the thinking about it is not making excuses to yourself. If I have to rewtite something, I don't just dismiss it as refactoring, I also ask myself if I could have got it right the first time. And try to estimate how long it will take, because you will improve with practice, and it is part of doing a professional job, even if it is thoroughly broken and abused at your place of work.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#154
post #32

Earlier quoted context omitted.

Also: have a framework in place, which supports worry-free refactoring. Comprehensive Unit/Integration tests, a robust type-system, pick whatever suits your style. It's a lot easier to refactor stuff when you don't have to worry about breaking something hard to debug with a big code change.

Unit tests are absolutely fantastic during refactoring. I once had to rewrite a piece of code where nobody really knew what it did or what it had to do, and the original author just made some guesses about the intention. I started out by writing unit tests for everything, which became my handhold and documentation for what the system originally did. Then I started reorganising the code into a more structured and more…

I'm intrigued what level you write these tests at, because one of the most painful things to me is having to do large refactorings in a codebase with lots of little niggly unit tests, but few integration or functional tests. During a refactoring, you're constantly breaking impelementation details, and many people feel compelled to have test coverage of these details. My favourite systems are ones where use cases are explicitly modeled, or at least you have a well-defined (and tested) service layer. This is often something you'll introduce first on top of a legacy system. In systems built this way, it's far easier modify the guts, knowing you're not breaking anything with business value, but without the overhead of small unit tests that are highly coupled to implementation details.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#156

Earlier quoted context omitted.

>simplifying stuff anywhere is possible https://raw.githubusercontent.com/aws-samples/aws-refarch-wo... Ironically, that's how you are supposed to run Wordpress on AWS.

You seem to be confusing Amazon's internal infrastructure and its development model to how AWS is used by customers.

I would have posted internal slides directly from app architects or even service(-prototyping) dev teams within AWS including similar vibes regarding this discussion, but for obvious reasons that's not a good idea. But whom am I telling that ... That "reference architecture" aimed at customers using their infrastructure regarding a Wordpress installation gets the general idea across, though.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#157

Earlier quoted context omitted.

>simplifying stuff anywhere is possible https://raw.githubusercontent.com/aws-samples/aws-refarch-wo... Ironically, that's how you are supposed to run Wordpress on AWS.

You seem to be confusing Amazon's internal infrastructure and its development model to how AWS is used by customers.

That's the result though, right? Radical internal simplicity forces incidental external complexity.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#158
post #90

Earlier quoted context omitted.

I've reprogrammed myself to only get the dopamine hit when I delete old or unneeded code :-)

Deleting code is one of the most satisfying things in the world. Puff and gone are all the worries, all the bugs, all the maintenance hassle, all the limitations that code imposed on you.

You'd be able to sum it up as different stages of the reward process:

As you learn to code, it becomes increasingly satisfying to solve problems, so you write more code, which provides more satisfaction, but it gets unwieldy quickly and you start to get lost in the spaghetti.

As you learn to abstract, it becomes increasingly satisfying to create abstractions and manipulate them afterwards, allowing you to not get lost thanks to the now beautiful mental map of your code. But at some point in time it leads to excessive abstractions and issues start to pile up, and you can't quite make sense of the map anymore because everything is a little bit too generic and doesn't carry any more meaning, and such abstractions invariably leak through the lasagna. You usually start by writing more abstractions, but this only feeds the loop.

And finally, as you learn to delete code, you get back to simplicity, and the disappearance of the mental and causality weight associated with that deleted code becomes increasingly satisfying. You start inlining code, you start repeating code, only for the repeating patterns to emerge by themselves, you start thinking about names for those, and the names come naturally. Instead of powering through, there is a state of continuous, unstoppable flow based on a loop of implementation and emergence, where a giant Rubik's cube effortlessly solves itself by throwing it in the air and falling into place.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#159
post #134

Something I've recently realised after having listened to Kevlin Henney talk about software engineering is how much of the existing knowledge we ignore. The early software engineers in the '60s and '70s were discovering pattern after pattern of useful design activities to make software more reliable and modular. Some of this work is really rigorous and well-reasoned. This is knowledge most engineers I've met complete…

Agreed. But something I'd like to add: You can read all the papers you want. At the end of the day, you actually need to practise writing code! Every piece of software involves a different domain, different requirements, etc. I feel like the majority of developers these days just copy and paste stuff from the internet and glue some npm/nuget/maven packages together, brush their hands together and feel like gods. That…

> feel like gods

I've been disappointed with the industry for the opposite reason: I don't want to be doing this, but this is what is being asked of me.

Re: Ask HN: How to avoid over-engineering software design for future use cases?

#160
post #84

You should immediately stop designing for future use cases. Use TDD and only write only just enough code for the currently known required functionality. When you get to know more required functionality the tests protect you from breaking existing functionality and you can extend your code to support the new use cases as well. At this point you should make your code just generic enough to support all known use cases w…

TDD has costs. It's expensive and only works on certain types of systems. It also makes exploration of the problem domain extremely costly and makes refactoring a nightmare. I dislike TDD full stop but there are some domains where it's unambiguously a bad idea. It implies: - Behaviour can and should be compartmentalised - Certain types of efficiency are ignorable - Data structures are better off being relatively simp…

The only thing that somewhat makes sense is that during exploration TDD may not always be the most practical. It immediately starts to be extremely practical once the exploratory phase is over.

I kind of feel you live in some kind of alternate universe. None of these sound true to me, at all. The most important misunderstanding that seems to be going on here is the assumption that in TDD it is a given that one is testing single classes and/or methods. This is not the case. In fact, in most cases it is much more beneficial to test a set of classes/methods at the same time in a way that is representative of something that the customer values.

Honestly, I have seen TDD work so well in so many different circumstances that I have started to consider people who do not do not write test for most possible scenarios as being on the not-so-very professional side of things.

Post reply on HN