Live data from Hacker News

Ask HN: Are we overcomplicating software development?

news.ycombinator.com

161–170 of 378 posts

Re: Ask HN: Are we overcomplicating software development?

#161
post #30

I've seen the addition of unit testing is a big cause of complexity. Previously simple classes now have to be more abstracted in order to unit test. Add mocks, testing classes & test frameworks. Some unit tests are handy, but I dont think it justifies the additional complexity. For the apps I write I'd like to see more emphasis on automated integration testing and fewer unit tests - so we can write simple classes aga…

The "threat" of having to add unit tests should force developers to write their classes and components in a way that is easy to reason about. In particular: * put as much functionality into pure functions * depend less on statically-linked globals * import all significant collaborators across seams that can be mocked. * keep state in a small atom, rather than strewn about

If you write code like that, you get many of the benefits of unit tests, whether or not they are actually written.

Perhaps it's a good idea to write a test harness (e.g. larger integration tests) for old so that you have a reasonable chance of catching it if it becomes broken, and focus on writing new code in a testable fashion.

Re: Ask HN: Are we overcomplicating software development?

#163
No it's not just you. In general, "follow latest trends blindly" has never been a winning strategy in software development at any point in computing history. Now, that is not to say that you never change your tools or methodologies once you've mastered your existing tools. But the new tools/techs need to pass a very high bar before you subject your team to these.

Re: Ask HN: Are we overcomplicating software development?

#164
post #114

Earlier quoted context omitted.

> Microservices probably reduces the asymptotic cost of scaling but add a huge constant factor. If this were Medium , I'd highlight the hell out of that. That's so true, and so nicely, succinctly put - it ought to be the reply to end every argument about whether microservices are good or bad.

I believe, for small shops, the real benefit of microservices is the logic split that forces good design and reduces cognitive load. You reap the scaling benefits way later, if ever.

> forces good design and reduces cognitive load

Except splitting into microservices is an unnecessarily complex design choice. That's almost always worse, and the cognitive load comes in when you now need to figure out how to get this stuff right. The scaling benefits also require that you get it right, small flaws in your system become massive issues.

Re: Ask HN: Are we overcomplicating software development?

#165

No, it's not just you and yes, we often do overcomplicate software development. It's been that way long before agile methodology or microservices though. Complexity-for-the-sake-of-complexity EverthingHasToBeAnAbstractClass frameworks have been plaguing the software development business since at least the 1990s and I'm sure there are similar stories from the 80s and 70s. It's hard to find a one-size-fits-all easy met…

> Favour disposable code over reusable code: Avoid the trap of premature optimisation, both in terms of performance and in terms of software architecture.

Some people call it "premature generalization". Relevant C2 page: http://wiki.c2.com/?PrematureGeneralization

Re: Ask HN: Are we overcomplicating software development?

#166
post #134

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

"putting your logic into types is better" Can you elaborate on that one? Sounds interesting to me.

Suppose you have some business logic that subtracts the cost of a transaction from an account balance and returns a new account balance. These things are probably integers, but in many languages you don't have to specify that. You write this function, then later your coworker comes across it and passes it a double. You might end up with weird small discrepancies in account balances (or mysterious errors that only happen sometimes) that could be totally prevented at the time your colleague wrote the code via static analysis, if you use put some logic (costs and balances are integers) into the types.

This can be more sophisticated, like "this function requires a sorted list" so lets make a sorted list type, or packaging things up into biz logic types (a cost type that contains an integer instead of just using integers), but you can catch a wide variety of errors with static analysis if you make your code and logic amenable.

Re: Ask HN: Are we overcomplicating software development?

#167

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

One thing that bothers me is the 'relational databases are good enough' statement, that is repeated in other contexts as well. But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history…

Databases are often the only stateful component in a system - statefulness is inherently complex.

Re: Ask HN: Are we overcomplicating software development?

#168
post #12

Much of the problem in the things you mention is that those things are specific solutions that have been confused with goals. I.e., "we're supposed to build microservices" is a horrible idea, as opposed to "given this particular situation a microservice is a great fit". Understanding the possible benefits and drawbacks of any solution is important. It's important in whether or not that solution is selected, but also…

As you've intimated, most people have a very superficial mental model.

Facebook == respected tech brand == someone I should copy. The end.

Guy I know uses Cassandra == developed by hot tech brand Facebook == cool by mental association with Facebook.

Guy I know uses MSSQL or Oracle == developed by crusty old Evil Empire Company that cool people don't want to work for == bad.

Conclusion: We must use "big data" so we can be like the cool people -- err, because we really have some big data.

This doesn't sound like the outcome we'd expect from technical people making these decisions, but we can obviously see that it's what we're getting.

Re: Ask HN: Are we overcomplicating software development?

#169

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

One thing that bothers me is the 'relational databases are good enough' statement, that is repeated in other contexts as well. But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history…

I think you're forgetting about practicality and not reinventing the wheel.

I'm going to use a car analogy here. Modern cars are incredibly complicated machines. They're also generally very reliable, thanks to about a century of development and engineering, and (relatively) inexpensive thanks to economies of scale.

If I want to transport myself to work on weekdays, and transport my girlfriend and maybe another friend on a weekend trip, and carry a bunch of groceries once a week, I can do all of that with a standard 5-person car. It isn't completely optimal for any of those tasks: it has more space than it needs for any of them, especially the weekday commuting. For my daily commute, I don't even reach highway speeds because I live close to work, so the engine is seriously overkill.

So I could have a custom-designed vehicle for each of these use-cases. Each vehicle would be a little less complex than my current car. But that's a lot to maintain, and would surely be far more expensive and less reliable, since each one is a one-off, requiring custom design and engineering, special parts, etc., and not benefiting from the economies of scale and engineering resources that a mass-market car gets. So instead, I just go buy a ready-made car and use it, and it works great and I'm happy.

Is PostgreSQL overkill for a lot of uses? Probably so. But it's designed to be used for all kinds of different tasks, and while it may not be quite as efficient for any of those tasks as some custom-designed solution, it's far more flexible, and it's readily available, plus it's benefitted from an enormous amount of engineering and debugging that a custom-designed solution would not. Things like CouchDB don't have nearly the number of users and amount of development, so while they may make sense for some tasks, the fact that PostgreSQL has more lines of code does not necessarily mean it's less reliable, in fact the opposite is likely true, just like an off-the-lot Honda or Toyota is likely much more reliable than some custom-designed car that someone built in their garage or some high-end limited-production exotic car like a Ferrari or Bentley.

The reason for using an off-the-shelf solution is because it's fast and easy and reliable. It doesn't matter if you're not making use of 80% of the features or capabilities. And software isn't like cars or engines; hard drive space is nearly free, and except for certain applications you're not likely to see a significant downside to just using a standard SQL database versus something more tailor-made. The main problem is cost (like with Oracle), but with PostgreSQL or MySQL this isn't an issue since they're free (and Free). It also helps that they use a standardized query language which makes them much more accessible.

Re: Ask HN: Are we overcomplicating software development?

#170

Earlier quoted context omitted.

> I can visibly tell the devs walking out of the room spend more time worrying about "velocity" and "organisation of work" than the money making work that needs to be done. It almost feels like the agile process gives them "one more job" of picking the doable things from the list of stuff that needs to be done so they look better than their peers with better velocity. Classic symptom of managers using agile as a (mic…

I've seen Jira tickets about creating Jira tickets :) In large lines I agree with this comment. Micromanagement of Agile teams is detrimental. Implicitly, the message is that managers should leave their teams to work in peace? The question I have: assume you are that manager. You have 5 agile teams working on 5 client projects. One team seems to get work done much slower than the other teams. What do you do? (And how…

> I've seen Jira tickets about creating Jira tickets :)

So long as it's one:many.

Post reply on HN