Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Are we overcomplicating software development?

news.ycombinator.com

Post reply on HN
Print view View original

281–290 of 378 posts

Re: Ask HN: Are we overcomplicating software development?

#281

Post by tjbiddle »

brwr wrote: post #198
BjoernKW wrote: post #28

No, it's not just you and yes, we often do overcomplicate software development. It's been that way long before agile methodology or microservices though. Complexity-for-the-sake-of-complexity EverthingHasToBeAnAbstractClass frameworks have been plaguing the software development business since at least the 1990s and I'm sure there are similar stories from the 80s and 70s. It's hard to find a one-size-fits-all easy met…

> Less code is better. I'd change this to "Write as little code as necessary, but no less." The problem with "Less code is better" is that some folks use that as justification to write clever one-liners that are difficult for other developers to read. That is not better. That aside, I agree with everything else you said!

> some folks use that as justification to write clever one-liners that are difficult for other developers to read.

Seriously - Please, if you do this, stop it! You're just slowing everyone else down.

Re: Ask HN: Are we overcomplicating software development?

#282

Post by flukus »

brwr wrote: post #198
BjoernKW wrote: post #28

No, it's not just you and yes, we often do overcomplicate software development. It's been that way long before agile methodology or microservices though. Complexity-for-the-sake-of-complexity EverthingHasToBeAnAbstractClass frameworks have been plaguing the software development business since at least the 1990s and I'm sure there are similar stories from the 80s and 70s. It's hard to find a one-size-fits-all easy met…

> Less code is better. I'd change this to "Write as little code as necessary, but no less." The problem with "Less code is better" is that some folks use that as justification to write clever one-liners that are difficult for other developers to read. That is not better. That aside, I agree with everything else you said!

There's also a tendency to "architect" things so that common things can be reduced to one liners. A common one is to create form generators via attributes. Things always end up being way more complicated this way.

Re: Ask HN: Are we overcomplicating software development?

#283

Post by zeptomu »

pbreit wrote: post #274
worik wrote: post #255

Earlier quoted context omitted.

Exactly. I start with text files and for most purposes I do not bother with anything else. Next is a key/value store. Simple. Relational databases carry large overhead in translating data (as above) and also in design and maintenance of the structure and getting data into and out of them. I spent many years with them, like them a lot, but they are too much complication for most purposes. Even with relational data RDB…

I'm stunned that you're stunned that people generally don't use text files as data stores.

How do you deal with concurrent write access, do you lock the file?

Re: Ask HN: Are we overcomplicating software development?

#284

Post by zeptomu »

SatvikBeri wrote: post #194
krptos wrote: post #134

Earlier quoted context omitted.

"putting your logic into types is better" Can you elaborate on that one? Sounds interesting to me.

Recently our team built a data pipeline: a few large inputs, a few large outputs, a lot of processing in between, a lot of parallelization & working w/large datasets needed. Essentially you could view the entire process as writing one very complex function. We approached this first outlining the procedure and specifying the types involved, then outlining functions from each type to the next. You could essentially thi…

This is interesting, do you mind being more specific - what was the data, how big was it, how long did the functions run?

Assuming you are talking about real types and having something like

  f1 :: t1 -> t2
and

  f2 :: t2 -> t3
you suggest you were able to do

  g :: t1 -> t3
  g = f2 . f1
which works perfectly well, but is sometimes nontrivial to do for more complex functions, in particular if they are not pure (e.g. they do IO as data is too big for memory) and you do some logging and house-keeping in-between and because of runtime behavior that might be hard to predict.

Does f1 consume all input before f2 can run? Is it "streamed-through", like in `sh`, e.g.

  $ find /home/foobar | grep hs$ | xargs wc -l
which is often done as an optimization?

I really like the concept and it works great, but for me it is simpler to apply to smaller constructs and I am still investigating how to apply it to more "business-logic".

Re: Ask HN: Are we overcomplicating software development?

#285

Post by nathanaldensr »

cookiecaper wrote: post #111
BjoernKW wrote: post #28

No, it's not just you and yes, we often do overcomplicate software development. It's been that way long before agile methodology or microservices though. Complexity-for-the-sake-of-complexity EverthingHasToBeAnAbstractClass frameworks have been plaguing the software development business since at least the 1990s and I'm sure there are similar stories from the 80s and 70s. It's hard to find a one-size-fits-all easy met…

I think the tendency to over-engineer is a symptom of retrofitting an assembly-line 9-5 shift onto the creative process of writing code. You sit a guy there 5 days a week for many years. He has to look busy, he has to do something with all of that time. He's not going to get paid if he writes the code in the most simple, concise, and straightforward way possible and then goes home until they're ready to make a new fe…

Thank you for shining a light on the psychological side of this discussion. I like to highlight psychology when I have these discussions with peers because too often technical folks view the world through technology lenses instead of human ones.

Re: Ask HN: Are we overcomplicating software development?

#286

Post by lisivka »

WalterSear wrote: post #227
lisivka wrote: post #156

Earlier quoted context omitted.

Kick project manager out of standup meeting and you will love them.

If that makes your standups better, you need a competent project manager.

You are right, problem is in project manager head, which abuses standup for status updates because of lack of understanding of purpose of standup meeting, or because of lack of discipline (developers may provide infrequent updates in git/jira). It's hard to debug development process by email. However, if developers will do their standups properly, then their development process will fix itself. (Or project manager will fire the most active one. ;-) )

Re: Ask HN: Are we overcomplicating software development?

#287

Post by kmicklas »

> 1) Choose languages that developers are familiar with, not the best tool for the job

This is probably true but also the root cause I think. Enough developers aren't familiar with the right tools and abstractions (modularity, abstraction, purity, reproducability, etc.) that we just keep rehashing the same bad ideas in a never ending stream of new languages and frameworks that push the same decades old ideas.

Re: Ask HN: Are we overcomplicating software development?

#288

Post by cthulhuology »

Honestly, it is probably just you (and your peers).

Quite frankly chances are the team you have sucks at operations, lacks the necessary experience to design complex systems, and probably doesn't do the fundamental engineering to make a reliable software product.

1 - false dichotomy, the best tool is one you have mastered, your team has individuals with 20+ years of development experience on it right? (Probably not)

2 - micro services are supposed to have small areas of concern and small functional domains to minimize operational complexity. Your services are programs that fit on a couple screens right? (Doesn't sound like it)

3 - redundancy's goal is to remove single points of failure, you should be able to kill any process and the system keeps working. (The word critical suggests you have spfs)

4 - CI is a dev tool to avoid merge hell by always be merging. CI is often used by orgs with massive monoliths because of the cost of testing small changes, and too many cook trying to share a pot. Ultimately if you don't have well defined interfaces ci won't save you. (You had well defined published interfaces with versions right?)

5 - agile is a marketing term for consulting services to teach large orgs how to act like small effective teams of experts. (Hint you need a team of self-directed experts with a common vision and freedom to execute it, you got that right?)

Most problems in tech are related to pop culture. Because we discount experience (because experienced developers are "expensive") we get to watch people reinvent existing things poorly. Microservices, soa, agile, ci, these things are older than many devs working today. The industry fads are largely just rebranding of old concepts to sell them to another clueless generation.

Computers are complex systems, networks of computers are complex systems. Complex systems are complex. Some complexity is irreducible, and complex system behavior is more than just a mere aggregation of the parts. People tend to over complicate their solutions when they don't understand their actual problem. They see things they are unfamiliar with as costly and overly complicated (as in your examples above).

Your problem is a culture that doesn't value experience and deep understanding. You and your team will over complicate things because you don't know better yet.

Re: Ask HN: Are we overcomplicating software development?

#289

Post by nathanaldensr »

swift wrote: post #128

I'd like to push back on continuous integration being over-complicated. It's easy to do using off-the-shelf software and it makes life a lot less stressful when you have confidence that your changes are good before landing them in production. It's such a win that I'd set it up even with a 10 person team.

I use TeamCity to automate running my unit tests and generating/publishing NuGet packages to my private NuGet server... and I work alone. It has value even there. :)

Re: Ask HN: Are we overcomplicating software development?

#290

Post by nathanaldensr »

rb808 wrote: post #22

The biggest issue I have is the current fashion for functional languages resulting in mixed style code bases. I've been working on established applications written in Java/C#/Python that have OO, imperative and now functional code all mixed together. If I had it my way we'd choose one or the other but no one can agree which is the best way to write code.

The style takes a backseat to readability and maintainability. Try creating complex object queries in .NET without LINQ; good luck at reading and maintaining that code. I'll take my lambda expressions any day over that, thank you. I remember "the good old days" of C# 1.0 and you'd be crazy to want to go back there.
Post reply on HN