Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Are we overcomplicating software development?

news.ycombinator.com

Post reply on HN
Print view View original

191–200 of 378 posts

Re: Ask HN: Are we overcomplicating software development?

#191

Post by brwr »

OJFord wrote: post #114
SatvikBeri wrote: post #63

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

> Microservices probably reduces the asymptotic cost of scaling but add a huge constant factor. If this were Medium , I'd highlight the hell out of that. That's so true, and so nicely, succinctly put - it ought to be the reply to end every argument about whether microservices are good or bad.

At the last company I was at, our search microservice was fast (average response was well under 100ms) and it didn't crash once while I was there. At a larger company, this may not be an accomplishment. At a startup, this is the bees knees.

Meanwhile, the rest of our codebase (a monolith) crashed every few days for one reason or another. We had an on-call rotation not because that's what you're supposed to do, but because we actually needed it.

Now I'm not saying that microservices make sense for everyone. In general, I agree that they are used incorrectly. Microservices are hot and software developers, generally speaking, like to use hot technologies. Yes, moving to a microservice was costly. We had to re-write a lot of code, we had to set up our own servers, and we had to get permission from the guardians that be to do all of this. But, for our use case, and I assume there are other use cases too, the benefits of detaching ourselves from the company's monolithic codebase far outweighed the costs for doing so.

TL;DR No argument is the end to every conversation. Few things are so black and white.

Re: Ask HN: Are we overcomplicating software development?

#193

Post by Animats »

I've used microservices, but on QNX, where you have MsgSend and MsgReceive, which make message passing not much harder than a subroutine call, and not much slower. UNIX/Linux was never designed for interprocess communication. You have to build several more layers before you can talk, and the result is clunky.

If you're crossing a language boundary, it's often better to use interprocess communication than to try to get two languages to play together in the same address space. That tends to create technical debt, because now two disparate systems have to be kept in sync.

Re: Ask HN: Are we overcomplicating software development?

#194

Post by SatvikBeri »

krptos wrote: post #134
SatvikBeri wrote: post #63

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

"putting your logic into types is better" Can you elaborate on that one? Sounds interesting to me.

Recently our team built a data pipeline: a few large inputs, a few large outputs, a lot of processing in between, a lot of parallelization & working w/large datasets needed. Essentially you could view the entire process as writing one very complex function.

We approached this first outlining the procedure and specifying the types involved, then outlining functions from each type to the next. You could essentially think of our types as tables, so our outline was f1: t1: -> t2, f2: t2 -> t3,...,fn: tn -> t_output (although not quite that linear.)

This let us split up work on the different functions across a team of 6 people. Specifying the input and output types was basically enough to make sure the functions were correct most of the time, and baking the interfaces into types enforced by the compiler made it easy to refactor & coordinate on changes when necessary. Feedback when we made an error was generally immediately available, because IntelliJ would highlight the function that produced an output value of the wrong type, or the compiler would catch it.

In contrast, if we had relied primarily on unit tests to check the functions, that would have made coordination more difficult, refactoring harder, and would have required us to either generate or acquire test data to feed through each function. But this architecture let us successfully build out most of the logic even while we had no access to real data & a different team was working on data ingestion.

Re: Ask HN: Are we overcomplicating software development?

#195

Post by pmontra »

Most of the time we're creating complexity when we can avoid it and we're often proud of it.

The problem is that's very difficult to find the right compromise between time, cost, an architecture that can support the growth of the service so either we build something too thin or something too complicated.

Re: Ask HN: Are we overcomplicating software development?

#196

Post by SatvikBeri »

eikenberry wrote: post #185
SatvikBeri wrote: post #63

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

TDD is not about the tests, it is about teaching yourself to code better. About forcing yourself to break up your code into small components with well defined interfaces. Something we all want to do but which is hard in practice without a tool to guide you. TDD is that tool.

In our case, we did that using functional programming & type-driven design (which ironically is also a TDD.)

Re: Ask HN: Are we overcomplicating software development?

#197

Post by brwr »

cookiecaper wrote: post #159
fsloth wrote: post #119

Earlier quoted context omitted.

I can't understand why people don't refer more often on Mr. Kays message. To be bluntly uncharitable and only half kidding, I do understand why consultants don't buy into it. Simpler systems that are less fragile mean less work.

Employees and management don't buy into it for the same reason that consultants don't buy into it. It means less work. Less opportunity to sound smart, seize control, and/or ego stroke. Less variety to break up the work-week's monotony.

Ego is one of the larger problems I've seen over the years. This usually shows, as you mention, with someone trying to sound smart. The irony here is that I've consistently found -- both inside and outside the software industry -- that the smartest people in the room are the ones who can speak about complex topics in a simple way.

Re: Ask HN: Are we overcomplicating software development?

#198

Post by brwr »

BjoernKW wrote: post #28

No, it's not just you and yes, we often do overcomplicate software development. It's been that way long before agile methodology or microservices though. Complexity-for-the-sake-of-complexity EverthingHasToBeAnAbstractClass frameworks have been plaguing the software development business since at least the 1990s and I'm sure there are similar stories from the 80s and 70s. It's hard to find a one-size-fits-all easy met…

> Less code is better.

I'd change this to "Write as little code as necessary, but no less." The problem with "Less code is better" is that some folks use that as justification to write clever one-liners that are difficult for other developers to read. That is not better.

That aside, I agree with everything else you said!

Re: Ask HN: Are we overcomplicating software development?

#199

Post by brwr »

juancampa wrote: post #165
BjoernKW wrote: post #28

No, it's not just you and yes, we often do overcomplicate software development. It's been that way long before agile methodology or microservices though. Complexity-for-the-sake-of-complexity EverthingHasToBeAnAbstractClass frameworks have been plaguing the software development business since at least the 1990s and I'm sure there are similar stories from the 80s and 70s. It's hard to find a one-size-fits-all easy met…

> Favour disposable code over reusable code: Avoid the trap of premature optimisation, both in terms of performance and in terms of software architecture. Some people call it "premature generalization". Relevant C2 page: http://wiki.c2.com/?PrematureGeneralization

The day I learned about premature generalization is that day my speed as a developer jumped by a factor of three. When you're generalizing, it's easy to overlook how much time that takes.

My rule these days is that, in most cases, I shouldn't generalize before the same code exists in three places. Why not two? In my experience, things that are done twice aren't necessarily done three times. But things that are done three times are likely to be done a fourth.

Re: Ask HN: Are we overcomplicating software development?

#200

Post by ajmurmann »

kosinus wrote: post #125
SatvikBeri wrote: post #63

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

One thing that bothers me is the 'relational databases are good enough' statement, that is repeated in other contexts as well. But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history…

I've found that as soon as you change your data model and now need to deal with old data, document stores get just as complicated but require more custom solutions.
Post reply on HN