Live data from Hacker News

Ask HN: Are we overcomplicating software development?

news.ycombinator.com

361–370 of 378 posts

Re: Ask HN: Are we overcomplicating software development?

#361
post #318
post #207

Earlier quoted context omitted.

If you need to change your microservice's API in a non-backwards compatible way, you have the exact same problem plus significant operational complexity.

Don't you just create a new one and let the old go obsolete when the "users" switch?

Which is basically what you do for a traditional library as well. Tweak the header so anything being recompiled against it gets a different function signature. Then old apps continue to work, and newly built apps get the fix.

Re: Ask HN: Are we overcomplicating software development?

#362

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

> Relational databases are the right choice 95% of the time, non-relational stores require a really specific use case. Relational databases are great, but I spent large parts of my life as a developer writing layers converting to/from SQL and later ORMs. There's a huge gain in just not translating data. I know Postgres (and others) deal with JSON, but I can't escape the feeling it's a bit shoe horned in there – basic…

You can implement key/value stores in RDMS's too. It only take a few minutes to create a key/value table in most databases, combined with a few minutes in your favorite language to map it to an appropriate get/set routine. I find this particularly useful for variable attributes against another table, especially when its really a "foreign index, key, value" table. That way its still possible to join the values to other parts of the database. This paradigm really lends itself to multiple FK/key/value tables, where each one extends another particular table.

All that said, doing this requires careful thought, and DB normalization when its discovered that there is a 1:1 relationship between rows in a table and a particular key/value table. So, its not something that should be taken to extreme, but I find it aids in quick development, as every time you discover you need to store another piece of data for some edge condition it doesn't require lots of DB normalization. Also, I wouldn't really consider making the "value" field a blob, rather a very limited int or string.

Re: Ask HN: Are we overcomplicating software development?

#363

Earlier quoted context omitted.

I believe, for small shops, the real benefit of microservices is the logic split that forces good design and reduces cognitive load. You reap the scaling benefits way later, if ever.

You can get that benefit by dividing your system up into libraries with defined, documented, tested APIs. There's no need to introduce all the complexity and failure modes of distributed systems just to force good design. When you need to scale, then you can easily throw your libraries behind an RPC framework and call it microservices, but there's no need to pay that cost until you actually face that problem.

Just putting libraries that were never designed to scale up behind RPC usually won't help you scale. These libraries tend to work with mutable, stateful objects and don't have any groundwork in place for partitioning.

That doesn't mean you can't scale up from a monolith (even one without clean interfaces) - every startup growth story is a testament otherwise - but it's never as as easy as strapping an RPC layer over your library.

Re: Ask HN: Are we overcomplicating software development?

#365
post #360

Earlier quoted context omitted.

Nope. That I could have respected and would have made some sense. It was code for "the team will work longer hours and over the weekend so that an arbitrary number is higher."

Hmmm. Weird. That sounds like people trying to engage in "growth hacking". I wonder if someone's bonus was tied to it. If they were really serious about "velocity" (for whatever reason; some are legit), they'd divide by man-hours, not weeks, anyway, and have actuals going back 3+ months (6+ is better) to baseline their sitrep before they started knob-twiddling.

This was just "work long hours to get the project back on track" being communicated as "improve velocity" with about as much success as you'd expect and less understanding than you're projecting.

Re: Ask HN: Are we overcomplicating software development?

#366
It isn't new when I say that it is hard to come up with simple solution.

In most cases people tend to work under pressure, which ends up with problem nicely fitted to tool at hand. You can hardly blame anybody for that. What we are not doing enough is going over "solution" again and again. Solving a problem second time around is always easier.

Re: Ask HN: Are we overcomplicating software development?

#367
post #338

Earlier quoted context omitted.

At the last place I was at. We had a micro serviced monolith. I can't even begin to describe that thing in common engineering terms. (note: it's better than it seems).

In case you're wondering about the downvotes, a micro services monolith sounds like an oxymoron. Could you expand on how the architecture actually looked? What made it a monolith and what made it micro serviced?

Maybe they were referring to a distributed monolith?

Re: Ask HN: Are we overcomplicating software development?

#368
post #357

Earlier quoted context omitted.

It's a dilemma, because "it takes one to know one". While a few smart people in the workplace may be able to appreciate a brilliant dilution of an extremely complex topic into something approachable, most will not understand the starting complexity and just assume it's an approachable topic. This is fine and everything, but it's bad self-promotion. If you want your bosses to give you a raise, you need them to think t…

Well, it depends. I personally don't feel the need to self-promote to get a raise. I'll probably lose out on a few raises or promotions because of that, but I make a good amount of money and I'm good at what I do. That's enough for me.

This is a fine position to take, but it demonstrates one of our pervasive social problems. People with the humility, modesty, and judgment to make good decisions are frequently passed over because they don't feel the need to lead people along or "prove" their value, whereas clowns frequently realize they have nothing except the show and actively work to manipulate human biases in their favor so that they'll continue to climb the ladder. This works very well. The end result is that good people end up hamstrung by incompetent-at-best managers, and they can take down the ship.

The dilemma re-emerges as one asks himself whether it is right to sit by and allow the dangerously incompetent to ascend based on mind games.

My answer used to be "Yeah, I'll just go to a place where that doesn't happen". I no longer believe such places exist.

Re: Ask HN: Are we overcomplicating software development?

#369

Earlier quoted context omitted.

One thing that bothers me is the 'relational databases are good enough' statement, that is repeated in other contexts as well. But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history…

>But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history of security malpractice. PostgreSQL and MySQL are very complex, but the complexity is entirely contained. Both are well-teste…

Minor aside; I don't think the complex bits of sql are things like cross apply - that's almost no different from a join, especially if you're from a non-sql background where "joins" are typically statements+loop-equivalents and typically hierarchical and ordered. If people have difficulty with cross apply, they're just not trying.

Of course, if you regard sql as something you'd rather not "waste" time on, of course you're going to find those kind of subtle distinctions confusing - sort of like how people think css is difficult.

The more reasonably "complex" bits are the update visibility semantics, i.e. which transaction isolation levels mean what in various scenarios.

That's really complex, and it's truly somewhat unique to sql in that most alternatives simply don't bother trying to solve those problems at all - that can be a bad thing, but it is simpler.

Re: Ask HN: Are we overcomplicating software development?

#370
1) Choosing JavaScript for a Math heavy project would likely be a mistake. There are plenty of other examples of picking the wrong language for the wrong job. That's where this statement falls apart.

2) Depending on how you bring them all together, yes this can be true. If you have something like AWS API Gateway, then microservices may be manageable. If you're rolling your own custom solution with something like nginx or haproxy, you're probably wasting a ton of cycles.

3) Again, I tend to agree with this. Premature optimization seems to be the norm these days. Especially when you get devops people involved. Do we need every single layer in our stack to be "highly available" if we have zero users? The answer is NO.

4) Well, this sounds clever, but I'm not sure it really means anything. Setting up something like Jenkins to watch your GitHub repos and build the branch and run the tests can alert you to issues early and really isn't that difficult to setup.

5) Nothing wrong with TDD as long as you don't go overboard. Nothing wrong with standups, planning or retros. Nothing wrong with short sprints.

Post reply on HN