Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Are we overcomplicating software development?

news.ycombinator.com

Post reply on HN
Print view View original

261–270 of 378 posts

Re: Ask HN: Are we overcomplicating software development?

#261

Post by ci5er »

dcminter wrote: post #121
RandomOpinion wrote: post #92

Earlier quoted context omitted.

> I can visibly tell the devs walking out of the room spend more time worrying about "velocity" and "organisation of work" than the money making work that needs to be done. It almost feels like the agile process gives them "one more job" of picking the doable things from the list of stuff that needs to be done so they look better than their peers with better velocity. Classic symptom of managers using agile as a (mic…

I was once on a project where an "agile" team, at the behest of their managers, held a sprint "to improve velocity." I kid you not. I will add that I was not on that team. Our scrum master, who was excellent, shielded us entirely from the management madness.

What was the idea behind this? Spending some dedicated time knocking through some accumulated crud? (I think people call it 'technical debt' these days). Is that a bad thing?

Re: Ask HN: Are we overcomplicating software development?

#262

Post by EdHominem »

simonpantzare wrote: post #235
algesten wrote: post #182

Earlier quoted context omitted.

> Relational databases are the right choice 95% of the time, non-relational stores require a really specific use case. Relational databases are great, but I spent large parts of my life as a developer writing layers converting to/from SQL and later ORMs. There's a huge gain in just not translating data. I know Postgres (and others) deal with JSON, but I can't escape the feeling it's a bit shoe horned in there – basic…

Things such as joins, transactions, and means of enforcing data integrity are useful when solving a whole slew of problems. Not to mention the tooling and community you benefit from when you use a common RDBMS. I never found data translation/serialization to be a big pain (just rely on a framework/lib that does it for you). It's a bigger pain to hand-roll joins that would be a one-liner in SQL or deal with issues tha…

I hear this data-integrity thing a lot but I don't run into these problems myself. I think it might be a functional-programming thing. It's much easier and safer to declare your constraints rather than trying to enforce them. If you aren't in a functional language I can see why you'd want to reach out to one but SQL in a separate process is just one of the options.

> Things such as joins, transactions, and means of enforcing data integrity are useful

If the domain needs transactions I'm already speccing them regardless of general usefulness. Yes all that stuff is useful, but all abstractions have a cost which isn't free just because it's hidden in the DB.

For a problem that didn't need transactions, but for which they were useful, why would you automatically want to couple the solution with your storage layer? If you're looking for the ability to express business logic clearly without cluttering it with error handling, for instance, software transaction memory would probably be a better level to work at.

Re: Ask HN: Are we overcomplicating software development?

#263

Post by brilliantcode »

codingdave wrote: post #259
brilliantcode wrote: post #239

Earlier quoted context omitted.

I asked the same question last year and to be honest, building a SPA is tough with just jQuery. It's more of my needs changing, I don't think SPA can be ignored in 2017, the progressive web app and AMP will put a huge dent in the native apps space. I just like to think that I'm developing a mobile app with front-end javascript framework....it's just the tooling and prerequisite knowledge is quite chaotic. Finding the…

I'm not sure why an SPA with jQuery is tough. $.ajax. Send data. Do stuff in a back-end. Return data. Update divs. If it is tough, you might be trying too hard. I'm not trying to be glib... it just sounds like you might be buying into the over-complexity that the original question was talking about.

it starts to get to be a problem when approaching 1k LOC with multiple developers. The benefit is separating the UI layer from the data layer. I didn't think much of this but the complexity is very real and there's a number attached to this slow & fragile & tightly coupled jQuery app.

If you are using jQuery to build a big SPA with multiple devs, you can certainly do it, but it's not enough. At the same time I'm against over engineered frameworks that imposes high cognitive friction on developers. Vue.js and Polymer hits the right notes for me. I feel that Vue.js is a reaction to React & Redux (seems like the trend is to use it when we don't need it). There will be use cases that makes one more favorable then the other but not many shops are employing 1000+ teams working on FB, therefore the premium you pay to mitigate complexity that you don't have results in loss of productivity, is how I like to think about it.

I won't use something because everyone else is. Often, the majority is wrong and easily influenced by marketing and authority (especially if you are good at lighting billions of dollars on fire to build an elusive monopoly). It happens all the time in dev communities but what makes it worse are the complete lack of insight into why you are not FB or Google. We blindly emulate them hoping to be like them.

I say, approach the problem with a blank sheet of paper. Figure out and evaluate what works for you and your team while watching out for marketing myths and other disinformation from the unicorns.

Re: Ask HN: Are we overcomplicating software development?

#264

Post by api »

cs02rm0 wrote: post #210
brwr wrote: post #191

Earlier quoted context omitted.

At the last company I was at, our search microservice was fast (average response was well under 100ms) and it didn't crash once while I was there. At a larger company, this may not be an accomplishment. At a startup, this is the bees knees. Meanwhile, the rest of our codebase (a monolith) crashed every few days for one reason or another. We had an on-call rotation not because that's what you're supposed to do, but be…

I tend to start with a monolithic service. Sooner or later you get a feel for which bits are becoming at least API stable and could run independently. That's when I split them out. Do it too soon and you end up choosing the wrong boundaries and tying yourself up in knots, do it too late and your monolith can become a mess that's difficult to detach the pieces of.

I tend to try to write monolithic services in such a way that they could be broken up into microservices if that were ever desired.

I don't go too far with this, just avoid things like shared static state and other anti-patterns.

Re: Ask HN: Are we overcomplicating software development?

#265

Post by flukus »

tunesmith wrote: post #183
nostrademons wrote: post #172

Earlier quoted context omitted.

You can get that benefit by dividing your system up into libraries with defined, documented, tested APIs. There's no need to introduce all the complexity and failure modes of distributed systems just to force good design. When you need to scale, then you can easily throw your libraries behind an RPC framework and call it microservices, but there's no need to pay that cost until you actually face that problem.

One caveat is that if you need to fix a bug in your library in an API-compatible way, you can't reach into all the codebases that are using your library. You can deploy a new version of the microservice, though.

> you can't reach into all the codebases that are using your library

You can deploy a new version of the dll and applications can pick it up when they restart. Linux will apply security patches this way.

Re: Ask HN: Are we overcomplicating software development?

#266

Post by EdHominem »

No, but your rules don't resonate with me even though I feel the same overall.

1) Not the best language, but not the worst either. There's no excuse except microcontrollers for C these days (even though I still like it) and the fairly decent JVM can't excuse Java. I think people can come up to speed in a new language pretty easily. It's paradigms that are hard to learn, not syntax.

2) Sounds like you don't have devops. That's a solve-it-once sort of problem. And you have to solve it soon enough for some pieces so it shouldn't be put off. You need to be good at it.

3) It certainly can. It is increasing the size of your system considerably - not just the original system, but also the debugging rules for that system plus (as noted) the debugging rules for the debugging rules ad-infinitum. But what do you propose as a solution? Perfectly trained humans on-call? A procedures manual as detailed as the hypothetical code?

4) Well, lack of CI seems insane regardless of what sort of architecture you have. It's a symptom of not understanding the tools.

5) Capitalized anything is always bunk. But if I hear agile as meaning "short-term goals inside long-term goals, and continuous re-evaluation" then it makes perfect sense and has helped as a consultant and in industry.

Re: Ask HN: Are we overcomplicating software development?

#267

Post by FatalBaboon »

the_gastropod wrote: post #184
FatalBaboon wrote: post #149

Earlier quoted context omitted.

I believe, for small shops, the real benefit of microservices is the logic split that forces good design and reduces cognitive load. You reap the scaling benefits way later, if ever.

Nothing about splitting your app into microservices _forces_ a good design. I've never seen microservices with well-defined seams. Every time, knowledge "leaked" between the apps, and any non-trivial change to the app required updating multiple repos, deployment synchronization, etc. Microservices are a tremendous burden that the vast majority of companies will not benefit from.

I did not mean microservice as in "just make it many apps!". I meant as do not share databases and expose everything as APIs.

It helps cognitive load because such apps can be reasoned about without reading code elsewhere.

Re: Ask HN: Are we overcomplicating software development?

#268

Post by sydd »

reacharavindh wrote: post #31
PaulHoule wrote: post #9

Continuous integration is a good thing. Back in the bad old days you'd have three people working on parts of the system for 6 months and plan to snap them together in 2 weeks and it would take more like another 6 months. Agile methods are also useful. If you can't plan 2 weeks of work you can probably not plan 6 months. When agile methods harden into branded processes and where there is no consensus on the ground rul…

> Agile methods are also useful. If you can't plan 2 weeks of work you can probably not plan 6 months. Hmmm. I was just thinking the opposite yesterday. I'm a performance engineer working closely with two teams. One doing Agile and the other basing on wikis and Adhoc in-person whiteboard discussions. I find the non agile team more productive, efficient and dare I say happy. The Agile based team makes me sit in on the…

You are not doing agile.

In a daily scrum you cannot have conversations, its just everyone stating 3 things: What I was working on yesterday, what I will do today, and whether I need help today from someone. For a team of 10 people (a large agile team!) it should not last for more than 15 mins.

I guess other parts are broken too, if they dont even know how to do a standup.

Re: Ask HN: Are we overcomplicating software development?

#269

Post by flukus »

TeMPOraL wrote: post #214
pg314 wrote: post #142

Earlier quoted context omitted.

The implementation of PostgreSQL is complex, no doubt about that. But if you need strong data consistency and durability guarantees, it provides a rock-solid foundation. SQL might take some getting used, but it is also not rocket science. It shouldn't take more than a week's study to master the basics. There is of course a lot of awful SQL code out there, exactly because most programmers don't even know the basics. Y…

> In my opinion dumping an ORM on top is also not the best way to leverage the strengths of an RDBM. I agree. Unfortunately, the way I see people usually using them is pretty bad - you should not let ORM-generated stuff dictate your business model. Database is a database. A storage layer. Business objects will not map 1:1 to ORM objects. Approaches like "let's inherit from ORM class and add business-related methods",…

> Approaches like "let's inherit from ORM class and add business-related methods", in my experience, lead to total disaster.

I don't disagree, in fact I'd go further and say that data and logic should not be coupled, but this is the Active record pattern which is far from the only way to use an ORM, most ORM's won't even support this pattern by default.

Re: Ask HN: Are we overcomplicating software development?

#270

Post by braveo »

OJFord wrote: post #114
SatvikBeri wrote: post #63

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

> Microservices probably reduces the asymptotic cost of scaling but add a huge constant factor. If this were Medium , I'd highlight the hell out of that. That's so true, and so nicely, succinctly put - it ought to be the reply to end every argument about whether microservices are good or bad.

It applies to TDD as well, unfortunately a lot of TDD proponents don't really acknowledge that.
Post reply on HN