Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Are we overcomplicating software development?

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 378 posts

Re: Ask HN: Are we overcomplicating software development?

#211

Post by TeMPOraL »

kosinus wrote: post #125
SatvikBeri wrote: post #63

Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing. Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell. Agile makes sense…

One thing that bothers me is the 'relational databases are good enough' statement, that is repeated in other contexts as well. But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history…

Go with SQLite if you can get away with it. It's a library, not an external engine, and databases are stored inside normal files, which makes a lot of things easier if you're building a standalone app (as opposed to server-side software).

SQL is well worth its time to learn. It's a good DSL for relational data. Most programming languages used for regular code are not very convenient with relational data. As for its security issues, this is actually simple - one has to respect SQL as a real programming language with its own syntax and grammar, instead of resorting to idiocies like gluing strings together in an ad-hoc manner.

Re: Ask HN: Are we overcomplicating software development?

#212

Post by DanielBMarkham »

You've thrown together a bunch of buzzwords and asked if we are over complicating things.

Buzzwords can mean freaking anything. I've seen great Agile teams that don't look anything like textbook Agile teams. Microservices can be a total clusterfuck unless you know what the hell you're doing -- and manage complexity. (Sound familiar?) CI/CD/DevOps can be anything from a lifesaver to the end of all life in the known universe.

So yes, we are over complicating software development, but the way we do it isn't through slapping around a few marketing terms. The way we do it is not understanding what our jobs are. Instead, we pick up some term that somebody, somewhere used and run with it.

Then we confuse effort with value. Hey, if DevOps is good, the more we do DevOps, the better we'll be, right? Well -- no. If Agile is good, the more Agile stuff we do the better we'll be, right? Hell no. We love to deep dive in the technical details. If there aren't any technical details, we'll add some!

Software development is too complicated because individual developers veer off the rails and make it too complicated. That's it. That's all there is to it. Throw a complex library at a good dev and they'll ask if we need the entire thing to only use 2 methods. Throw a complex library at a mediocre Dev and they'll spend the next three weeks writing 15 KLOC creating the ultimate system for X, which we don't need right now and may never need.

It has nothing to do with the buzzwords, the tech, or software development in general. It's us.

Re: Ask HN: Are we overcomplicating software development?

#213

Post by crooked-v »

nostrademons wrote: post #190
kornish wrote: post #187

Earlier quoted context omitted.

I mean, you _can_ if you organize your code such that you can. For example, Google's monorepo lets maintainers of a library find all internal usages and fix them. This is one of the benefits Dan Luu notes in http://danluu.com/monorepo/ .

I think he means that you can't force all teams that use your library to recompile and pickup the updated code, while if you deploy it as a service, you recompile and redeploy and everyone talking to your service gets the most up-to-date version. This is a real problem - I recall that Sanjay Ghemawat et al was working on it when I left Google, though I dunno if the solution they came up with is public yet. It's unlik…

> you recompile and redeploy and everyone talking to your service gets the most up-to-date version

Sure, but if you do that in place it will still break stuff that assumes it works like the last version, and if you do a versioned API or the like you still can't force all teams to adopt the new version.

Re: Ask HN: Are we overcomplicating software development?

#214

Post by TeMPOraL »

pg314 wrote: post #142
kosinus wrote: post #125

Earlier quoted context omitted.

One thing that bothers me is the 'relational databases are good enough' statement, that is repeated in other contexts as well. But especially here, where we're talking about reducing complexity, it feels off to me. PostgreSQL and MySQL seem to me like incredibly complex packages. SQL, the language, is not easy to master either; most programmers I meet know mostly basics. On top of that, there's a long ongoing history…

The implementation of PostgreSQL is complex, no doubt about that. But if you need strong data consistency and durability guarantees, it provides a rock-solid foundation. SQL might take some getting used, but it is also not rocket science. It shouldn't take more than a week's study to master the basics. There is of course a lot of awful SQL code out there, exactly because most programmers don't even know the basics. Y…

> In my opinion dumping an ORM on top is also not the best way to leverage the strengths of an RDBM.

I agree. Unfortunately, the way I see people usually using them is pretty bad - you should not let ORM-generated stuff dictate your business model. Database is a database. A storage layer. Business objects will not map 1:1 to ORM objects. Approaches like "let's inherit from ORM class and add business-related methods", in my experience, lead to total disaster. One has to respect the boundary between storage layer and business model layer.

Re: Ask HN: Are we overcomplicating software development?

#216

Post by adamnemecek »

There's this book that I've been mentioning around here called Elements of Programming https://www.amazon.com/Elements-Programming-Alexander-Stepan... that makes exactly this claim, that we are writing too much code.

It proposes how to write C++-ish (it's an extremely minimal subset of C++ proper) code in a mathematical way that makes all your code terse. In this talk, Sean Parent, at that time working on Adobe Photoshop, estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!) if they followed ideas from the book https://www.youtube.com/watch?v=4moyKUHApq4&t=39m30s Another point of his is that the explosion of written code we are seeing isn't sustainable and that so much of this code is algorithms or data structures with overlapping functionalities. As the codebases grow, and these functionalities diverge even further, pulling the reigns in on the chaos becomes gradually impossible. Bjarne Stroustrup (aka the C++ OG) gave this book five stars on Amazon (in what is his one and only Amazon product review lol). https://smile.amazon.com/review/R1MG7U1LR7FK6/

This style might become dominant because it's only really possible in modern successors of C++ such as Swift or Rust that have both "direct" access to memory and type classes/traits/protocols, not so much in C++ itself (unless debugging C++ template errors is your thing).

Re: Ask HN: Are we overcomplicating software development?

#217

Post by RandomOpinion »

hectormalot wrote: post #146
RandomOpinion wrote: post #92

Earlier quoted context omitted.

> I can visibly tell the devs walking out of the room spend more time worrying about "velocity" and "organisation of work" than the money making work that needs to be done. It almost feels like the agile process gives them "one more job" of picking the doable things from the list of stuff that needs to be done so they look better than their peers with better velocity. Classic symptom of managers using agile as a (mic…

I've seen Jira tickets about creating Jira tickets :) In large lines I agree with this comment. Micromanagement of Agile teams is detrimental. Implicitly, the message is that managers should leave their teams to work in peace? The question I have: assume you are that manager. You have 5 agile teams working on 5 client projects. One team seems to get work done much slower than the other teams. What do you do? (And how…

> And how does one actually track progress of an agile team to begin with?

You've hit upon the key question: you want a progress metric that's in-line with productivity. IMO, assessing that comes down to evaluating whether functionality/code delivered (at a high level) per sprint is reasonable, evaluating whether the task breakdown the team is operating against is sensible and whether tasks are being accomplished in a reasonable amount of time relative to their difficulty, the skills of the person doing them, etc. In other words, the evaluation needs to be specific and include the circumstances: Saying "Why did implementing XYZ take longer than one would normally, even taking into account ABC?" is going to result in fixing the real issue whereas saying "Why is your velocity number so low?" is going to result in "fixing" the number.

That, in turn, requires the manager to either possess solid software engineering skills or have access to someone possessing those skills who can make the assessment in their place. And, yes, it's a lot more work. But, as has been amply documented, attempting to manage off a single number (a self-reported number, no less), simply doesn't work.

Re: Ask HN: Are we overcomplicating software development?

#218

Post by cookiecaper »

freehunter wrote: post #201
cookiecaper wrote: post #159

Earlier quoted context omitted.

Employees and management don't buy into it for the same reason that consultants don't buy into it. It means less work. Less opportunity to sound smart, seize control, and/or ego stroke. Less variety to break up the work-week's monotony.

Engineers don't buy into it because it's not cool. Complex systems are cool. It goes back to the phrase "well-oiled machine". Swiss clocks. People standing around a classic car with its hood open. A complex system of things working just perfectly is super cool, and fixing them when they break is a popular pastime.

Yeah, this is what I'm getting at with the variety thing. I think that good talent tempers this tinkering impulse when a potential breakage could imperil production. They learn that as fun as complexity can be in the right context, having to lose a weekend staying awake until 5am on Saturday night/Sunday morning trying to fix something stupid cancels it out.

Having a lab and doing experimental stuff is great, but choosing to stake your company's products on it should be a much weightier consideration. In practice, we see that this weight is apparently not felt by many.

Re: Ask HN: Are we overcomplicating software development?

#219

Post by brilliantcode »

brwr wrote: post #203
brilliantcode wrote: post #179

1997: I created my first website on Netscape Navigator. I was 10. 2007: I created a textbook trading RoR web app. I was 20. 2017: I'm struggling to create my first front-end website on Chrome and I haven't decided on the back-end. I'm 30. The barrier to entry is indeed very high and no signs of slowing. I blame the explosion of low-interest capital from VC's fueling this fracturing.

Building a website doesn't have to be complicated. You build a Rails site 10 years ago. You probably used jQuery, if you used JavaScript at all. Why can't you do the same today? The real problem here is that not enough developers understand that "just because you can, doesn't mean you should." Once more of us get a handle on that, life will be better.

> You build a Rails site 10 years ago. You probably used jQuery, if you used JavaScript at all. Why can't you do the same today?

You can and I do. However, I am thinking of using polymer or vue.js as I think they are a much lighter candidate than react.js & angular.

The power of marketing is underrated in the developer circles.

Re: Ask HN: Are we overcomplicating software development?

#220

Post by devbug »

SatvikBeri wrote: post #196
eikenberry wrote: post #185

Earlier quoted context omitted.

TDD is not about the tests, it is about teaching yourself to code better. About forcing yourself to break up your code into small components with well defined interfaces. Something we all want to do but which is hard in practice without a tool to guide you. TDD is that tool.

In our case, we did that using functional programming & type-driven design (which ironically is also a TDD.)

This this this!

Most often I see TDD used as a couch for a really bad type system.

Post reply on HN