Why does it take so long to build software? (2020) • Hacker News Board

Re: Why does it take so long to build software? (2020)

#161

Post by gilbetron » Wed, Jan 27, 2021, 4:00 PM UTC

tom_walters wrote: post #93

I fundamentally disagree with the conclusions of this article. It doesn't take long to "build software" - git was built in a weekend, Facebook a similar timeline. When the environment is right there's no upper bound on the pace of delivery, it's just that modern day dev is, in many places, less about quality of the output (in terms of product/market fit and user value, not code quality) and more about the satisfactio…

Git wasn't built in a weekend: https://www.linuxfoundation.org/blog/2015/04/10-years-of-git...

And git wasn't even built in 10 days, as Linus has said elsewhere. The core structure was, kind of. But there's been a large effort on top of that.

And the idea that Facebook was built in a weekend is just completely wrong. Sure, some basic Facebook-ish app was created over a weekend, sorta.

Re: Why does it take so long to build software? (2020)

#162

Post by bkirkby » Wed, Jan 27, 2021, 4:04 PM UTC

i believe one of the culprits for software taking so long is planning. this is alluded to in marty cagan's excellent book _Inspired_.

first, the caveat: a software dev project will take up as much time as is allotted to it. when you have more than enough time allotted for a project, most people either spend more time in research or allow themselves partial implementation paths of multiple solutions to help decide which one is best. on the other end of the project, every project can be polished forever. so dev projects will take as least as much time as has been allotted to it.

when we go through the planning process, it usually follows something like this:

1. dev estimates how many "points" (which translates into time in most peoples mind) to allot to a task. dev then pads it to account for dev optimism and wanting to make sure they have the chance of meeting the "committment" 2. program/project/product manager pads the number a bit further because they've been burned in the past on dev optimistic estimates and it's better to underpromise and overdeliver 3. dev proceeds on the task with the bloated estimate baked in and the task takes the full time allotted to it. it's baked inefficiency.

part of the problem is the concept of "committment" for estimates. good scrum organizations have changed that term to "prediction" instead of "committment" since this has been such a big problem. but even that isn't enough to solve the political capital lost when you "predict" wrong.

i think that any estimation above "t-shirt size" is going to bake this inefficiency into your process. the solution is less planning and more doing.

Re: Why does it take so long to build software? (2020)

#163

Post by mstipetic » Wed, Jan 27, 2021, 4:10 PM UTC

pm90 wrote: post #149

mstipetic wrote: post #54

Earlier quoted context omitted.

You're just proving my point. As a profession we base our decisions on blog posts and potential fun. I'm all for these technologies, I'm against the fact that regularly as a profession we order multi million dollar digging equipment to dig a small hole. I've literally seen a bored group of devops start building their own cicd solution and give up after a few months of (highly paid) work Not to mention that all these…

It’s becoming pretty clear that you’ve never seen the power of these tools in action. I don’t think I proved your point at all. I very explicitly showed the opposite. I’ve worked at companies that ingest petabytes of data, processing it would just not be possible without hadoop and sister tech (chiefly hdfs). Hundreds of developers working on a common codebase that would not have been possible without modern ci/cd so…

Mate I'm with you, they're extremely powerful tools and it's basically magic what a few good engineers can build with modern tooling.

When you say enterprise engineers that's 80+ percent of our profession. My gripe is only with lack of standards and misusing those tools without understanding them properly, which happens all the time

Re: Why does it take so long to build software? (2020)

#164

Post by pm90 » Wed, Jan 27, 2021, 4:25 PM UTC

mstipetic wrote: post #163

pm90 wrote: post #149

Earlier quoted context omitted.

It’s becoming pretty clear that you’ve never seen the power of these tools in action. I don’t think I proved your point at all. I very explicitly showed the opposite. I’ve worked at companies that ingest petabytes of data, processing it would just not be possible without hadoop and sister tech (chiefly hdfs). Hundreds of developers working on a common codebase that would not have been possible without modern ci/cd so…

Mate I'm with you, they're extremely powerful tools and it's basically magic what a few good engineers can build with modern tooling. When you say enterprise engineers that's 80+ percent of our profession. My gripe is only with lack of standards and misusing those tools without understanding them properly, which happens all the time

Understood. Apologies for the Curt tone in my earlier comment. Cranky before morning coffee.

Re: Why does it take so long to build software? (2020)

#165

Post by lloeki » Wed, Jan 27, 2021, 4:53 PM UTC

xmprt wrote: post #65

TuringTest wrote: post #62

Earlier quoted context omitted.

If that were true, it wouldn't take long to build from scratch a functionally identical clone of an existing application. Yet it does. The conclusion is that the tools to build software from a known specification to working code are sub-optimal.

That's like saying a Ford F150 is functionally identical to a Ford Model A because they are both cars. The specific implementation under the hood requires a lot of work. And very rarely do people ever make functionally identical clones without massive refactoring to allow for extensibility.

Building the first F150 is an act of creation and design. Assembling the second one is finding the same parts and doing the same assembly steps, the thinking is already sorted out.

Building the first Model A means going back to the drawing board because it's not the same. Even at a smaller scope building a F150 with an updated exhaust means going back to the drawing board for that part, and making sure it keeps working correctly with all the other parts.

It turns out that with software, once you have one fully assembled "F150", a second identical one can be created out of thin air (cp / git clone / run compiler / whatevs). So the effort is ~100% on thinking and design.

Now why would you want 2..n F150s? Because you want to have them in the wild doing something useful. That's making them available and distributing them and having more of these as needs increase. That's cp'ing your code onto servers and deploying and running it and making it reachable. You do that with DigitalOcean, Chef, K8S, or whatever AWS service du jour, and hopefully you automate as much of it as possible just like we have car factories with robots to not build these with human hands.

Deploying and spinning up more of that same code so that it keeps up with the scale of whatever it needs to do... that's ops, and that's basically the only part of software that remotely resembles industrial work. The part before that is development, as in "research & development", and that's 100% not "industrial" by any stretch of the imagination.

Thinking that developing software is like building a house or a car and "industrialising" things with waterfall or Kanban on SCRUM and pulling bogus time estimates to completion is the biggest lie ever, just like pulling out the next vaccine or devising a new maths theorem cannot have any reliable time to completion because you're constantly up against unknown unknowns. This field just flat out doesn't work that way.

Re: Why does it take so long to build software? (2020)

#166

Post by lisper » Wed, Jan 27, 2021, 5:20 PM UTC

_ph_ wrote: post #74

lisper wrote: post #28

Earlier quoted context omitted.

I dunno, there's a lot of really high quality stable software out there. It's not all crap. But as someone who has had the luxury of taking time to do things the Right Way let me tell you from firsthand experience: doing things the Right Way is incredibly hard in no small measure because figuring out what you actually want to do is incredibly hard. There have been many times when I thought I was building something fo…

You are right, not all is crap. But too many people are not aware how much is crap and are pretty naive about using libraries. I don't think it is an accident, that e.g. the Java universe has uncountable numbers of libraries and Java projects have unbelievable many dependencies and e.g. Lisp has a percived lack of libraries. There are many reasons for that, but also, that Lisp programmers tend to rely on libraries le…

> Lisp has a percived [sic] lack of libraries

Lisp is also widely perceived as an interpreted language. Willful ignorance does not make reality even if it is widespread.

Re: Why does it take so long to build software? (2020)

#167

Post by lisper » Wed, Jan 27, 2021, 5:22 PM UTC

xupybd wrote: post #79

lisper wrote: post #28

Earlier quoted context omitted.

I dunno, there's a lot of really high quality stable software out there. It's not all crap. But as someone who has had the luxury of taking time to do things the Right Way let me tell you from firsthand experience: doing things the Right Way is incredibly hard in no small measure because figuring out what you actually want to do is incredibly hard. There have been many times when I thought I was building something fo…

Wow, after your comment about having the luxury of taking time I wanted to find out how. I found your website in your bio. Very impressive and very cool to see even huge successes find the time to comment on HN.

Thanks for the kind words. HN is one of the few remaining bastions of sanity in today's on-line world, so yeah, I make time for it.

Re: Why does it take so long to build software? (2020)

#168

Post by jstimpfle » Wed, Jan 27, 2021, 5:39 PM UTC

ozim wrote: post #147

jstimpfle wrote: post #133

Earlier quoted context omitted.

That's an overstatement. You can build impressive MVP and tech demos in a short amount of time, yes. But that requires that the author has already been thinking about the types of problems that come up in this domain for a long long time. Rest assured that Git certainly hasn't been created in a weekend. Also, you can't create a product in this short amount of time. (There are some websites that do almost nothing at a…

I agree, saying Git was created in a weekend is like saying olympic gold medalist in 100m dash won competition in 10 seconds. Linus was working with DVCS software already and was annoyed with it so he had an idea on how to do it, he did not came up with it on Saturday like it never existed and coded it on Sunday. Other part about Facebook - well if you take first version that was just well photos + some description a…

> saying Git was created in a weekend is like saying olympic gold medalist in 100m dash won competition in 10 seconds

Genius!!!

Re: Why does it take so long to build software? (2020)

#169

Post by invalidOrTaken » Wed, Jan 27, 2021, 8:24 PM UTC

username90 wrote: post #115

krisoft wrote: post #76

Earlier quoted context omitted.

I don’t want to sound smartass but building an identical clone of an existing application is very fast and often totally automated. That’s what your compiler chain does.

Change the criteria that you should rewrite it in another framework and the point still stands. You already have a perfect specification in the existing code, no unknowns at all, just implement it. Yet it still takes months or years.

A very interesting post on this topic: https://vvvvalvalval.github.io/posts/2018-07-23-datascript-a...

Re: Why does it take so long to build software? (2020)

#170

Post by forgotmypw17 » Wed, Jan 27, 2021, 9:14 PM UTC

giantDinosaur wrote: post #119

forgotmypw17 wrote: post #109

Earlier quoted context omitted.

It seems to me like that stance does nothing helpful, only lets the developer off the hook of attempting something difficult and annoying. At the same time, it leaves human users who can't change their browser high and dry.

Worry not, in IE terms, 'rapidly' means 'within this decade', so I will unfortunately be supporting the last, not really venerable, version of it for a while. But even just messing around with CSS in old versions is painful: it's not just 'stack of 15 years ago' if you support IE6, it's also leaving out pretty much everything except the basics of text content, lest you spend really horrible amounts of time getting so…

You're right that a lot needs to be left out or put behind feature checks. But I can't say I've spent an inordinate amount of time tweaking for IE6 than most other browsers. I've probably spent the most tweaking work on IE3, Mosaic, and Netscape 2.