Live data from Hacker News

Why software projects take longer than you think: a statistical model (2019)

erikbern.com

81–90 of 133 posts

Re: Why software projects take longer than you think: a statistical model (2019)

#81

Earlier quoted context omitted.

The 80-20 rule: the last 20 percent of the work takes 80 percent of the time.

I've hear it as, "the first 90% of the task takes the first 90% of the time, and the last 10% of the task takes the other 90% of the time." :)

Amin

Re: Why software projects take longer than you think: a statistical model (2019)

#82
There are still people who develop software in traditional projects?

As software can easily changed according to changing business needs, software is never really done. So, it's unnecessary to put it in a project.

While you do a factory building project and decide a point at which you are done, you wouldn't do a car manufacturing project.

Software development is more similar to building cars than to building factories. So instead of projects it is better to think of assembly lines (sprints, kanban boards, etc) and throughput (e.g. burndown). The question is how many business changes come in every week and how many can you solve every week.

Measuring and viewing things the right way then also shows how successful you actually are as a development team.

PS: If your company isn't working like this yet, you might consider getting some external help quickly. This is the standard for quite some time now, but admittedly it's hard to find out what the standard is if you rarely get new people added to your team.

Re: Why software projects take longer than you think: a statistical model (2019)

#83

Sometimes it comes down to poorly understood user requirements, necessitating design work in the middle of a sprint , causing a cascade of other issues like new data models, different table relationships, updates to UI screens, and on and on. Agile might pride itself on being able to handle such things, but...it’s not. Nothing is. Poor planning, no matter the methodology, results in slow, cumbersome, error-prone exec…

I don't know where you worked, but have I worked in a place that did a design document, dropped it off on my desk, and left me to work.

Them: "Why didn't you include the functionality to twiddle the foo?" Me: "What? I don't know what you are talking about." Them: "I swear it was in the design document, it's one of the most important features!" But it was not.

Every design document has flaws, and even if they are as accurate as possible, many projects had to be updated after the fact as even the designers themselves did not predict some of the functionality needed. I've found that when developing new things you need to have steady communication with the people who need to use it, to understand what they need and why.

For recreating old the things (CRUD apps and whatnot) I'm sure this is not nearly as important, but I have always wondered why recreating old software with modern tools has usually resulted in everyone just using the old software. I do know in one case I've done this, I was told not to include certain features because no one uses them... only to find our users not updating, because everyone needed those features. Some times even the vendors don't know what the customers want.

Re: Why software projects take longer than you think: a statistical model (2019)

#84
post #57
post #43

Earlier quoted context omitted.

The rule of thumb that I was taught for software delivery is to double it for every layer of management you have to deal with. So if it's just you to a customer, 1 week becomes 2. If it's you, reporting to your manager, to a customer 1 week becomes 4. Although I generally don't apply multipliers this liberally, I do keep in mind how many stakeholders there are that can add uncertainty and friction to a project.

I've noticed that increasing the number of something usually indicates increases complexity. Working with another team takes more time than just keeping the work in one team. Working with two takes longer. Same with things like the number of repos, languages, or services.

That may be an expression of Brooks Law; increasing the number of people involved increases the average error count.

Re: Why software projects take longer than you think: a statistical model (2019)

#86
post #60

Earlier quoted context omitted.

I'm not sure if I would really consider that a legitimate worry. Software developers' expertise is in developing software, not in analyzing statistics to predict the future. Developer estimates are one of the best data points you have to predict how long something will take, but if you think that you can just take them at face value, you are not doing your job as a manager. You need to compare them to past estimates…

> Software developers' expertise is in developing software, not in analyzing statistics to predict the future. Which is a shame and causes so much wasted work. I wish we all got better at this.

Yep. I would claim analyzing statistics to predict the future is an integral component of being a good software engineer.

Re: Why software projects take longer than you think: a statistical model (2019)

#87

I'm not at all experienced on the subject, but, so far, the mindset I liked most is the one presented on "Software Estimation: Demystifying the Black Art": one must be aware about the difference between an estimate and a plan. Estimates are statistical by nature; they are made of "educated guesses" and historical and empirical data available. Estimates must be unbiased. They are unnegotiatable and they are not commit…

Depending on the complexity of the plans and the dependencies between activities, you absolutely can go over a planned date, so long as it's the early start. In critical path method scheduling, you will typically show an "early start" and "late finish" dates for an activity and maintain float to the early or late start of the next (driven by other activities in the DAG and availability of resources assigned to them). I (and others) like to explicitly state contingency factors and/or risks as successor activities that consume this float. They can also be used for Monte Carlo analysis where those activities are randomly increased in duration (usually on some statistical curve as discussed in the article)

What we are really doing is simulating many many permutations of the effects of known risk likelihoods and consequences across the graph. Otherwise in a large program of works we will tell Project Managers or Contractors that they can manage their own float which gives them flexibility to execute.

Another method that simplifies this is PERT[1] method, where you take, optimistic, pessimistic and normal durations to produce an expected duration.

[1]https://en.wikipedia.org/wiki/Program_evaluation_and_review_...

Re: Why software projects take longer than you think: a statistical model (2019)

#88
post #82

There are still people who develop software in traditional projects? As software can easily changed according to changing business needs, software is never really done. So, it's unnecessary to put it in a project. While you do a factory building project and decide a point at which you are done, you wouldn't do a car manufacturing project. Software development is more similar to building cars than to building factorie…

Don't get confused between Projects and Operations. What you are describing is actually akin to operational projects (albeit at the risk of bespoke designing and building each car).

Projects in the traditional sense produce new assets with new value with new benefits meant to be operationalized. You need to segregate associated costs so that you can depreciate the value of the asset once it's operational. Usually the cost is considered an investment and funded by separate finance.

Operational Projects support, extend, enhance existing business assets. This is the assembly line mentality and is more aligned with continuous improvement for which there is usually a fixed budget for the year and is completed during the normal course of business.

Yes there is some grey area but both are necessary.

Re: Why software projects take longer than you think: a statistical model (2019)

#89
post #16

> A reasonable model for the “blowup factor” (actual time divided by estimated time) would be something like a log-normal distribution. Interestingly, we did extensive time tracking on a multi-year in-house software project and collected data comparing the estimated completion time of tickets with their actual time. The software department was under a lot of pressure to improve their forecasting, and were somewhat de…

> Another interesting thing is that you'd expect, by the central limit theorem, that sufficiently large tasks would eventually become normally distributed rather than lognormal, because they're composed of a large number of subtasks. But it turns out that lognormals are a pretty pathological case; sums of n lognormals can continue to look nearly lognormal until n becomes really, really large.

I’d say that the CLT wouldn’t apply here since the sub tasks aren’t independent? E.g. a discovery when doing sub task C could mean a change to already-completed sub tasks A and B

Post reply on HN